We are seeking a highly skilled Azure Data Engineer to design, build, and maintain scalable data pipelines and cloud-based analytical solutions. You will collaborate with cross-functional teams to transform raw data into actionable insights while ensuring high performance, security, and reliability of our data infrastructure.
Key Responsibilities:
- Design and implement robust ETL/ELT pipelines using Azure Data Factory and Databricks.
- Develop and optimize complex SQL queries and stored procedures for data processing.
- Manage and monitor data storage solutions including Azure Data Lake Storage and Azure SQL Database.
- Ensure data quality, integrity, and security across all cloud-based data platforms.
- Troubleshoot and resolve data pipeline performance bottlenecks and production issues.
Required Skills & Experience:
- Minimum 4 years of experience in data engineering with a focus on Microsoft Azure.
- Proficiency in Python, PySpark, or Scala for data transformation tasks.
- Robust expertise in Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.
- Advanced knowledge of SQL and experience working with NoSQL databases like Cosmos DB.
- Experience with CI/CD pipelines and version control systems such as Git.
Nice to Have:
- Certification in Azure Data Engineer Associate (DP-203).
- Experience with data visualization tools like Power BI.
- Familiarity with infrastructure as code tools such as Terraform or ARM templates.
- Knowledge of stream processing technologies like Azure Stream Analytics or Kafka.
Education:
- Bachelors degree in Computer Science, Information Technology, or a related field.