Role & responsibilities

Roles & Responsibilities

- Design, develop, and maintain scalable ETL/ELT pipelines using Python, PySpark, and Azure Databricks.
- Build and orchestrate end-to-end data workflows using Azure Data Factory (ADF).
- Develop, optimize, and maintain SQL queries, stored procedures, and data models to support business and analytical requirements.
- Ingest, transform, and process large volumes of structured and unstructured data from multiple sources.
- Develop, test, troubleshoot, and maintain REST APIs to enable seamless data integration across applications and platforms.
- Build and deploy Azure Functions to support serverless data processing and event-driven workflows.
- Collaborate with data architects, business analysts, and application teams to understand data requirements and deliver scalable solutions.
- Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and scalability.
- Implement data quality, validation, and error-handling mechanisms to ensure data accuracy and integrity.
- Follow best practices for cloud-native development, version control, documentation, and CI/CD in the Azure ecosystem.
- Participate in Agile ceremonies, code reviews,



and technical discussions to ensure high-quality deliverables.

Preferred candidate profile

Preferred Candidate Profile

- 47 years of experience as a Data Engineer with hands-on experience in Azure-based data platforms.
- Solid proficiency in Python, PySpark, and SQL for large-scale data processing.
- Proven experience with Azure Databricks and Azure Data Factory (ADF) for building enterprise-grade data pipelines.
- Good understanding of Azure cloud services and cloud-based data engineering best practices.
- Experience developing and maintaining REST APIs and integrating them into enterprise applications.
- Hands-on experience developing and deploying Azure Functions in serverless architectures.
- Strong knowledge of data modeling, database design, query optimization, and performance tuning.
- Experience working with Git, CI/CD pipelines, and Agile/Scrum development methodologies.
- Excellent analytical, debugging, and problem-solving skills with the ability to troubleshoot complex data issues.
- Strong communication and collaboration skills, with experience working in cross-functional and globally distributed teams.

📌 Data Engineer (Bengaluru)
🏢 Allegis Group
📍 Bengaluru