Role & responsibilities

Roles & Responsibilities
Design, develop, and maintain scalable ETL/ELT pipelines using Python, PySpark, and Azure Databricks.
Build and orchestrate end-to-end data workflows using Azure Data Factory (ADF).
Develop, optimize, and maintain SQL queries, stored procedures, and data models to support business and analytical requirements.
Ingest, transform, and process large volumes of structured and unstructured data from multiple sources.
Develop, test, troubleshoot, and maintain REST APIs to enable seamless data integration across applications and platforms.
Build and deploy Azure Functions to support serverless data processing and event-driven workflows.
Collaborate with data architects, business analysts, and application teams to understand data requirements and deliver scalable solutions.
Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and scalability.
Implement data quality, validation, and error-handling mechanisms to ensure data accuracy and integrity.
Follow best practices for cloud-native development, version control, documentation, and CI/CD in the Azure ecosystem.
Participate in Agile ceremonies, code reviews,



and technical discussions to ensure high-quality deliverables.

Preferred candidate profile

Preferred Candidate Profile
47 years of experience as a Data Engineer with hands-on experience in Azure-based data platforms.
Solid proficiency in Python, PySpark, and SQL for large-scale data processing.
Proven experience with Azure Databricks and Azure Data Factory (ADF) for building enterprise-grade data pipelines.
Valuable understanding of Azure cloud services and cloud-based data engineering best practices.
Experience developing and maintaining REST APIs and integrating them into enterprise applications.
Hands-on experience developing and deploying Azure Functions in serverless architectures.
Solid knowledge of data modeling, database design, query optimization, and performance tuning.
Experience working with Git, CI/CD pipelines, and Agile/Scrum development methodologies.
Excellent analytical, debugging, and problem-solving skills with the ability to troubleshoot complex data issues.
Strong communication and collaboration skills, with experience working in cross-functional and globally distributed teams.

📌 Data Engineer Bengaluru
🏢 Allegis Group
📍 Bengaluru