Key Responsibilities
Design, develop, and maintain scalable data pipelines using PySpark and Azure Data Factory.
Work closely with business stakeholders, analysts, and data scientists to understand data requirements and deliver reliable solutions.
Optimize ETL workflows for performance, scalability, and reliability.
Implement best practices for data ingestion, transformation, and integration across multiple sources.
Ensure data quality, governance, and security across the data lifecycle.
Troubleshoot and resolve issues related to data pipelines, storage, and performance.

Required Skills & Qualifications
5+ years of total experience, with 3+ years relevant in PySpark, Azure Data Factory, and Python.

Candidate Location must be India
Robust experience in building large-scale data pipelines and ETL workflows.
Hands-on expertise in PySpark for data processing and transformation.
Proficiency in Azure Data Factory (ADF)



for orchestrating and automating workflows.
Solid understanding of Python for scripting, data handling, and automation.
Solid SQL skills and ability to work with relational and non-relational databases.
Positive knowledge of data warehousing concepts and performance optimization.
Exposure to Azure ecosystem (Data Lake, Databricks, Synapse Analytics, etc.) preferred.
Excellent problem-solving, analytical, and communication skills.

Nice to Have (Optional)
Experience with CI/CD pipelines for data solutions.
Knowledge of data governance, security, and compliance frameworks.
Familiarity with real-time data streaming technologies (Kafka, Event Hubs, etc.).

Additional Details
Cloud Preference: Azure only (AWS experience not required).

📌 Data Engineer Hyderabad
🏢 Career Maker Consultant BPC
📍 Hyderabad