• Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory and Azure Databricks.
• Develop data ingestion and transformation workflows using Python, SQL, and PySpark.
• Work with Apache Spark and PySpark for large-scale distributed data processing.
• Implement batch processing, incremental loading, full loading, and Change Data Capture (CDC) mechanisms.
• Integrate data from multiple sources, including relational databases, APIs, and cloud storage.
• Work with Azure Data Lake Storage Gen2 and Azure Blob Storage.
• Develop and optimize data models, fact tables, dimension tables, and data warehouse structures.
• Perform data cleansing, validation, deduplication, and aggregation.
• Implement and manage workflows using Azure Data Factory and Apache Airflow.
• Optimize SQL queries, Spark jobs, and data pipelines for better performance.
• Work with Azure Synapse Analytics and relational databases.
• Implement error handling, logging, monitoring, and alerting for production pipelines.
• Manage version control and CI/CD pipelines using Git, GitHub, and Azure DevOps.
• Troubleshoot production issues, perform root cause analysis, and provide production support.
• Maintain technical documentation and collaborate with cross-functional teams. Requirements Programming Languages
• Python, SQL, PySpark.
• Pandas, NumPy, OOP concepts, exception handling, and API integration. Big Data Technologies
• Apache Spark and PySpark.
• Spark DataFrames, Spark SQL, transformations, actions, joins, aggregations,



and window functions.
• Partitioning, caching, and performance optimization. Azure Cloud Technologies
• Azure Data Factory (ADF).
• Azure Databricks.
• Azure Data Lake Storage Gen2 (ADLS Gen2).
• Azure Blob Storage.
• Azure Synapse Analytics.
• Azure Key Vault and Azure Monitor. ETL & Data Engineering
• ETL/ELT pipeline development.
• Data ingestion, transformation, and migration.
• Change Data Capture (CDC).
• Incremental loading and full loading.
• Batch and distributed processing. Database Technologies
• Microsoft SQL Server, PostgreSQL, MySQL, and Oracle.
• Complex SQL queries, indexing, stored procedures, and query optimization. Data Modeling & Warehousing
• Dimensional and relational data modeling.
• Fact and dimension tables.
• Data normalization and denormalization.
• Schema design and data warehouse development. Workflow Orchestration & DevOps
• Azure Data Factory and Apache Airflow.
• Git, GitHub, Azure DevOps, and CI/CD pipelines. File Formats
• CSV, JSON, XML, Parquet, and Delta Lake.
• 4–5 years of relevant experience in Data Engineering.
• Strong hands-on experience with Azure Data Factory, Azure Databricks, PySpark, Python, and SQL.
• Experience developing and optimizing scalable data pipelines.
• Good understanding of cloud-based data engineering and data warehousing concepts.
• Robust analytical, debugging, and problem-solving skills.
• Good communication and collaboration skills.

📌 Senior Data Engineer (India)
🏢 InFynd
📍 India