- Designing and developing end-to-end data pipelines using Azure Data Factory and Databricks, PySpark, SQL, Delta Lake,azure data lake Building and optimizing ETL workflows for ingestion, transformation, and loading of structured and semi-structured data into Azure Synapse Analytics and ADLS.
- Implementing data validation and quality frameworks to ensure data reliability.Writing efficient PySpark and SQL code for complex transformations, aggregations, and performance tuning.
- Managing secure access using Azure Key Vault, role-based permissions, and compliance best practices.
- Working in an Agile/Scrum environment, delivering solutions aligned with business needs, currently as part of Cognizant??s Data Engineering practice.
- Very good judgement skills and ability to shape compelling solutions.
- Very positive collaboration skills and ability to interact with multi-cultural and multi-functional teams spread
Your Profile
- Should have 4+ years of experience in Azure Databricks with strong pyspark experience
- Experience with Azure Data Bricks, Data Factory
- Experience with Azure Data components such as Azure SQL Database, Azure SQL Warehouse, SYNAPSE Analytics
- Experience in Python/Pyspark/Scala/Hive Programming.
- Experience with Azure Databricks/ADB
- Experience with building CI/CD pipelines in Data environments.
Location - Bengaluru,Hyderabad,Chennai,Mumbai