Azure data bricks with fact & dimension table, Spark SQL, Pyspark, autoloader, unity catalogue & file format JSon - Mandatory

Mandate Skills : Hands on experience in Azure Databricks & Medallion Architecture

7 years - 9 years. in

Azure Databricks technologies.

Experience in

Medallion Architecture is mandatory.

- Good understanding of ETL , BI and DW technologies.
- Hands-on experience on

Jason or XML files as sources. • Experience in autoloader for incremental loading and explode options for flattening data • Experience in loading

Delta live tables • Experience in

PySpark or

Spark using data frames.

- Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure.




- Work together with data scientists and analysts to understand the needs for data and create effective data workflows.
- Create and maintain data storage solutions including Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
- Utilizing Azure Data Bricks to create and maintain

ETL

(Extract, Transform, Load) operations.

- Implementing data validation and cleansing procedures will ensure the quality, integrity, and dependability of the data.
- Improve the scalability, efficiency, and cost-effectiveness of data pipelines.
- Monitoring and resolving data pipeline problems to guarantee consistency and availability of the data.
- Positive communication skills – Oral & Written

📌 Data Engineer (Hyderabad)
🏢 Tech Mahindra
📍 Hyderabad