Role: Azure Databricks

Experience Range: 6-10 years

Location: Chennai, Bengaluru ,Hyderabad ,Pune ,Kochi, Kolkata, NCR ,Delhi .

Roles & Responsibilities:

- Handson data engineering experience, including with Spark on Databricks in Azure.
- Strong expertise in Spark Structured Streaming using both Scala/Spark and/or PySpark for near realtime ingestion and transformations.
- Deep experience building and optimizing ELT/ETL pipelines on ADLS Gen2 with Delta Lake and/or Apache Iceberg tables.
- Solid SQL skills for complex data transformations, windowing functions, and performance tuning on large datasets.
- Experience implementing timepartitioning, schema evolution, CDC/SCD patterns, and incremental data processing.
- Handson experience with Azure services such as Event Hubs/Kafka, Azure Data Factory/Databricks Workflows, Key Vault,



and Azure DevOps.
- Solid understanding of data quality, testing, and observability for pipelines (unit tests, data validation, monitoring, and alerting).

Preferred Skills-

- Experience with performance tuning of Spark jobs (partitioning, shuffle optimization, broadcast joins, caching, cluster sizing).
- Familiarity with data governance, catalog, and lineage tools (Unity Catalog, OpenMetadata, Purview, etc.).
- Exposure to ML/analytics consumption patterns (feature tables, curated marts) and BI/reporting tools.
- Experience mentoring junior engineers and contributing to engineering standards and best practices.

📌 Azure Databricks (Hyderabad)
🏢 Tata Consultancy Services
📍 Hyderabad