07 Nov
|
adesso India
|
Bengaluru
07 Nov
adesso India
Bengaluru
Location
Bangalore (IN), Chennai (IN), Cochin (IN), Pune (IN)
YOUR ROLE
Design, build and own scalable data pipelines on Azure using Databricks, PySpark and Spark SQL, from requirements to productive operation.
Implement Delta Lake solutions, including ingestion, transformation and Medallion architecture (Bronze/Silver/Gold).
Define and implement data governance, access control and lineage with Unity Catalog.
Tune Spark jobs, clusters and storage for performance and cost (partitioning, caching, cluster sizing).
Ensure data quality through validation, testing and monitoring, and resolve data issues with root-cause analysis.
Integrate data from databases, APIs, files and streaming platforms.
Work with data analysts, data scientists and architects, and prepare data for business users, including dashboards on Databricks Apps.
Keep code in Git and follow CI/CD and documentation best practices; share your working style through reviews and pairing.
YOUR PROFILE
6–8 years of data engineering experience, including 3–4 years of hands-on Databricks.
Robust Python, PySpark and Spark SQL.
Experience with Azure services: ADLS Gen2, Data Factory, Key Vault, Event Hubs.
Strong command of Delta Lake, Unity Catalog, Lakeflow Jobs (formerly Databricks Workflows) and Lakeflow Declarative Pipelines (formerly Delta Live Tables).
Valuable understanding of data modelling and data warehousing, and proven experience designing a data platform on Azure.
Robust communication skills and the ability to work with international clients (good English).
Bachelor's or Master's degree in Computer Science, IT or a related field.
Databricks Certified Data Engineer Professional
Exposure to data science and ML workflows
📌 Senior Databricks Data Engineer Bengaluru
🏢 adesso India
📍 Bengaluru