Job description
6–8 years of experience in data engineering.
Robust hands-on experience with Databricks on Azure.
Solid experience with PySpark and large-scale data processing.
Proven experience designing and implementing ETL/ELT pipelines in Azure.
Solid experience with Azure services such as ADLS, Key Vault, etc.
Strong SQL and data warehousing concepts; experience with orchestration tools (e.g., Databricks Jobs) preferred.
Solid experience in performance tuning and optimization of PySpark/Databricks jobs.
Hands-on experience with monitoring and logging for data pipelines (e.g.Databricks workflows logs).
Exposure to data quality frameworks and practices (validation, reconciliation, error handling).
Experience conceptualizing and delivering PoCs and converting them into scalable production-ready solutions.
Willing to work in 2nd shift and Hybrid mode and work from client office.