Role & responsibilities
Design, develop, and maintain scalable ETL/ELT pipelines using Databricks.
Build and optimize data workflows using Apache Spark (PySpark/Scala).
Develop and manage data lakes and lakehouse architectures using Delta Lake.
Ingest, transform, and process structured and unstructured data from multiple sources.
Implement data quality, governance, and security best practices.
Optimize Spark jobs and Databricks workloads for performance and cost efficiency.
Collaborate with data architects, analysts, and business stakeholders to deliver data solutions.
Integrate Databricks with cloud-native services across Azure, AWS, or GCP.
Monitor, troubleshoot, and resolve data pipeline and platform issues.
Participate in architecture reviews and provide technical recommendations.
Preferred candidate profile
4+ years of Data Engineering experience.
Hands-on experience with Databricks Platform.
Robust expertise in PySpark/Spark SQL.
Proficiency in Python and SQL.
Experience with Delta Lake, Databricks Workflows, and Unity Catalog.
Experience in ETL/ELT pipeline development.
Knowledge of data modeling concepts.
Experience with Git, CI/CD, and Agile methodologies.
Understanding of data warehousing concepts.