Role & responsibilities

Design, implement, and manage data pipelines on Databricks (PySpark, SQL, Delta Lake) hosted on AWS.

Develop ETL/ELT workflows for structured and unstructured data using Databricks, AWS Glue, Lambda, and Step Functions.

Build and optimize data lakes (S3 + Delta Lake) and data warehouses (Redshift, Databricks SQL).

Enable data democratization via Notebooks, APIs, and self-service analytics.

Implement data governance, quality, and lineage frameworks using AWS Lake Formation, Databricks Unity Catalog, or similar tools.

Collaborate with Data Scientists and Analysts to deliver ML-ready datasets and advanced analytics.

Tune clusters, jobs, and queries for performance and cost-efficiency.

Manage CI/CD pipelines for data engineering workloads (Terraform, GitHub/GitLab CI/CD, Databricks CLI).

Ensure security, compliance, and best practices in data handling, access controls, and monitoring.

Lead troubleshooting and incident management for large-scale data pipelines.

Preferred candidate profile

📌 Senior Databricks Engineer (Bengaluru)
🏢 Svdigisol
📍 Bengaluru