Job highlights

- Expertise in Python, SQL, PySpark, Spark SQL, Databricks, Delta Lake, Lakehouse and Medallion Architecture, Spark Performance Tuning, Unity Catalog, and CI/CD on cloud platforms

- Design, develop, and optimize scalable data pipelines; build Lakehouse solutions; implement data ingestion, transformations, and quality checks; manage data governance; support deployments and mentor juniors

Mandatory Skills:

- Python, SQL, PySpark and Spark SQL

- Databricks and Delta Lake

- Lakehouse and Medallion Architecture

- Spark Performance Tuning and Troubleshooting

- Unity Catalog and Data Governance

- Data Modeling, Schema Evolution and Incremental Processing

- Batch Processing and Structured Streaming

- CI/CD and Cloud Platforms (Azure/AWS/GCP)

Preferred candidate profile





Key Responsibilities:

- Design, develop, and optimize scalable data pipelines.

- Build Lakehouse solutions using Medallion Architecture.

- Develop and optimize PySpark applications.

- Implement data ingestion, transformations, and data quality checks.

- Manage data access and governance using Unity Catalog.

- Support automated deployments and mentor junior team members as required.

Valuable to Have: Auto Loader, Lakeflow Connect, Lakeflow Declarative Pipelines, Databricks Asset Bundles, Delta Sharing and Lakehouse Federation.

📌 Databricks Data Engineer (Mangalore)
🏢 Novigo
📍 Mangalore