Technical Data Engineer Databricks

Location: Noida

Work Mode: Hybrid

Experience: 5+ Years

Relevant Databricks Experience: 2 to 5 Years

Employment Type: Full-Time

Job Summary

We are looking for a skilled Technical Data Engineer with solid hands-on experience in Databricks, Python, PySpark, SQL, and AWS to design, develop, and maintain scalable data engineering solutions.

The ideal candidate will have experience building production-grade ETL/ELT pipelines, integrating data from databases, Amazon S3, files, and REST APIs, and working with Delta Lake, Unity Catalog, Medallion Architecture, and Databricks Workflows.

Key Responsibilities

- Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, Python, and SQL.
- Integrate data from multiple sources including databases, Amazon S3, files, and REST APIs.
- Build and manage data pipelines using Databricks Unity Catalog.
- Implement business logic, data transformations, and dimensional data models.
- Create, schedule, monitor, troubleshoot, and optimize Databricks Jobs and Workflows.
- Design and manage Delta Lake tables using Bronze, Silver, and Gold layers.
- Implement data quality checks, validations, error handling, logging, and monitoring.
- Optimize Spark workloads for performance, scalability, and reliability.
- Handle batch processing, incremental data loading, and large-volume data processing.
- Collaborate with engineering, business, and cross-functional teams to deliver production-ready data solutions.
- Follow best practices for version control, CI/CD, testing, documentation, and deployment.

Required Technical Skills

- Strong hands-on expertise in Python, PySpark, and Advanced SQL.
- Strong experience with the Databricks Lakehouse Platform.
- Hands-on knowledge of:

- Unity Catalog

- Delta Lake




- Databricks Jobs & Workflows

- Databricks Clusters

- Notebooks

- Repos

- Medallion Architecture
- Experience with REST API integration for data ingestion and data export.
- Strong understanding of ETL/ELT, batch processing, incremental loading, and data transformation.
- Experience with data modeling, including Star Schema, Snowflake Schema, Fact & Dimension tables, and SCD concepts.
- Good understanding of data warehousing concepts and best practices.
- Experience handling structured and semi-structured data such as CSV, JSON, Parquet, and Delta.
- Knowledge of partitioning, file optimization, Spark performance tuning, and query optimization.
- Experience with Git and CI/CD practices.
- Good understanding of data quality, monitoring, troubleshooting, and production support.

Good to Have

- Experience with Databricks Auto Loader.
- Knowledge of Spark Declarative Pipelines.
- Experience with Kafka, Airflow, or dbt.
- Databricks certification.
- Experience working in Agile development environments.

Experience & Qualifications

- 5+ years of overall experience in Data Engineering.
- Minimum 2+ years of hands-on experience with Databricks.
- Strong programming and analytical skills.
- Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field preferred.

What We are Looking For The ideal candidate should be a hands-on Data Engineer who can independently build, optimize, troubleshoot, and maintain production-grade data pipelines on Databricks. Strong practical experience in PySpark + SQL + Databricks + AWS + REST APIs is essential.

Location: Noida

Work Mode: Hybrid

Contact -

Call / What's App Resume

(phone hidden)

Email - [email protected]

Manager Global Talent Acquisition

MOPTRA INFOTECH PVT LTD

📌 Technical Data Engineer (Delhi)
🏢 Moptra Infotech
📍 Delhi