Databricks Engineer

Experience: 8+ Years

Remote

Job Summary

We are looking for an experienced Databricks Engineer to design, develop, and maintain scalable data engineering solutions using Databricks, Apache Spark, and cloud-based data platforms. The ideal candidate should have strong experience in data pipelines, ETL/ELT development, data modeling, and big data processing.

Key Responsibilities

Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.

Develop ETL/ELT workflows using PySpark and Python.

Build and optimize data processing solutions using Delta Lake and Delta tables.

Work with cloud platforms such as Azure, AWS, or Google Cloud.

Implement data ingestion, transformation, validation, and integration processes.

Optimize Spark jobs, cluster configurations, and SQL queries for performance and cost efficiency.

Implement data quality checks, error handling, monitoring, and troubleshooting.

Collaborate with data architects, analysts, and business teams to understand data requirements.

Follow best practices for data security, governance, access control, and documentation.

Use CI/CD and version control tools to deploy data engineering solutions.

Required Skills





Strong hands-on experience with Databricks.

Proficiency in Python, PySpark, and SQL.

Robust knowledge of Apache Spark architecture and distributed data processing.

Experience building ETL/ELT pipelines and data integration workflows.

Hands-on experience with Delta Lake and Lakehouse architecture.

Experience working with cloud data platforms such as Azure, AWS, or GCP.

Knowledge of data warehousing, data modeling, and database concepts.

Experience with Git and CI/CD pipelines.

Strong analytical, debugging, and problem-solving skills.

Good to Have

Experience with Azure Data Factory, AWS Glue, or similar orchestration and integration tools.

Knowledge of Databricks Workflows, Unity Catalog, and Delta Live Tables / Lakeflow Spark Declarative Pipelines.

Experience with streaming technologies such as Kafka or Structured Streaming.

Familiarity with data governance, access controls, and monitoring tools.

Relevant cloud or Databricks certifications.

Education

Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent practical experience.

📌 Databricks Engineer (India)
🏢 Edvensoft Solutions
📍 India