Position: Lead Data Engineer Databricks or Snowflake

Experience: 5-8 years

Location: Hyderabad

Employment Type: Full-time

Role Overview

We are looking for a Lead Data Engineer with robust expertise in Databricks and Snowflake to design, build, and lead modern cloud data platforms. The role involves hands-on engineering, technical leadership, data architecture, and mentoring data engineering teams.

Key Responsibilities

- Lead the design and development of scalable data engineering and analytics platforms using Databricks and Snowflake.
- Build robust ETL/ELT pipelines for batch and streaming data.
- Develop solutions using Databricks, Apache Spark, PySpark, Delta Lake, and SQL.
- Design and optimize Snowflake data warehouses, schemas, queries, and data models.
- Implement Medallion Architecture (Bronze, Silver, Gold) using Delta Lake.
- Develop data ingestion pipelines from APIs, databases, files, and cloud storage.
- Implement data quality, governance, lineage, security, and access controls.
- Optimize Spark jobs and Snowflake workloads for performance and cost.
- Build CI/CD pipelines for data engineering workloads.
- Work with cloud services such as Azure/AWS/GCP, depending on the environment.
- Provide technical direction, conduct code reviews, and mentor engineers.
- Collaborate with architects, product owners, analysts, and business stakeholders.
- Troubleshoot production data pipelines and establish monitoring and alerting.

Required Technical Skills

- Databricks:



Spark, PySpark, Delta Lake, Unity Catalog, Workflows.
- Snowflake: SQL, Snowpipe, Streams & Tasks, Dynamic Tables, Time Travel, performance tuning.
- Strong Python and SQL skills.
- Strong experience with Apache Spark and distributed data processing.
- Experience with ETL/ELT and data warehouse architecture.
- Experience with Azure/AWS/GCP data services.
- CI/CD using tools such as Azure DevOps, GitHub Actions, Jenkins, or GitLab.
- Experience with orchestration tools such as Databricks Workflows, Airflow, Azure Data Factory, or similar.
- Strong understanding of data modeling, dimensional modeling, and data lake/lakehouse architecture.

Leadership Skills

- Lead technical design and architecture discussions.
- Break down complex requirements into engineering solutions.
- Mentor and guide junior/mid-level engineers.
- Establish coding, testing, deployment, and operational standards.
- Communicate effectively with both technical and non-technical stakeholders.
- Drive engineering best practices across the team.

Preferred candidate profile

- Experience migrating legacy data platforms to Databricks/Snowflake.
- Experience with real-time/streaming data using Kafka or similar technologies.
- Knowledge of Terraform/IaC.
- Experience implementing data governance and security.
- Databricks or Snowflake certifications are a plus.
- Experience working in Agile/Scrum environments

📌 Lead Data Engineer (Hyderabad)
🏢 Artha Solutions
📍 Hyderabad