Chennai, Tamil Nadu

Job Summary

We are seeking a highly skilled SQL-focused Data Engineer / Developer with strong expertise in Databricks . The ideal candidate will be responsible for designing, optimizing, and managing data pipelines, with a strong focus on query performance tuning, scalability, and efficient data processing .

Key Responsibilities

Key Responsibilities

Develop, optimize, and maintain SQL queries, stored procedures, and data pipelines

Design scalable and efficient data processing solutions using Databricks

Work with large-scale structured and semi-structured datasets

Ensure data quality, integrity, and reliability across systems

Collaborate with data analysts, engineers, and business stakeholders

Support data migration, transformation, and integration initiatives

Performance Tuning & Optimization (Mandatory Focus)

Optimize SQL queries for performance using:

Indexing strategies (clustered/non-clustered)

Query execution plan analysis

Partitioning and data distribution techniques

Improve data processing efficiency in Databricks:

Spark job optimization (partitioning, caching, broadcast joins)

Delta Lake optimizations (Z-ordering, vacuum, optimize commands)

Identify and resolve performance bottlenecks in pipelines and queries

Reduce query runtime and resource utilization

Additional Skills (Preferred)

Experience with cloud platforms (Azure preferred)





Knowledge of ETL tools and pipelines

Familiarity with Python / PySpark

Exposure to CI/CD pipelines and version control (Git)

Understanding of big data frameworks and distributed systems

Skill Requirements

Secondary Skills (Positive to Have)

1. Databricks

Hands-on experience with Azure Databricks / Databricks Lakehouse platform

Strong knowledge of:

Apache Spark (SQL & PySpark)

Delta Lake architecture

Experience building:

ETL/ELT data pipelines

Data transformations using Spark

Familiarity with:

Notebooks, clusters, jobs, and workflows

Understanding of data lake and medallion architecture (Bronze, Silver, Gold layers)

Other Requirements

Secondary Skills (Good to Have)

1. Databricks

Hands-on experience with Azure Databricks / Databricks Lakehouse platform

Strong knowledge of:

Apache Spark (SQL & PySpark)

Delta Lake architecture

Experience building:

ETL/ELT data pipelines

Data transformations using Spark

Familiarity with:

Notebooks, clusters, jobs, and workflows

Understanding of data lake and medallion architecture (Bronze, Silver, Gold layers)

#body.unify div.unify-button-container .unify-apply-now: focus, #body.unify div.unify-button-container .unify-apply-#body.unify div.unify-button-container .unify-apply-now: focus, #body.unify div.unify-button-container .unify-apply-

📌 Azure Data Technical Specialist (Chennai)
🏢 HCLTech
📍 Chennai