06 Aug
|
Tata Consultancy Services
|
Chennai
06 Aug
Tata Consultancy Services
Chennai
Dear Professionals
Greetings from Tata consultancy Services,
Experience : 6-8 year
sLocation: Chennai/Bengaluru/Hyderabad/Pun
e
? Job Description: Data Engineer – Databricks (6–8 Years
)
? Role Summa
ryWe are looking for a Data Engineer with strong Databricks expertise to design and build scalable data pipelines on a contemporary Lakehouse architecture. The ideal candidate should have hands-on experience with Apache Spark, Delta Lake, and cloud data platforms, and be capable of delivering robust, production-grade data solution
s.
? Role focus: Pipeline engineering + Lakehouse architecture + performance optimisat
ion? Key Responsibilit
ies•Design, build, and optimize scalable data pipelines using Databricks (batch & streami
ng)•Develop data processing frameworks using PySpark / Spark
SQL•Implement Delta Lake-based data storage ensuring ACID compliance and reliabil
ity•Develop and maintain medallion architecture (Bronze, Silver, Gold) data lay
ers•Perform data ingestion from multiple sources (APIs, Kafka, cloud storage, databas
es)•Build data models for analytics and reporting use ca
ses•Ensure data quality, validation, and reconciliation mechani
sms•Optimise performance using partitioning, caching, and query tun
ing•Implement data governance using Unity Catalog / equivalent to
ols•Integrate pipelines with orchestration tools (Airflow / ADF / Compos
er)•Collaborate with stakeholders for requirements, design, and deliv
ery
? Enterprise alignm
ent:•Databricks roles focus on “designing, building, and managing scalable data pipelines using Spark and Delta La
ke”
? Mandatory Skills (Core Must
-Have)✅ Databricks & Bi
g Data•Strong hands-on Databricks (Azure / GCP)-Primarily GCP pre
ferred•Expertise in Apache Spark (Py
Spark)•Experience with Delta Lake (ACID, schema evol
ution)
✅ Programming & Q
uerying•Strong Pytho
n + SQL
✅ Data Engineering Fund
amentals•ETL/ELT pipeline dev
elopment•Data modelling (dimensional / La
kehouse)•Batch + streaming data pr
ocessing
✅ Cloud Pl
📌 Data Engineer (Chennai)
🏢 Tata Consultancy Services
📍 Chennai