Job Description: Principal / Lead Data Engineer (PySpark, SQL)
Role & Requirements Overview
Exp: 10+ Years
Locations; 100% Remote

Role Overview
We are seeking a highly experienced and motivated Principal / Lead Data Engineer to join our dynamic data platform team in either our Gurgaon or Chennai office. The successful candidate will be a critical player in designing, building, and optimizing our next-generation data architecture and pipelines.

Key Responsibilities
Data Pipeline Development & Optimization
· Design and Build: Architect, develop, and maintain robust, scalable, and fault-tolerant ETL/ELT pipelines for ingesting data from diverse sources (e.g., databases, APIs, streaming sources) into our data lake and data warehouse.
· PySpark Expertise: Write and optimize complex data transformation jobs using PySpark and the Spark DataFrame API to process petabytes of structured and unstructured data efficiently.

Architecture and Infrastructure
· Data Modeling: Design and implement optimal data models (e.g., Dimensional Modeling, Data Vault, Snowflake Schema)



for our data warehouse to support business intelligence and analytical needs.
· Cloud Integration: Drive cloud-native data solutions primarily leveraging Azure Databricks, Azure Data Lake, and Synapse (or comparable frameworks like AWS S3/Redshift and Google BigQuery) to build and deploy data solutions.

Required Technical Skills (Mandatory)
· Certification: Active Azure Databricks Certification (e.g., Databricks Certified Data Engineer Associate/Qualified).
· PySpark: Expert-level, hands-on experience in developing, tuning, and optimizing large-scale data processing applications using PySpark (Python for Apache Spark).
· SQL: Mastery of Advanced SQL (including window functions, complex joins, stored procedures, and query performance tuning) across various database systems (e.g., Snowflake, Redshift, PostgreSQL).
· Programming: Strong proficiency in Python for scripting, automation, and general data manipul

📌 Lead Data Engineer (India)
🏢 solve IT consultant
📍 India