Experience

3-5 years of hands-on experience in data engineering, with a strong track record of delivering production grade data pipelines.

Experience in the banking or financial services domain is highly preferred.

Experience implementing data solutions on cloud-native platforms is an advantage.

Technical Skills

Solid hands-on proficiency in PySpark for large-scale data processing on Cloudera Data Platform (CDP).

Experience with distributed data processing frameworks on CDP.

Solid understanding in PySpark Coding, debugging and Productionization skills.

Exposure to AI Assisted Coding tools like Claude Code is a huge advantage.

Solid understanding of Data Vault 2.0 and dimensional modelling techniques.

Familiarity with data warehousing concepts, Big Data architectures/ components and technologies (Hive,



Spark, Oozie, Airflow, etc.) is a must.

Experience working with CI/CD pipelines and version control (e.g., Git, Jenkins). Knowledge of data quality frameworks, metadata management, and data governance practices.

Proficiency in Exploratory Data Analysis (EDA) to understand data characteristics and inform pipeline design.

Exposure to ETL and BI reporting tools such as Microsoft Power BI is an advantage.

Experience with Feature engineering for analytical or ML use cases is an added advantage.

Familiarity with GitHub Actions for pipeline automation is a plus.

📌 Data Engineer (Chennai)
🏢 Sphere It Consultants Dwc
📍 Chennai