Job Details

Data Engineer (Data Retention Project)

Bengaluru

5 Days Onsite

Shift Timing - 02:00 PM to 11:00 PM

Role & Responsibilities:
Design and maintain scalable data pipelines for data ingestion, transformation, and processing.
Build and optimize ETL/ELT workflows using PySpark, Databricks, SQL, and Airflow.
Develop data models and solutions for retention, reporting, and analytics needs.
Ensure data quality, integrity, and reliability through validation and monitoring frameworks.
Integrate data from multiple sources and optimize processing performance.
Collaborate with cross-functional teams to deliver data solutions and resolve data issues.
Maintain technical documentation, data lineage, and mapping artifacts.
Support production deployments, troubleshooting, and continuous improvement through engineering best practices.

Technical Skills Requirements:
PySpark & Spark SQL:



Hands-on experience writing production-grade PySpark code
Databricks: Proven experience working within Databricks environments, including cluster configuration and Delta Lake
Apache Airflow: Robust experience building and managing pipelines in Apache Airflow
SQL Mastery: Advanced SQL skills (window functions, CTEs, query optimization, indexing strategies)
Data Modeling: Solid understanding of contemporary data warehousing concepts
Version Control: Proficiency with Git and CI/CD deployment workflows
AWS: Experience in Amazon Web Services (AWS) cloud infrastructure
ETL/ELT: Proven track record of designing, building, and optimizing scalable ETL/ELT pipelines and data platforms

📌 Data Engineer Data Retention Project Bengaluru (India)
🏢 Xforia Technologies
📍 India