JOB ID:QT-AJH-08-604

Job Summary

We are looking for an experienced Data Engineer with solid expertise in building scalable data pipelines, lakehouse architectures, and data transformation workflows. The ideal candidate must have hands-on experience with Apache Airflow, Data Lake, SQL, Apache Iceberg, dbt, and PySpark.

Mandatory Skills
- Apache Airflow
- Data Lake / Data Lakehouse
- Advanced SQL
- Apache Iceberg
- dbt (Data Build Tool)
- PySpark

Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using Apache Airflow and PySpark.
- Build and optimize data processing workflows on Data Lake/Lakehouse platforms.
- Develop and manage Apache Iceberg tables, including partitioning, schema evolution, and data optimization.
- Write complex and optimized SQL queries for data transformation and analytics.
- Develop modular, reusable data transformation models using dbt.
- Create and maintain Airflow DAGs with proper scheduling, dependency management, retries, monitoring, and alerting.
- Implement data quality, validation,



and reconciliation checks across pipelines.
- Optimize PySpark jobs for performance, scalability, and efficient resource utilization.
- Work with large-scale structured and semi-structured datasets.
- Implement incremental data loading and CDC-based processing where required.
- Collaborate with Data Scientists, Analysts, Architects, and Application teams to understand data requirements.
- Troubleshoot pipeline failures, performance issues, and data inconsistencies.
- Follow best practices for data security, governance, version control, and CI/CD.

Technical Requirements
- Strong hands-on experience with Apache Airflow and workflow orchestration.
- Excellent knowledge of SQL, including complex joins, CTEs, window functions, and query optimization.
- Strong experience with PySpark and distributed data processing.
- Practical experience working with Apache Iceberg and lakehouse data architectures.
- Strong experience with db

📌 Data Engineer (Nagpur)
🏢 Qloron Technology
📍 Nagpur