25 Aug
|
Qloron
|
Bengaluru
? Job ID: QTAJH08604
? Role: Data Engineer
? Work Mode: Full-Time
? Location: Bangalore / Hyderabad
⏳ Experience: 5+ Years
? Mandatory Skills
Candidates must have hands-on experience in all of the following technologies:
✅ Apache Airflow
✅ Data Lake / Data Lake Architecture
✅ SQL
✅ Apache Iceberg
✅ dbt (Data Build Tool)
✅ PySpark
⚠️ All listed skills are mandatory. Candidates without hands-on experience in any of these technologies may not be considered.
? Key Responsibilities
Design, develop and maintain scalable data engineering pipelines.
Build and orchestrate data workflows using Apache Airflow.
Develop and optimize data processing solutions using PySpark.
Work with up-to-date Data Lake architectures and large-scale datasets.
Implement and manage Apache Iceberg tables and data pipelines.
Develop reliable transformation models using dbt.
Write complex and optimized SQL queries for data extraction, transformation and analysis.
Build batch and incremental data processing pipelines.
Implement data quality, validation and monitoring frameworks.
Optimize data pipelines for performance, scalability and cost.
Collaborate with Data Architects, Analytics Engineers and other technical teams.
Troubleshoot pipeline failures and production data issues.
Follow best practices for data governance, security and CI/CD.
? Required Technical Expertise
Data Engineering
Strong Data Engineering fundamentals
Data Lake / Lakehouse architecture
Batch and incremental processing
ETL/ELT pipeline development
Pipeline Orchestration
Apache Airflow
DAG development
Scheduling and dependency management
Pipeline monitoring and troubleshooting
Big Data
PySpark
Distributed data processing
Performance optimization
Data Lake / Lakehouse
Apache Iceberg
Partitioning
Schema evolution
Time travel
Snapshot management
Transformation
dbt
Data modeling
Incremental models
Testing and documentation
Database
Advanced SQL
Query optimization
Joins, CTEs, Window Functions
Performance tuning
? Candidate Profile
We are l
📌 Data Engineer – Airflow | Data Lake | SQL | Iceberg | dbt | PySpark (Bengaluru)
🏢 Qloron
📍 Bengaluru