13 Aug
|
Qloron Technology
|
Hyderabad
13 Aug
Qloron Technology
Hyderabad
Job Summary
We are seeking a motivated and skilled Data Engineer with 3-4 years of experience in building scalable data pipelines, ETL processes, and cloud-based data solutions. The ideal candidate should have solid hands-on expertise in Databricks, Python, SQL, and Apache Airflow, along with
experience supporting data migration initiatives from on-premises platforms to modern cloud environments. The role involves designing, developing, optimizing, and maintaining data pipelines that support analytics, reporting, and business intelligence requirements.
Key Responsibilities
Data Engineering & ETL Development
- Design, develop, and maintain scalable ETL/ELT pipelines using Python, PySpark,
Databricks, and SQL.
- Build reusable data ingestion, transformation, and data quality frameworks.
- Develop batch and near real-time data processing solutions.
- Perform data cleansing, transformation, validation, and enrichment activities.
Databricks Development
- Build and optimize Spark/PySpark applications in Azure Databricks.
- Work with Delta Lake, data partitioning, caching,
and performance tuning techniques.
- Create and maintain notebooks, jobs, workflows, and reusable libraries.
- Implement medallion architecture (Bronze, Silver, Gold layers).
Workflow Orchestration
- Develop and maintain Apache Airflow DAGs for scheduling and orchestration.
- Monitor pipeline execution and troubleshoot failures.Classification: Internal
- Implement alerting, logging, and retry mechanisms.
Cloud Migration Activities
- Participate in migration of on-premise data assets and ETL jobs to cloud platforms.
- Analyze legacy ETL workflows and redesign them using cloud-native services.
- Support data validation, reconciliation, and migration testing activities.
- Assist in cutover and production deployment activities.
SQL & Data Modeling
- Develop complex SQL queries, stored procedures, and performance optimization techniques.
- Design dimensional and normalized data models.
- Support dat
📌 Junior Data Engineer (Hyderabad)
🏢 Qloron Technology
📍 Hyderabad