Job Summary

We are seeking a motivated and skilled Data Engineer with 3-4 years of experience in building scalable data pipelines, ETL processes, and cloud-based data solutions. The ideal candidate should have strong hands-on expertise in Databricks, Python, SQL, and Apache Airflow, along with

experience supporting data migration initiatives from on-premises platforms to modern cloud environments. The role involves designing, developing, optimizing, and maintaining data pipelines that support analytics, reporting, and business intelligence requirements.

Key Responsibilities

Data Engineering & ETL Development

- Design, develop, and maintain scalable ETL/ELT pipelines using Python, PySpark,

Databricks, and SQL.

- Build reusable data ingestion, transformation, and data quality frameworks.
- Develop batch and near real-time data processing solutions.
- Perform data cleansing, transformation, validation, and enrichment activities.

Databricks Development

- Build and optimize Spark/PySpark applications in Azure Databricks.
- Work with Delta Lake, data partitioning, caching, and performance tuning techniques.
- Create and maintain notebooks, jobs, workflows, and reusable libraries.
- Implement medallion architecture (Bronze, Silver, Gold layers).

Workflow Orchestration

- Develop and maintain Apache Airflow DAGs for scheduling and orchestration.
- Monitor pipeline execution and troubleshoot failures.Classification: Internal
- Implement alerting, logging, and retry mechanisms.

Cloud Migration Activities

- Participate in migration of on-premise data assets and ETL jobs to cloud platforms.
- Analyze legacy ETL workflows and redesign them using cloud-native services.
- Support data validation, reconciliation, and migration testing activities.
- Assist in cutover and production deployment activities.

SQL & Data Modeling

- Develop complex SQL queries,



stored procedures, and performance optimization techniques.
- Design dimensional and normalized data models.
- Support data warehousing and reporting requirements.

Collaboration & Governance

- Collaborate with business analysts, architects, and stakeholders to understand data requirements.
- Participate in Agile ceremonies, sprint planning, and code reviews.
- Ensure adherence to security, governance, and data quality standards.

Required Technical Skills

Skill Experience

Azure Databricks - 3+ Years

Python / PySpark - 3+ Years

SQL - 3+ Years

Apache Airflow - 2+ Years

ETL / ELT Development - 3+ Years

Cloud Migration Projects - 1+ Years

Git / Version Control - Working Knowledge

Azure Data Lake Storage (ADLS) - Preferred

CI/CD (Azure DevOps, GitHub Actions) - Preferred

Required Qualifications

- Bachelor's degree in Computer Science, Information Technology, Engineering, or related

field.

- 3 to 4 years of experience in Data Engineering.
- Solid understanding of Data Warehousing concepts.
- Experience working with large-scale structured and semi-structured datasets.
- Good knowledge of cloud-based analytics platforms.

Preferred Skills

- Azure Data Factory (ADF)
- Delta Lake
- Snowflake
- Kafka/Event Streaming
- Azure Synapse Analytics
- Data Quality and Data Governance frameworks
- Terraform or Infrastructure as Code
- Exposure to CI/CD pipelines

Key Competencies

- Strong analytical and problem-solving skills.
- Excellent communication and stakeholder management.
- Ability to work independently and within Agile teams.
- Strong debugging and performance tuning capabilities.
- Focus on quality, automation, and continuous improvement.

Nice to Have

- Azure Data Engineer Associate Certification.Classification: Internal
- Experience with enterprise-scale cloud migration programs.
- Exposure to healthcare, finance, telecom, or retail data domains.

📌 Junior Data Engineer (Hyderabad)
🏢 Qloron Technology
📍 Hyderabad