We are looking for an experienced Data Engineer with a deep understanding of Azure and Databricks platforms. The ideal candidate will have at least 3+years of experience in data engineering, with expertise in designing, developing, and maintaining data pipelines, optimizing data processing workflows, and ensuring the reliability and scalability of our data infrastructure.

Key Responsibilities
- Data Pipeline Development:
Design, develop, and maintain scalable data pipelines and ETL processes using Azure Data Factory and Databricks. Implement automation and orchestration to streamline data processing.
- Dynamic OpCo Deployments:
Refactor existing code and RESTful services to support energetic OpCo (Operating Company) deployments, enabling multitenant scalability across the platform.
- Collaboration:
Work closely with data scientists, analysts, and business stakeholders to understand their data requirements and translate them into technical specifications that drive analytics and machine learning initiatives.
- Optimization & Scalability:




Optimize and manage data processing workflows for performance, scalability, and reliability. Implement best practices for efficient data storage and retrieval.
- Data Quality & Validation:
Implement robust data quality checks and validation processes to ensure the accuracy, consistency, and integrity of data across pipelines. Update and refine code testing strategies to continually improve data quality and reliability.
- Monitoring & Troubleshooting:
Monitor, troubleshoot, and optimize data pipelines to ensure timely and accurate data delivery, proactively identifying and addressing potential issues.
- Data Handling:
Work with large-scale datasets structured, semi-structured, and unstructured to support various analytics, machine learning, and business intelligence initiatives. Integrate new data pipeline executions with the HDP Upload Portal to streamline data ingestion and ensure seamless pipeline execution.
- Security & Compliance:

📌 Data Engineer (India)
🏢 HireFlex
📍 India