Job Responsibilities :

Data Integration and Orchestration :
- Design, develop, and maintain ETL pipelines using Azure Data Factory (ADF) to integrate data from various sources into the data lake or data warehouse.
- Implement data workflows and orchestrate data movement and transformation processes within the ADF environment.

Data Processing and Analytics :
- Utilize Databricks to develop and manage scalable data processing pipelines.
- Write and optimize PySpark scripts for data transformation, aggregation, and analysis within Databricks.
- Perform real-time and batch data processing on large datasets to support analytics and reporting needs.

Python Scripting and Automation : - Develop Python scripts to automate data processing tasks, enhance data pipelines, and integrate with other Azure services.
- Implement data validation and error-handling mechanisms within Python scripts to ensure data accuracy and reliability.

Data Modeling and Storage :
- Design and implement data models that support efficient querying, storage, and retrieval of data in the Azure ecosystem.




- Optimize data storage solutions within Azure, including Azure Data Lake, Azure SQL Data Warehouse, and other related services.

Collaboration and Stakeholder Engagement :
- Work closely with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
- Provide technical expertise and guidance on best practices for data engineering and cloud-based solutions.

Performance Tuning and Optimization :
- Monitor, troubleshoot, and optimize data pipelines for performance, scalability, and cost efficiency.
- Implement best practices for data security, governance, and compliance within the Azure environment.

Required Qualifications :

Experience :
- Minimum of 5 years of experience in data engineering or related roles.
- Solid hands-on experience with Azure Data Factory for data integration and orchestration.
- Proven expertise in using

📌 Data Engineer (Noida)
🏢 Forward Eye Technologies
📍 Noida