**Job Responsibilities :

Data Integration and Orchestration :**

- Design, develop, and maintain ETL pipelines using Azure Data Factory (ADF) to integrate data from various sources into the data lake or data warehouse.
- Implement data workflows and orchestrate data movement and transformation processes within the ADF environment.

Data Processing and Analytics :

- Utilize Databricks to develop and manage scalable data processing pipelines.
- Write and optimize PySpark scripts for data transformation, aggregation, and analysis within Databricks.
- Perform real-time and batch data processing on large datasets to support analytics and reporting needs.

Python Scripting and Automation : - Develop Python scripts to automate data processing tasks, enhance data pipelines, and integrate with other Azure services.

- Implement data validation and error-handling mechanisms within Python scripts to ensure data accuracy and reliability.

Data Modeling and Storage :

- Design and implement data models that support efficient querying, storage, and retrieval of data in the Azure ecosystem.
- Optimize data storage solutions within Azure, including Azure Data Lake, Azure SQL Data Warehouse, and other related services.

Collaboration and Stakeholder Engagement :

- Work closely with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
- Provide technical expertise and guidance on best practices for data engineering and cloud-based solutions.

Performance Tuning and Optimization :

- Monitor, troubleshoot, and optimize data pipelines for performance, scalability, and cost efficiency.




- Implement best practices for data security, governance, and compliance within the Azure environment.

**Required Qualifications :

Experience :**

- Minimum of 5 years of experience in data engineering or related roles.
- Strong hands-on experience with Azure Data Factory for data integration and orchestration.
- Proven expertise in using Databricks for data processing and analytics.
- Proficiency in Python scripting, particularly in the context of data processing and automation.

Technical Skills :

- In-depth knowledge of PySpark for large-scale data processing.
- Familiarity with Azure services such as Azure Data Lake, Azure SQL Data Warehouse, and Azure Synapse Analytics.
- Experience with data modeling, ETL processes, and data architecture design in cloud environments.
- Robust understanding of CI/CD pipelines and version control using Git.

Soft Skills :

- Excellent problem-solving skills with the ability to debug and resolve complex data engineering challenges.
- Strong communication skills for effective collaboration with technical and non-technical stakeholders.
- Ability to work independently and manage multiple priorities in a fast-paced environment.

Preferred Qualifications :

- Experience with data orchestration tools such as Apache Airflow.
- Familiarity with other cloud platforms (e.g., AWS, GCP) is a plus.
- Understanding of data governance and security practices in the cloud.

Keywords :

- Azure Data Factory (ADF)

- Databricks

- PySpark

- Python

- Data Engineering

- Cloud Data Solutions

- ETL Pipelines

- Data Integration

- Data Modeling
- Azure Data Lake.

Location - Hyderabad , Ahmedabad , pune , chennai , kolkata.

📌 Data Engineer - Synapse Analytics (Noida)
🏢 Forward Eye Technologies
📍 Noida