Job Responsibilities :

SQL Development :
- Write medium to complex SQL queries to support data extraction, transformation, and analysis.
- Optimize SQL queries for performance and maintainability.PySpark/Spark Development :
- Develop, test, and deploy data processing applications using PySpark or Spark with Scala.
- Implement and maintain ETL/ELT data pipelines, ensuring efficient data processing and integration.ETL/ Data Engineering Pipeline Design :
- Design and implement robust ETL pipelines to support data ingestion, transformation, and loading.
- Collaborate with data architects and engineers to ensure seamless data flow and integration across systems.

Cloud Technology Utilization :
- Leverage cloud platforms (AWS, Azure, Google Cloud, etc.) to build scalable and effective data solutions.
- Work with cloud

-based data storage solutions (e.g., S3, Google Cloud Storage) and compute environments.

Debugging and Issue Resolution :
- Debug data processing issues and resolve challenges independently as an individual contributor.




- Troubleshoot and optimize existing pipelines to improve reliability and performance.Continuous Integration and Continuous Deployment (CI/CD) :
- Implement and manage CI/CD pipelines for automated testing, deployment, and monitoring.
- Ensure code quality and best practices through version control (e.g., Git) and CI/CD tools.

Desired Skills :

Data Modeling :
- Design and maintain logical and physical data models to support business requirements.
- Ensure data models are optimized for performance, scalability, and ease of use.Airflow :
- Use Apache Airflow to orchestrate and schedule complex data workflows.
- Develop and maintain DAGs for task automation and monitoring.

Kafka :
- Utilize Apache Kafka for building real-time data streaming pipelines.
- Integrate Kafka with other systems for event-driven data processing.Big Data Technologies :
- Experience with big data processing frameworks such as Hive, EMR, and Databricks.
- Familiarity

📌 Data Engineer (Noida)
🏢 Forward Eye Technologies
📍 Noida