Job Responsibilities :

SQL Development :
Write medium to complex SQL queries to support data extraction, transformation, and analysis.
Optimize SQL queries for performance and maintainability.PySpark/Spark Development :
Develop, test, and deploy data processing applications using PySpark or Spark with Scala.
Implement and maintain ETL/ELT data pipelines, ensuring effective data processing and integration.ETL/ Data Engineering Pipeline Design :
Design and implement robust ETL pipelines to support data ingestion, transformation, and loading.
Collaborate with data architects and engineers to ensure seamless data flow and integration across systems.

Cloud Technology Utilization :
Leverage cloud platforms (AWS, Azure, Google Cloud, etc.) to build scalable and effective data solutions.
Work with cloud

-based data storage solutions (e.g., S3, Google Cloud Storage) and compute settings.

Debugging and Issue Resolution :
Debug data processing issues and resolve challenges independently as an individual contributor.




Troubleshoot and optimize existing pipelines to improve reliability and performance.Continuous Integration and Continuous Deployment (CI/CD) :
Implement and manage CI/CD pipelines for automated testing, deployment, and monitoring.
Ensure code quality and best practices through version control (e.g., Git) and CI/CD tools.

Desired Skills :

Data Modeling :
Design and maintain logical and physical data models to support business requirements.
Ensure data models are optimized for performance, scalability, and ease of use.Airflow :
Use Apache Airflow to orchestrate and schedule complex data workflows.
Develop and maintain DAGs for task automation and monitoring.

Kafka :
Utilize Apache Kafka for building real-time data streaming pipelines.
Integrate Kafka with other systems for event-driven data processing.Big Data Technologies :
Experience with big data processing frameworks such as Hive, EMR, and Databricks.
Familiarity

📌 Data Engineer Noida
🏢 Forward Eye Technologies
📍 Noida