Data Pipeline Development: Design and develop scalable data pipelines to process and transform large volumes of structured and unstructured data.

- Data Processing: Build and optimize data processing frameworks using Apache Spark.
- Data Integration: Integrate data from multiple sources to create reliable and efficient data workflows.
- Data Management: Ensure productive storage, retrieval, and transformation of data for analytics and reporting.
- System Performance: Monitor and optimize data pipelines for performance, scalability, and reliability.
- Troubleshooting: Identify and resolve issues in data pipelines and processing systems to ensure data accuracy and availability.
- Collaboration: Work closely with data scientists, backend engineers, and product teams to understand data requirements and deliver solutions.
- Documentation: Maintain clear documentation of data pipelines, data models, and workflows.
- Continuous Improvement: Stay updated with emerging technologies and best practices in big data and data engineering

Roles and Responsibilities

Data Pipeline Development:



Design and develop scalable data pipelines to process and transform large volumes of structured and unstructured data.

- Data Processing: Build and optimize data processing frameworks using Apache Spark.
- Data Integration: Integrate data from multiple sources to create reliable and efficient data workflows.
- Data Management: Ensure efficient storage, retrieval, and transformation of data for analytics and reporting.
- System Performance: Monitor and optimize data pipelines for performance, scalability, and reliability.
- Troubleshooting: Identify and resolve issues in data pipelines and processing systems to ensure data accuracy and availability.
- Collaboration: Work closely with data scientists, backend engineers, and product teams to understand data requirements and deliver solutions.
- Documentation: Maintain transparent documentation of data pipelines, data models, and workflows.
- Continuous Improvement: Stay updated with emerging technologies and best practices in big data and data engineering

📌 SSE- Data Engineer (Noida)
🏢 Info Edge
📍 Noida