Responsible for designing, implementing, and maintaining efficient data ingestion and transformation workflows within the Cloudera cluster. By collaborating with data analysts, project leaders, and technical teams, the role ensures the delivery of high-quality data solutions that meet business needs. The position also focuses on establishing development standards, optimizing data pipelines, and supporting agile project execution to drive continuous improvement and operational excellence.

1. KEY RESPONSIBILITIES

Collaboration and Requirements Gathering

Engage with data analysts and project leaders to understand business needs and data flow requirements for various projects.

Translate business requirements into technical specifications for data ingestion and transformation processes.

Data Ingestion and Transformation

Design and specify data ingestion and transformation workflows in the Cloudera cluster (ELT), ensuring scalability, efficiency, and reliability.

Monitor data pipelines to ensure smooth operations and address any issues proactively.

Job Implementation and Development

Develop and implement Spark-based jobs and orchestration workflows to support data processing and transformation.

Establish and enforce development standards to accelerate project delivery and maintain code quality.

Agile Methodology and Team Collaboration

Actively participate in agile practices, including daily stand-ups, sprint planning, and retrospective meetings,



to ensure alignment with project goals.

Collaborate with technical leads to report progress, share updates, and address challenges.

Code Review and Quality Assurance

Conduct code reviews to ensure adherence to best practices, coding standards, and performance optimization.

Provide constructive feedback to team members to improve code quality and maintain consistency.

Incident Management and Evolution Requests

Follow up on incidents or requests for system evolution, ensuring timely resolution and continuous improvement of data pipelines.

Work closely with stakeholders to implement changes and enhancements as needed.

REQUIRED TECHNICAL COMPETENCIES

- Mastering of Hadoop & Cloudera workplace (including understanding insertion, retrieval performance, orchestration with yarn...)
- Mastering of other Big Data stack components (Kafka, HDFS, Iceberg)
- Knowledge of Azure Big Data systems and cloud computing.
- High development skills (Spark, PySpark)
- Knowledge of agile methodology
- Very rigorist in incident management procedure
- Fast adaptability to new technologies
- Business oriented for fast development (sens of delivery in time is very important)
- Team spirit
- Capability to work under time pressure

PREFERRED TECHNICAL COMPETENCIES

- Experience with business intelligence tools
- Ability to mentor and guide team members, ensuring adherence to development standards

📌 Data Engineer (Hyderabad)
🏢 Arcelormittal
📍 Hyderabad