Data Engineer The Opportunity The Data Engineer is responsible for designing, building, and maintaining scalable data engineering solutions that enable reliable, secure, and high-quality data for analytics, reporting, and business decision-making. This role plays a critical part in the data and analytics lifecycle by developing production-grade data platforms and pipelines across cloud and on-premises environments. Guided by a data-driven and innovation-focused approach, this role contributes to transforming raw data into valuable business insights through up-to-date data engineering practices, cloud technologies, and big data frameworks. The position offers an exciting opportunity to work with cutting-edge technologies such as Azure, Databricks, Apache Spark, and real-time data processing solutions while collaborating with cross-functional teams across Business, Technology, Operations, and Data & Analytics functions.

Key Responsibilities
• Design, build, and maintain robust ETL/ELT pipelines on Azure cloud and on-premises platforms for batch and real-time data processing.
• Develop scalable data solutions to collect, ingest, transform, and store large volumes of structured and unstructured data.
• Monitor, optimize, and troubleshoot data pipelines to ensure high availability, reliability, scalability, and performance.
• Ensure adherence to data governance, security, privacy, compliance, and quality standards across all data engineering solutions.
• Develop and implement data architectures including data lakes, data warehouses, data marts, and Medallion architecture frameworks.
• Build real-time and batch data processing solutions using Apache Spark and streaming technologies.
• Collaborate with stakeholders across Business, Operations, Technology, Data Governance, Data Quality, Data Architecture, Data Science, DevOps, and BI teams.
• Support data platform modernization initiatives and continuous improvement of engineering practices.



Required Qualifications
• Bachelor's degree in Computer Science, Information Technology, Engineering, or an equivalent educational qualification.
• 5-8+ years of experience in Data Engineering, Big Data, Data Integration, or related disciplines.
• Hands-on expertise in SQL and Python and/or Scala for data engineering and application development.
• Strong experience in ETL/ELT pipeline development and large-scale data processing.
• Experience with Big Data technologies including Apache Spark, Hadoop, and Hive.
• Proficiency with Azure Data Engineering services such as Azure Data Factory, Azure Databricks, Synapse Analytics, Event Hub, and Azure Functions.
• Knowledge of distributed and NoSQL databases such as Cosmos DB and HBase.
• Experience designing and supporting Data Lakes, Data Warehouses, Data Marts, and Medallion Architecture.
• Strong understanding of performance tuning, optimization techniques, and data quality validation.
• Experience in both batch and real-time streaming data processing.
• Excellent communication, analytical thinking, stakeholder management, and structured problem-solving skills.
• Ability to work independently in an individual contributor role while collaborating effectively with diverse teams. Preferred Qualifications
• Experience with DevOps practices including Git, Azure DevOps, and CI/CD pipeline implementation.
• Knowledge of Unix Shell Scripting, MongoDB, and Apache NiFi.
• Exposure to Generative AI technologies, tools, and AI-driven data solutions.
• Experience in cloud migration and modern data platform implementation projects.
• Strong business acumen and ability to translate business requirements into scalable data solutions.
• Relevant cloud or data engineering certifications on Azure will be an advantage. Experience 5-8+ Years of Relevant Experience Education Bachelor's Degree in Computer Science, Information Technology, Engineering, or Equivalent Qualification.

📌 Big Data Engineer II (Pune)
🏢 MetLife
📍 Pune