18 Nov
|
Infosys
|
Hyderabad
Position:Big Data Engineer (Hadoop & PySpark)
Experience: 2 to 5 years
Location: Hyderabad
Employment Type: Full time
Roles & Responsibilities
Design, develop, and maintain scalable data pipelines using PySpark.
Process and transform large datasets from multiple data sources.
Build and optimize ETL/ELT workflows for data ingestion and processing.
Develop Spark jobs for batch and real-time data processing.
Write complex SQL queries for data extraction and reporting.
Perform data cleansing, validation, and quality checks.
Optimize Spark performance by tuning jobs, partitions, and cluster configurations.
Integrate data from APIs, databases, cloud storage, and enterprise systems.
Work with data architects, analysts, and business stakeholders to understand requirements.
Troubleshoot production issues and implement performance improvements.
Ensure data security, governance, and compliance standards are followed.
Participate in code reviews and follow best development practices.
Maintain technical documentation for data pipelines and processes.
📌 Big Data Engineer Hadoop & Pyspark Hyderabad
🏢 Infosys
📍 Hyderabad