Roles & Responsibilities
Design, develop, and maintain scalable data pipelines using PySpark.
Process and transform large datasets from multiple data sources.
Build and optimize ETL/ELT workflows for data ingestion and processing.
Develop Spark jobs for batch and real-time data processing.
Write complex SQL queries for data extraction and reporting.
Perform data cleansing, validation, and quality checks.
Optimize Spark performance by tuning jobs, partitions, and cluster configurations.
Integrate data from APIs, databases, cloud storage, and enterprise systems.
Work with data architects, analysts, and business stakeholders to understand requirements.
Troubleshoot production issues and implement performance improvements.
Ensure data security, governance, and compliance standards are followed.
Participate in code reviews and follow best development practices.
Maintain technical documentation for data pipelines and processes.
📌 Big Data Engineer Bengaluru
🏢 Infosys
📍 Bengaluru
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.