Design, develop, and maintain scalable and high-performance data pipelines.

Build distributed data processing solutions using Apache Spark / PySpark.

Develop data ingestion, transformation, and integration processes from multiple data sources.

Work with AWS data engineering and cloud processing services.

Process structured, semi-structured, and unstructured data including JSON, logs, PDFs, and other file formats.

Develop complex and optimized SQL queries for data processing and analysis.

Work with both relational and non-relational databases.

Implement data validation, quality checks, and monitoring processes.

Troubleshoot complex data issues and perform root-cause analysis.

Optimize data pipelines, queries, and distributed processing jobs for performance and scalability.

Collaborate with stakeholders to understand business requirements and translate them into scalable data solutions.





Take ownership of data engineering initiatives and deliver solutions independently.

Stay updated with emerging AWS, Big Data, and Data Engineering technologies and best practices.

Requirements

Hands-on Python + PySpark for large-scale data processing

Strong/advanced SQL, including complex queries and optimization

Robust AWS / cloud data engineering experience

Apache Spark and distributed data processing

ETL/ELT pipeline development

Data ingestion, transformation and integration

Structured, semi-structured and unstructured data — JSON, logs, PDFs, etc.

Relational and non-relational databases

Data quality, validation, performance tuning and optimization

Deep-dive data analysis and troubleshooting

Scalable data solution design

📌 Senior Data Engineer (Chennai)
🏢 Mags Consulting
📍 Chennai