Design, develop, and maintain scalable and high-performance data pipelines.
Build distributed data processing solutions using Apache Spark / PySpark.
Develop data ingestion, transformation, and integration processes from multiple data sources.
Work with AWS data engineering and cloud processing services.
Process structured, semi-structured, and unstructured data including JSON, logs, PDFs, and other file formats.
Develop complex and optimized SQL queries for data processing and analysis.
Work with both relational and non-relational databases.
Implement data validation, quality checks, and monitoring processes.
Troubleshoot complex data issues and perform root-cause analysis.
Optimize data pipelines, queries, and distributed processing jobs for performance and scalability.
Collaborate with stakeholders to understand business requirements and translate them into scalable data solutions.




Take ownership of data engineering initiatives and deliver solutions independently.
Stay updated with emerging AWS, Big Data, and Data Engineering technologies and best practices.

RequirementsHands-on Python + PySpark for large-scale data processing
Strong/advanced SQL, including complex queries and optimization
Solid AWS / cloud data engineering experience
Apache Spark and distributed data processing
ETL/ELT pipeline development
Data ingestion, transformation and integration
Structured, semi-structured and unstructured data — JSON, logs, PDFs, etc.
Relational and non-relational databases
Data quality, validation, performance tuning and optimization
Deep-dive data analysis and troubleshooting
Scalable data solution design

📌 Senior Data Engineer (Chennai)
🏢 Mags Consulting
📍 Chennai