"We need a Senior data engineer to design, build, and maintain end-to-end data pipelines and reporting solutions using Spark-based frameworks

(Spark SQL/Streaming) and Python across ETL/ELT workflows.

Develop and optimize high volume multi threaded batch processing on big data platforms, with robust performance tuning and production scheduling via

Autosys or Control-M

Implement Cloud Native data engineering solutions on AWS (S3, lake formation, Glue/EMR, Glue Catalog, Athena, Kinesis/MSK, Airflow/Lambda)"

Experience across the data life cycle spark-based frameworks for end-to-end ETL, ELT & reporting solutions using key components like spark, SQL and Spark streaming

Strong hands on working experience of Big Data stack including Spark and Python (Pandas, Spark SQL)

Good understanding on RDBMS database, Relational, No SQL and Linux/UNIX

Strong knowledge of multi threading and high volume batch processing

Should be good in perforemance tuning on for Python and spark along with Autosys or Control-M schedule

Formal training or certification on software engineering concepts + 3 years applied experience.

📌 Architect (Pune)
🏢 Virtusa
📍 Pune