"We need a Senior data engineer to design, build, and maintain end-to-end data pipelines and reporting solutions using Spark-based frameworks (Spark SQL/Streaming) and Python across ETL/ELT workflows. Develop and optimize high volume multi threaded batch processing on big data platforms, with strong performance tuning and production scheduling via Autosys or Control-M Implement Cloud Native data engineering solutions on AWS (S3, lake formation, Glue/EMR, Glue Catalog, Athena, Kinesis/MSK, Airflow/Lambda)" Experience across the data life cycle spark-based frameworks for end-to-end ETL, ELT & reporting solutions using key components like spark, SQL and Spark streaming Strong hands on working experience of Big Data stack including Spark and Python (Pandas, Spark SQL) Good understanding on RDBMS database, Relational, No SQL and Linux/UNIX Strong knowledge of multi threading and high volume batch processing Should be valuable in perforemance tuning on for Python and spark along with Autosys or Control-M schedule Formal training or certification on software engineering concepts + 3 years applied experience.

📌 Architect (Pune)
🏢 Virtusa
📍 Pune