About the Role:We are looking for an experienced Senior Data Engineer who can quickly onboard and contribute to the design, development, optimization and support of large-scale data platforms. The ideal candidate should be hands-on, comfortable working with complex data pipelines and distributed systems, and capable of independently troubleshooting and delivering solutions in a fast-paced workplace.Key Responsibilities:- Design, develop and optimize large-scale data pipelines using Apache Spark and Scala.- Work with high-volume batch and streaming workloads across multiple data sources.- Develop and support data processing solutions using Hive, Kafka and GCP services.- Troubleshoot complex data, performance and production issues with minimal supervision.- Contribute to cloud migration and platform modernization initiatives.- Build and maintain ETL workflows and data orchestration pipelines.- Perform performance tuning and optimization of Spark jobs and large datasets.- Develop reusable, maintainable and production-ready data solutions.- Implement and maintain CI/CD pipelines and engineering automation.- Collaborate with engineering, product, analytics and platform teams to deliver end-to-end solutions.- Participate in code reviews, testing,



production deployments and operational support.Technical Requirements:- Strong hands-on expertise in Apache Spark and Scala for building and optimizing large-scale data pipelines.- Experience working with high-volume batch and streaming workloads across multiple data sources.- Strong experience with Hive and distributed data processing.- Hands-on experience with Kafka and event/streaming-based architectures.- Strong programming experience in Scala and/or Python.- Experience designing, developing and troubleshooting large-scale ETL/data pipelines.- Strong understanding of data modelling, data quality, performance tuning and distributed systems.- Experience working with at least one major cloud platform - GCP, AWS or Azure.- Experience with BigQuery and GCP services is preferred.- Experience with ETL workflows and orchestration tools such as Oozie or equivalent.- Hands-on experience with Git, Jenkins and CI/CD practices.Preferred Qualifications:- Strong Google Cloud Platform (GCP) experience.- Exposure to BigQuery, Google Cloud Storage (GCS), Dataproc and Pub/Sub.- Experience migrating on-premise Spark/Hive workloads to cloud platforms.- Experience with large-scale data migration and reconciliation.- Knowledge of data quality and observability frameworks.- Experience working with multiple upstream and downstream integrations.- Exposure to multiple cloud environments. (ref:hirist.tech)

📌 Senior Data Engineer - Apache Spark (Bengaluru)
🏢 Scaling Theory Technologies
📍 Bengaluru