20 Nov
|
AndorTech
|
Bengaluru
20 Nov
AndorTech
Bengaluru
Role : Senior Data Engineer (7+ Years).We are looking for an experienced Senior Data Engineer with strong hands-on expertise in Spark, Scala, Hive, and Kafka to join our team. The ideal candidate should have experience building and optimizing large-scale data pipelines, working with distributed data processing systems, and handling complex production environments. Candidates should be comfortable taking ownership of deliverables and contributing independently with minimal supervision.Key Responsibilities:- Design, develop, and optimize large-scale data pipelines using Spark and Scala.- Work on high-volume batch and real-time data processing.- Develop and maintain ETL workflows and data orchestration pipelines.- Troubleshoot complex data and performance issues in production environments.- Perform Spark job tuning and optimization.- Contribute to cloud migration and data platform modernization initiatives.- Implement CI/CD pipelines and automated deployment processes.- Collaborate with cross-functional teams to deliver scalable data solutions.Required Skills:- 7+ years of experience in Data Engineering / Big Data Development.- Strong hands-on experience with Apache Spark, Scala, and Hive.- Experience with Kafka and distributed data processing.- Strong programming skills in Scala and/or Python.- Hands-on experience building and troubleshooting large-scale ETL pipelines.- Experience with at least one cloud platform: GCP, AWS, or Azure.- Knowledge of Git, Jenkins, CI/CD, and workflow orchestration tools such as Oozie.Valuable to Have:- GCP services - BigQuery, GCS, Dataproc, and Pub/Sub.- Experience with cloud migration of on-premise Spark/Hive workloads.- Experience with large-scale data migration and reconciliation.- Knowledge of data quality and observability frameworks.What We're Looking For:- Someone who can ramp up quickly, independently troubleshoot complex issues, take end-to-end ownership, and deliver production-ready solutions with minimal handholding. (ref:hirist.tech)
📌 Senior Data Engineer - Spark/Scala (Bengaluru)
🏢 AndorTech
📍 Bengaluru