21 Aug
|
Arminus Software
|
India
21 Aug
Arminus Software
India
As a Senior Big Data Engineer, your role will involve designing and developing enterprise-scale big data systems for governing and managing batch and online jobs. You should have 11 years of hands-on experience in design and development of enterprise-scale applications, with a focus on Big Data technologies in AWS EMR and Hadoop ecosystem. Your expertise should include Spark, Scala, PySpark, Airflow, Kafka, Delta Tables, Iceberg, HBase, Hive, HDFS, MapReduce, and more.Key Responsibilities:- Design and develop a system to govern and manage batch and online jobs- Define technical scope and objectives through research and requirements-gathering- Ingest and process data from various sources in different formats into the Big Data ecosystem- Perform real-time data feed processing using Big Data ecosystem- Design, review, implement, and optimize data transformation processes- Test and prototype new data integration tools and methodologies- Participate in overall test planning for application integrations and projects- Collaborate with cross-functional teams in an Agile/Scrum environment to ensure high-quality product deliveryQualifications Required:- 11 years of hands-on experience in design and development of enterprise-scale applications- 8 years of expertise in Big Data technologies in AWS EMR and Hadoop ecosystem- Proficiency in Spark, Scala, PySpark, Airflow, Kafka, Delta Tables, Iceberg, HBase, Hive, HDFS, MapReduce, etc.In this role, you will work closely with cross-functional teams in an Agile/Scrum environment to deliver quality products. As a Senior Big Data Engineer,
your role will involve designing and developing enterprise-scale big data systems for governing and managing batch and online jobs. You should have 11 years of hands-on experience in design and development of enterprise-scale applications, with a focus on Big Data technologies in AWS EMR and Hadoop ecosystem. Your expertise should include Spark, Scala, PySpark, Airflow, Kafka, Delta Tables, Iceberg, HBase, Hive, HDFS, MapReduce, and more.Key Responsibilities:- Design and develop a system to govern and manage batch and online jobs- Define technical scope and objectives through research and requirements-gathering- Ingest and process data from various sources in different formats into the Big Data ecosystem- Perform real-time data feed processing using Big Data ecosystem- Design, review, implement, and optimize data transformation processes- Test and prototype new data integration tools and methodologies- Participate in overall test planning for application integrations and projects- Collaborate with cross-functional teams in an Agile/Scrum setting to ensure high-quality product deliveryQualifications Required:- 11 years of hands-on experience in design and development of enterprise-scale applications- 8 years of expertise in Big Data technologies in AWS EMR and Hadoop ecosystem- Proficiency in Spark, Scala, PySpark, Airflow, Kafka, Delta Tables, Iceberg, HBase, Hive, HDFS, MapReduce, etc.In this role, you will work closely with cross-functional teams in an Agile/Scrum environment to deliver quality products.
📌 Big Data Engineer with AWS EMR (India)
🏢 Arminus Software
📍 India