We are looking for a Big Data Engineer who will help us build ETL projects on Bigdata Stack
Your primary focus will be in building ETL scripts using Spark and Kafka.
Essentially anyone who has done some work on building data migration through ETL pipelines implemented in Bigdata.
Role Description:
- Develop ETL/Data Integration pipelines among various source systems and data warehouse
- Expertise in Spark with Java on Big Data/Cloud Platforms, Kafka for Pub-sub is must to design, develop, test and implement data pipelines that are developed as part of the application functionality.
Other
Bigdata tools like Hive, Hbase, Sqoop, Oozie are also required.
- Developer should have knowledge of any of cloud native technologies including AWS/Azure.
- Understand the existing data flows and recreate or design new data flows, graphs, plans and implement them in production
- Manage and maintain existing and new business rules using ETL or rules engine and test and integrate them in the data pipelines
- Work with source side teams in understanding data models,
and source data requirements to create staging and final Data Lake Data models and create HLDs, LLDs for the required data models
- Use SQL skills to query data and understand the data relationships. Also use ad-hoc querying skills to understand the data flows, data transformations and data reconciliation, validation
- Test the data pipelines in development and QA workplace
Qualifications:
Any Graduate degree but Computer Science preferred
Desired Skills and Competencies:
- 3 -4 years of solid hands-on experience in Big Data technologies such as Hadoop, Spark (Java/Python), Sqoop, Hive, Kafka
- Must have experience on Big Data Stack
- Must have hands on experience on Hive. Strong experience in Data Integration and ETL/ELT techniques
- Should have experience with SQL database programming, SQL performance tuning, relational model analysis.
- Must have good communications skills and should be able to lead meetings, technical discussions, and escalation calls
- Must have good documentation Techniques.
📌 Spark developer with ETL/ Informatica Dev experience (Bengaluru)
🏢 Labq Global Technologies
📍 Bengaluru
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.