01 Oct
|
EVOKE HR
|
Chennai
Position :- Big Data Engineer
Location :- Bangalore / Chennai
Experience 4 +Years
Job Summary:
We are looking for a skilled Big Data Engineer to design, develop, and maintain scalable data processing solutions. The ideal candidate should have solid experience with Python/Java/Scala, Spark, Hadoop, SQL, ETL pipelines, and cloud platforms.
Key Responsibilities
- Design and develop scalable Big Data processing and ETL/ELT pipelines.
- Develop data processing solutions using Apache Spark / PySpark.
- Work with large and complex datasets to perform data transformation and analysis.
- Build and optimize data pipelines using Python, Java, or Scala.
- Develop complex SQL queries and optimize database performance.
- Work with Hadoop ecosystem components such as HDFS, Hive, Kafka, and YARN.
- Implement real-time and batch data processing solutions.
- Integrate data from multiple sources into data lakes/data warehouses.
- Monitor, troubleshoot,
and optimize data pipelines and jobs.
- Ensure data quality, reliability, security, and performance.
- Collaborate with Data Scientists, Data Analysts, Architects, and business teams.
- Participate in code reviews, testing, deployment, and production support.
- Apache Iceberg is mandatory skill.
Required Skills
- Strong programming experience in Python / Java / Scala.
- Hands-on experience with Apache Spark / PySpark.
- Strong SQL skills.
- Experience with ETL/ELT and data pipeline development.
- Good understanding of Hadoop ecosystem.
- Experience with Hive, Kafka, HDFS, YARN.
- Experience with Data Lake / Data Warehouse concepts.
- Knowledge of Git, CI/CD, and DevOps practices.
- Good understanding of distributed computing and data processing.
📌 Data Engineer (Chennai)
🏢 EVOKE HR
📍 Chennai