04 Sep
|
Meta For Data Technologies Private
|
Pune
04 Sep
Meta For Data Technologies Private
Pune
We are looking for a Senior Data Engineer with solid Scala and Apache Spark expertise to build and optimize scalable data pipelines and Big Data solutions.
Key Responsibilities
Develop scalable data pipelines using Scala and Apache Spark.
Work with Spark DataFrames, RDDs, Spark SQL, joins, aggregations, and transformations.
Develop batch and streaming pipelines using Kafka and Spark.
Process structured and semi-structured data such as JSON, Avro, XML, and
Parquet.
Optimize Spark jobs and SQL queries for performance and cost efficiency.
Work with Hive, HDFS, Iceberg, and cloud platforms.
Perform data quality, validation, troubleshooting, and production support.
Follow coding, version control, and CI/CD practices using Git and Jenkins.
Required Skills
6–7 years of Data Engineering experience.
Strong hands-on Scala and Apache Spark experience.
Positive knowledge of Kafka, Hive, HDFS, SQL, and Big Data technologies.
Experience with AWS/GCP cloud platforms.
Solid Unix/Linux and Git knowledge.
Good understanding of data processing and performance optimization
📌 Data Engineer Scala Pune
🏢 Meta For Data Technologies Private
📍 Pune