We are looking for an experienced Big Data Engineer with strong expertise in Scala and Apache Spark to design, develop, and optimize large-scale data processing solutions. The ideal candidate should have hands-on experience building robust data pipelines and working with distributed data platforms.
Key Responsibilities
- Design and develop scalable big data applications using Scala and Apache Spark.
- Build and maintain batch and real-time data processing pipelines.
- Optimize Spark applications for performance and scalability.
- Develop ETL/ELT workflows and data integration solutions.
- Collaborate with cross-functional teams to translate business requirements into technical solutions.
- Troubleshoot and resolve performance and data quality issues.
- Ensure adherence to coding standards and best practices.
Required Skills
- 6-8 years of experience in Data Engineering and Big Data technologies.
- Strong proficiency in Scala and Apache Spark.
- Hands-on experience with Hadoop ecosystem components.
- Robust SQL and data modeling skills.
- Experience with cloud platforms (Azure/AWS/GCP).
- Familiarity with distributed computing concepts and performance tuning.
- Experience with version control systems such as Git.
Preferred Skills
- Experience with Databricks, Kafka, Delta Lake, or Spark Streaming.
- Exposure to Airflow, Docker, and Kubernetes.
- Knowledge of CI/CD pipelines and DevOps practices.
Education
- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
Keywords Spark, Scala, Big Data, Hadoop, Kafka, Databricks, Data Engineering, ETL, Azure, AWS, GCP, Spark Streaming, Delta Lake
Interested candidates can apply with their updated resume.