Job Title: Cloud Data Engineer - Location - Bangalore - (Hybrid - 3Days a Week)

Looking for Immediate Joiners / Serving Notice Peroid.

Experience

- 5 7 years of relevant experience in Data Engineering, Big Data, or Cloud Data Engineering.
- Strong hands-on experience with Hadoop, Hive, HDFS, and PySpark is mandatory.

Key Responsibilities & Technical Skills
- Develop, maintain, and optimize scalable data engineering solutions using Python and Apache Spark/PySpark.
- Design and build reusable, scalable data ingestion and data transformation frameworks for large-scale data processing.
- Work extensively with the Hadoop ecosystem, including Hadoop, Hive, HDFS, and Impala, preferably in Cloudera-based environments.
- Demonstrate strong understanding of HDFS architecture and internals, including:
- Data storage architecture
- Partition management
- Compaction strategies
- Resource utilization
- Cluster performance optimization

- Troubleshoot complex platform, storage, compute, and cluster-level issues, perform detailed root cause analysis, and implement performance improvements.
- Work with a variety of database technologies and object storage platforms, ensuring optimized connectivity, data access, and processing patterns.
- Apply robust knowledge of distributed data processing, Spark optimization, and data platform engineering best practices.




- Develop and support applications using Kubernetes, including:

- Containerized Spark workloads
- Pod management
- Application scaling
- Troubleshooting
- Operational support

- Build, package, and deploy Spark workloads using JFrog-managed container images and automated CI/CD pipelines.
- Leverage AI Agents, Generative AI, and LLM-powered development tools to improve engineering productivity and accelerate software delivery.
- Collaborate with engineering and platform teams to develop reliable, scalable, and high-performance data solutions.

Mandatory Skills
- Hadoop
- Hive
- HDFS
- PySpark / Apache Spark
- Python
- Distributed data processing
- Spark optimization

Good to Have / Added Advantage
- Cloudera ecosystem
- Impala
- Kubernetes
- Containerized Spark workloads
- JFrog
- CI/CD pipelines
- Object storage platforms
- Scala development
- Generative AI / AI Agents / LLM-powered development tools

Preferred Profile Candidates should have 5 7 years of hands-on experience in data engineering or big data platforms, with strong practical expertise in Hadoop, Hive, HDFS, and PySpark.

Experience in Cloudera, Kubernetes, containerized Spark, CI/CD, and platform performance optimization will be an added advantage.

📌 Cloud Data Engineer (Karnataka)
🏢 SWITS DIGITAL Private
📍 Karnataka