Candidates ready to join immediately can share their details via email for quick processing.
? CCTC | ECTC | Notice Period | Location Preference
[email protected]
Act fast for immediate attention ⏳?

Key Responsibilities
Design, develop, and maintain scalable data pipelines using PySpark
Build and manage batch and real-time data processing systems
Develop and integrate Kafka-based streaming solutions
Optimize Spark jobs for performance, cost, and scalability
Work with cloud-native services to deploy and manage data solutions
Ensure data quality, reliability, and security across platforms
Collaborate with data scientists, analysts, and application teams
Participate in code reviews, design discussions, and production support
Must-Have Skills
Strong hands-on experience with PySpark / Apache Spark
Solid understanding of distributed data processing concepts
Experience with Apache Kafka (producers, consumers, topics, partitions)
Hands-on experience with any one cloud platform:
AWS (S3, EMR, Glue, EC2, IAM) or




Azure (ADLS, Synapse, Databricks) or
GCP (GCS, Dataproc, BigQuery)
Proficiency in Python
Strong experience with SQL and data modeling
Experience working with large-scale datasets
Familiarity with Linux/Unix environments
Understanding of ETL/ELT frameworks
Experience with CI/CD pipelines for data applications
Positive-to-Have Skills
Experience with Spark Structured Streaming
Knowledge of Kafka Connect and Kafka Streams
Exposure to Databricks
Experience with NoSQL databases (Cassandra, MongoDB, HBase)
Familiarity with workflow orchestration tools (Airflow, Oozie)
Knowledge of containerization (Docker, Kubernetes)
Experience with data lake architectures
Understanding of security, governance, and compliance in cloud environments
Exposure to Scala or Java is a plus
Prior experience in Agile/Scrum environments

📌 Senior Data Engineer – PySpark, Cloud (Bengaluru)
🏢 UST
📍 Bengaluru