Responsibilities

- Architect, design, and deliver large-scale cloud-native data platforms primarily on AWS, with exposure to Azure and GCP, using Databricks and distributed processing frameworks to build highly scalable and resilient systems.
- Work hands-on across the technology stack - including Python, PySpark, Apache Spark, Databricks, AWS services, event-driven architectures, and SQL/NoSQL databases - to solve complex engineering challenges and maintain platform excellence.
- Lead organization-wide technical initiatives focused on performance optimization, scalability, reliability, security, governance, and cost efficiency.
- Review and influence critical architectural decisions across teams while establishing engineering best practices and architectural governance standards.
- Partner closely with global engineering, product management, architecture, and business stakeholders to align technical solutions with strategic business objectives.
- Own the end-to-end software development lifecycle, including requirements gathering, solution design, development, deployment, observability, and documentation.
- Mentor and guide senior and junior engineers, fostering a culture of innovation, accountability, collaboration, and technical excellence.

Qualifications

- B.E/B.Tech/M.Tech/MCA in Computer Science, Information Technology or a related field.
- 10-13 years of robust software engineering experience, with significant expertise in large-scale data engineering and distributed systems architecture.
- Solid expertise in Data Warehousing, Data Lakes, Delta Lake architecture, and modern big data ecosystem designs.
- Deep hands-on expertise in Databricks, Python,



PySpark and Apache Spark, with proven experience building high-performance distributed data processing solutions and managing massive-scale datasets for analytics and business intelligence workloads.
- Solid experience with AWS services such as S3, Glue, Redshift, EMR, Athena, Lambda, and EventBridge for building scalable and reliable cloud-native data platforms.
- Experience with real-time and near real-time streaming technologies including Kafka, AWS Kinesis, SQS, and RabbitMQ.
- Strong understanding of both relational and NoSQL databases, including PostgreSQL, SQL Server, Aurora, RDS, MongoDB, and DynamoDB.
- Hands-on experience with Infrastructure as Code (IaC) tools such as Terraform or Ansible.
- Strong understanding of CI/CD and DevOps practices using tools such as Jenkins, Github/GitLab, Bitbucket, GoCD and automated deployment pipelines.
- Proven ability to influence technical direction, drive multi-functional initiatives, and lead complex engineering programs at scale.
- Exposure to Generative AI technologies including LLMs, RAG architectures, and Agentic AI systems, with experience designing and deploying AI driven solutions.
- Nice to have: AWS and Databricks certifications and experience with Azure / Google Cloud Platform (GCP).
- Working knowledge of TypeScript, Node.Js and UX design & development.
- Experience building data platforms in privacy-safe or cleanroom environments.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Staff Software Engineer (Bengaluru)
🏢 Epsilon Data Management
📍 Bengaluru