Software Development Engineer II (SDE II)-Pyspark

Location: Bengaluru, Karnataka, India

Experience: 4 - 5 Years

Function: Engineering / Technical / Analytics

About the Role

We are looking for a highly skilled and motivated Software Development Engineer III to join our high-impact engineering teams supporting strategic data and analytics initiatives for leading global financial institutions. The role demands strong ownership across the entire software development lifecycle, with a focus on scalable distributed systems, Big Data engineering, and enterprise-grade development practices.

This position offers an opportunity to work on large-scale data platforms, real-time data pipelines, and next-generation AI-driven solutions, while collaborating with global stakeholders and driving innovation.

Key Responsibilities

Engineering & Delivery

- Own the end-to-end lifecycle including requirements, design, development, testing, deployment, and production support.
- Design and build scalable, high-performance applications and distributed data pipelines.
- Develop reusable frameworks, APIs, and integration patterns to solve complex enterprise and distributed computing problems.
- Implement robust data ingestion, transformation, and processing solutions in parallel processing environments.
- Ensure solid software engineering practices such as code quality, version control, CI/CD, automated testing, and secure deployment.

Big Data & Distributed Systems

- Provide technical expertise in Big Data and distributed computing technologies, including:
- Apache Spark, Kafka, Flink
- Hadoop ecosystem (MapReduce, HDFS, Hive, HBase, Flume, Sqoop)
- NoSQL databases such as Cassandra and HBase
- Build and optimize real-time and batch data pipelines.
- Explore and evaluate new technologies to improve scalability, performance, and cost efficiency.

Technical Leadership & Innovation

- Drive adoption and contribution to open-source technologies.
- Provide technical mentorship and leadership to junior engineers.




- Promote best practices in system design, coding, and performance optimization.
- Align engineering solutions with strategic client initiatives including AI, automation, and data governance.

Client Collaboration

- Interface daily with global stakeholders to understand business and technical requirements.
- Translate strategic business needs into scalable engineering solutions.
- Ensure delivery excellence and measurable business impact.

AI-First & Continuous Learning

- Stay updated with emerging technologies such as Generative AI, data platforms, and cloud ecosystems.
- Contribute to innovation initiatives including automation, intelligent workflows, and next-generation analytics.

Required Skills & Experience

- 6+ years of strong hands-on software development experience.
- Proficiency in Java, Scala, Python, or C++, with strong coding and debugging skills.
- Deep understanding of Data Structures, Algorithms, OOPS, and design patterns.
- Experience in building enterprise-scale applications.
- Strong understanding of SDLC, Agile methodologies, and DevOps practices.
- Experience with application integration, data management, and production support.
- Ability to work independently and adapt quickly in fast-paced environments.

Preferred Qualifications

- Hands-on experience in Core Java or Scala.
- Exposure to functional programming paradigms.
- Experience with third-party libraries, frameworks, and open-source ecosystems.
- Exposure to cloud platforms such as AWS, Azure, or GCP.
- Experience in financial services, compliance, or regulated environments is a plus.
- Understanding of data governance, security, and regulatory controls.

Culture & Mindset We are looking for individuals who:

- Demonstrate strong ownership and accountability.
- Are strategic, analytical, and data-driven.
- Can think creatively and solve problems unconventionally.
- Thrive in a fast-growing, entrepreneurial, and agile environment.
- Are both leaders and hands-on doers.
- Show high energy, curiosity, and commitment to excellence.

📌 SDE 2 Pyspark Data Engineer (Bengaluru)
🏢 Astar Data
📍 Bengaluru