Job Description

Key Responsibilities

- Technical Leadership: Lead the technical design, architecture, and implementation of high-throughput data ingestion systems and ETL pipelines.

- Team Mentorship: Guide, mentor, and elevate a team of Engineers, fostering a culture of technical excellence and best practices.

- Hands-on Development: Write clean, scalable, and maintainable code primarily in Java and Python.

- Performance Optimization: Proactively monitor and resolve complex issues related to high memory and CPU consumption, bottlenecks in data transfer, and queue processing.

- Agile Delivery: Partner with product managers and scrum masters to drive Agile/Scrum ceremonies, sprint planning, and timely delivery of features.

- End-to-End Ownership: Oversee the full software development lifecycle, from system design and data formatting/analytics to deployment and infrastructure management.

Technical Requirements

Core Backend & Data Engineering:

- Languages: Expert-level proficiency in Java and Python.

- Data & Big Data: Deep understanding of Apache Spark, ETL pipeline design, data formatting, and data analytics.

- Databases:



Strong hands-on experience with both SQL (e.g., PostgreSQL, MySQL) and NoSQL (e.g., DynamoDB, MongoDB, Cassandra) databases.

- Messaging/Queues: Proven experience working with message brokers and queues (e.g., Kafka, AWS SQS, RabbitMQ) to manage asynchronous data transfer.

Cloud & Infrastructure:

- Cloud Platform: Extensive experience with AWS cloud services (e.g., S3, EC2, EMR, Lambda, RDS).

- DevOps & CI/CD: Hands-on experience setting up and managing deployment pipelines using GitLab or Jenkins.

- Containerization: Solid understanding and experience deploying applications using Kubernetes (K8s) and Docker.

- Infrastructure as Code (IaC): Familiarity with IaC tools (e.g., Terraform, AWS CloudFormation) to provision cloud resources.

Additional Skills:

- Frontend/UI: Basic understanding of UI development (HTML, CSS, JavaScript, or up-to-date frameworks) to collaborate effectively with full-stack or frontend teams.

- Methodology: Strong practical experience working within and leading Scrum/Agile environments.

📌 Senior Member Technical Staff (Bengaluru)
🏢 The Nielsen
📍 Bengaluru