Job Description

Key Responsibilities
Technical Leadership: Lead the technical design, architecture, and implementation of high-throughput data ingestion systems and ETL pipelines.
Team Mentorship: Guide, mentor, and elevate a team of Engineers, fostering a culture of technical excellence and best practices.
Hands-on Development: Write clean, scalable, and maintainable code primarily in Java and Python.
Performance Optimization: Proactively monitor and resolve complex issues related to high memory and CPU consumption, bottlenecks in data transfer, and queue processing.
Agile Delivery: Partner with product managers and scrum masters to drive Agile/Scrum ceremonies, sprint planning, and timely delivery of features.
End-to-End Ownership: Oversee the full software development lifecycle, from system design and data formatting/analytics to deployment and infrastructure management.

Technical Requirements

Core Backend & Data Engineering:
Languages: Expert-level proficiency in Java and Python.
Data & Big Data: Deep understanding of Apache Spark, ETL pipeline design, data formatting, and data analytics.
Databases:



Solid hands-on experience with both SQL (e.g., PostgreSQL, MySQL) and NoSQL (e.g., DynamoDB, MongoDB, Cassandra) databases.
Messaging/Queues: Proven experience working with message brokers and queues (e.g., Kafka, AWS SQS, RabbitMQ) to manage asynchronous data transfer.

Cloud & Infrastructure:
Cloud Platform: Extensive experience with AWS cloud services (e.g., S3, EC2, EMR, Lambda, RDS).
DevOps & CI/CD: Hands-on experience setting up and managing deployment pipelines using GitLab or Jenkins.
Containerization: Solid understanding and experience deploying applications using Kubernetes (K8s) and Docker.
Infrastructure as Code (IaC): Familiarity with IaC tools (e.g., Terraform, AWS CloudFormation) to provision cloud resources.

Additional Skills:
Frontend/UI: Basic understanding of UI development (HTML, CSS, JavaScript, or up-to-date frameworks) to collaborate effectively with full-stack or frontend teams.
Methodology: Robust practical experience working within and leading Scrum/Agile environments.

📌 Senior Member Technical Staff Bengaluru (India)
🏢 The Nielsen
📍 India