Role: Data/DB Engineer Lead (AWS / 5+ years of experience)
Location: Ahmedabad, Pune
Years of Experience: 9 to 15 yrs
Notice: Looking for 0-30 Days Joiner
Introduction
Our team builds a data platform that reliably collects, processes, and retrieves large-scale IoT data based on AWS. We are seeking a senior engineer who can work on unobstructed flow even among billions of data records, push data storage performance to the limit, and technically efficiently manage service growth.
Key Responsibilities
High-availability data pipeline design: AWS MSK (Kafka), end-to-end data streaming architecture design and operation using Logstash
Storage optimization and sharding strategies: Optimizing large-scale search performance through OpenSearch's index design, sharding, and lifecycle (ISM) management
Data Modeling and Partitioning: Table partitioning and data modeling in PostgreSQL/MySQL, architecture guide for efficient querying
Performance Tuning and Guidance: Perform SQL optimization across services and spreading technical best practices to backend development teams
Release Management:
Review deployment details and validate details before deployment to ensure zero-error deployments without roll back needs.
Infrastructure automation: DB schema versioning (such as Flyway) and deployment process automation systems
Requirements
Over 5 years of experience in data engineering or related backend development
Extensive experience in operating infrastructure handling large-scale traffic in AWS cloud environments
Solid capabilities in building real-time data pipelines and troubleshooting using Kafka
Experience in OpenSearch (Elasticsearch)
Deep understanding of the internal structure of RDBMS (PostgreSQL/MySQL) and ability to tune complex queries
Preferred Qualifications
Seamless technical collaboration capabilities: Communication skills to discuss infrastructure optimization with DevOps engineers and provide efficient data modeling and query guides to backend developers
Global co