We are looking for a hands-on Lead Data Engineer to design, build, and operate a cloud-native, event-driven Service Layer platform supporting enterprise Quote-to-Cash (QTC), customer lifecycle, and finance integrations.

The role requires strong expertise in Golang, Python, AWS, Kafka, distributed systems, microservices, and enterprise integration. The candidate will provide technical leadership to engineers while remaining deeply involved in architecture, development, delivery, production support, and platform reliability.

Key Responsibilities

- Own end-to-end technical delivery of key engineering and integration initiatives.
- Lead and mentor engineering teams through technical guidance, task planning, code reviews, and delivery execution.
- Design scalable event-driven and asynchronous integration platforms using Golang, Python, AWS, and Kafka.
- Build and enhance cloud-native microservices and orchestration workflows for enterprise integrations.
- Design canonical data models, data contracts, transformation frameworks, and validation mechanisms.
- Implement Kafka-based solutions supporting retries, idempotency, sequencing, replay, DLQs, and fault recovery.
- Design and implement ETL, migration, reconciliation, and data-quality solutions using AWS Glue, Step Functions, Snowflake, PostgreSQL, and ClickHouse.
- Develop secure and scalable REST APIs and enterprise system interfaces.
- Design highly available AWS applications with strong focus on scalability, performance, security, and reliability.
- Implement monitoring, alerting, dashboards, and end-to-end observability using CloudWatch, Grafana, Datadog, Prometheus, or OpenTelemetry.




- Drive engineering standards across CI/CD, automated testing, security, observability, resiliency, and cloud-native architecture.
- Partner with architects, product managers, enterprise data teams, and business stakeholders to define technical roadmaps.
- Lead architecture/design reviews, code reviews, technical governance, and continuous improvement initiatives.
- Support production systems, perform incident management and RCA, and drive permanent resolution of technical issues.

Must-Have Skills

- 10+ years of software engineering experience with strong experience in cloud-native, data engineering, integration, or distributed platforms.
- 5+ years of hands-on Golang experience in production environments.
- Strong hands-on Python development experience.
- 5+ years of AWS experience designing and implementing cloud-native applications.
- 5+ years of production experience with Kafka or equivalent messaging technologies such as AWS MSK/RabbitMQ.
- Robust knowledge of distributed systems, microservices, and event-driven architecture.
- Solid experience with AWS services including:
Lambda, API Gateway, S3, RDS/Aurora PostgreSQL, DynamoDB, SQS, SNS, EventBridge, IAM, CloudWatch, and VPC.
- Strong experience with PostgreSQL, DynamoDB/NoSQL, and ClickHouse,



including data modelling and performance optimization.
- Experience designing RESTful APIs and enterprise integration services.
- Strong understanding of canonical data models, data contracts, transformation, and validation frameworks.
- Hands-on experience with CI/CD, automated testing, code quality, and secure software delivery.
- Experience with tools such as GitLab CI, Jenkins, SonarQube, Trivy, or Semgrep.
- Solid observability experience with CloudWatch, Grafana, Datadog, Prometheus, or OpenTelemetry.
- Strong understanding of cloud security including IAM, OAuth2, JWT, encryption, Secrets Manager, vulnerability management, and audit logging.
- Strong production support, troubleshooting, RCA, resiliency, and performance-tuning experience.
- 2+ years of experience as a Technical Lead / Engineering Lead, including team mentoring and technical delivery ownership.

Qualifications

- Bachelor's degree in Computer Science, Engineering, Information Technology, or a related discipline.
- Proven experience designing and operating high-availability, business-critical enterprise integration platforms.
- Robust communication, problem-solving, stakeholder management, and technical leadership skills.

Good to Have

- AWS certifications such as AWS Certified Data Engineer, Solutions Architect, or Developer.
- Experience in Quote-to-Cash, CRM, ERP, Customer Lifecycle Management, Finance, or Enterprise Data domains.
- Experience with large-scale enterprise integration and orchestration platforms.
- Knowledge of cloud-native security and DevSecOps practices.

📌 Lead Data Engineer_Golang/AWS/Python (Bengaluru)
🏢 Zorba Consulting India
📍 Bengaluru