Senior / Lead Data Engineer – Service Layer / QTC Integration Platform Role Details
• Role: Senior / Lead Data Engineer
• Primary

Skills:
Go / Golang, Python, AWS, Kafka / MSK, Microservices, Distributed Systems, Data Engineering
• Platform Exposure: AWS Serverless, Kafka, Snowflake, PostgreSQL, ClickHouse, Event-Driven Architecture
• Domain Exposure: Quote-to-Cash, CRM, ERP, Customer Lifecycle, Enterprise Data preferred
•

Experience:
8+ years
• Location/ Mode: Bengaluru/Hybrid
• Budget: 27 LPA
• Contract Duration: 12 months Role Overview We are looking for a strong, hands-on, and highly autonomous Senior / Lead Data Engineer to design, build, and operate the Service Layer platform powering enterprise Quote-to-Cash (QTC), customer lifecycle, finance, and enterprise application integrations. This role requires deep expertise in distributed systems, event-driven architecture, microservices, data engineering, cloud-native development, and platform reliability. The candidate will also provide technical leadership to a team of engineers, driving solution design, engineering standards, delivery execution, and operational support across multiple integration initiatives. The platform is built using AWS, Kafka / AWS MSK, Go, Python, canonical data models, asynchronous messaging, and cloud-native architecture patterns, processing high-volume, business-critical events with strong focus on reliability, security, scalability, observability, and operational readiness.

Key Responsibilities
Technical Delivery & Engineering Leadership
• Own end-to-end technical delivery of key initiatives, ensuring solutions are delivered with quality, scalability, security, and operational readiness.
• Lead and mentor a team of engineers through task planning, technical guidance, code reviews, design reviews, and delivery management.
• Drive engineering best practices across software development, CI/CD, observability, resiliency, security, and cloud-native architecture.
• Manage multiple concurrent initiatives while balancing business priorities, technical debt, and platform evolution. Architecture & Platform Design
• Drive architecture and design decisions for event-driven integrations, canonical data models, orchestration frameworks, and platform capabilities.
• Define and enforce engineering standards, secure-by-design principles, cloud best practices, data governance, and non-functional requirements.
• Design and build scalable Service Layer foundation capabilities including messaging, orchestration, observability, reconciliation, data migration, operational tooling, and canonical data management.
• Ensure platform design supports resilience, scalability, fault tolerance, performance, and supportability. Event-Driven Integration & Microservices
• Design, develop, and operate real-time event-driven integration services using Go / Golang and Python.
• Build and enhance the Service Layer platform to orchestrate Quote-to-Cash workflows across enterprise applications.




• Design and implement scalable Kafka / AWS MSK-based event processing with support for:
• Retries
• Idempotency
• Sequencing
• Message replay
• Fault recovery
• Dead-letter queues
• Eventual consistency
• Build secure and scalable RESTful APIs, integration services, and enterprise system interfaces. Data Engineering, Migration & Reconciliation
• Design and implement large-scale ETL, migration, reconciliation, and data-quality solutions using:
• AWS Glue
• Step Functions
• Snowflake
• PostgreSQL
• ClickHouse
• Build and enforce canonical data models, data contracts, transformation layers, and validation frameworks.
• Support enterprise integrations by standardizing communication across multiple business systems.
• Ensure data accuracy, traceability, consistency, and operational reliability across the platform. AWS Cloud-Native Engineering
• Design and build cloud-native applications on AWS using:
• AWS Lambda
• API Gateway
• S3
• RDS / Aurora PostgreSQL
• DynamoDB
• SQS
• SNS
• EventBridge
• IAM
• CloudWatch
• VPC
• Build high-availability, business-critical applications with strong focus on performance, security, and operational supportability. Observability, Monitoring & Operations
• Design and implement observability, monitoring, alerting, and operational dashboards for end-to-end workflow visibility.
• Work with tools such as:
• CloudWatch
• Grafana
• Datadog
• Prometheus
• OpenTelemetry
• Support production operations including incident management, root-cause analysis, troubleshooting, and performance optimization.
• Drive metrics-driven improvements for reliability, scalability, and platform performance. Collaboration & Stakeholder Management
• Partner with Product Managers, Architects, Enterprise Data teams, Engineering, DevOps, and business stakeholders.
• Define technical roadmaps, execution plans, dependencies, and delivery milestones.
• Communicate technical risks, trade-offs, design decisions, and mitigation plans clearly to stakeholders.
• Collaborate across multiple teams to ensure integration initiatives are delivered efficiently. Must-Have Skills Leadership & Delivery
• Proven experience leading and mentoring engineering teams.
• Strong experience in task planning, technical guidance, code reviews, and delivery management.
• Ability to drive architecture decisions for large-scale distributed systems and enterprise integration platforms.
• Experience managing multiple parallel technical initiatives. Programming & Backend Engineering
• Strong hands-on experience in:
• Go / Golang
• Python
• Experience building scalable backend services,



event-driven integrations, orchestration workflows, and data processing solutions.
• Strong understanding of REST API development and secure API design. AWS Cloud & Serverless
• Strong experience designing and building cloud-native applications on AWS.
• Hands-on experience with:
• Lambda
• API Gateway
• S3
• RDS / Aurora PostgreSQL
• DynamoDB
• SQS / SNS
• EventBridge
• IAM
• CloudWatch
• VPC
• Strong understanding of AWS serverless architecture patterns. Kafka / Event-Driven Systems
• Strong experience designing and operating event-driven and asynchronous systems using:
• Kafka
• AWS MSK
• RabbitMQ or equivalent messaging technologies
• Solid understanding of:
• Fault tolerance
• Idempotency
• Retry patterns
• Dead-letter queues
• Message replay
• Eventual consistency
• Sequencing
• Event orchestration Databases & Data Engineering
• Strong experience with:
• PostgreSQL
• ClickHouse
• Relational databases
• DynamoDB / NoSQL databases
• Strong knowledge of:
• Data modelling
• Query optimization
• Performance tuning
• Data validation
• Reconciliation frameworks
• Experience with AWS Glue, Step Functions, Snowflake, and large-scale data workflows. CI/CD, Security & Observability
• Experience with CI/CD pipelines, automated testing, code quality, and secure delivery using tools such as:
• GitLab CI
• Jenkins
• SonarQube
• Trivy
• Semgrep
• Strong understanding of cloud security and secure-by-design principles including:
• IAM
• OAuth2
• JWT authentication
• Encryption at rest and in transit
• Secrets Manager
• Vulnerability management
• Audit logging
• Compliance controls Required Qualifications
• Bachelor’s degree in Computer Science, Engineering, Information Technology, or related discipline.
• 8+ years of software engineering experience.
• At least 4+ years focused on cloud-native, data engineering, integration, or distributed platforms.
• 4+ years building cloud-native solutions on AWS.
• 4+ years working with Go / Golang, Python, AWS services, and Kafka in production environments.
• Proven experience designing, building, and operating highly available enterprise integration and event-driven platforms.
• Strong experience in production support, incident management, root-cause analysis, and performance optimization. Nice to Have
• AWS certifications such as:
• AWS Certified Data Engineer – Associate
• AWS Certified Solutions Architect – Associate
• AWS Certified Developer – Associate
• Equivalent AWS certifications
• Experience in:
• Quote-to-Cash
• CRM
• ERP
• Customer Lifecycle Management
• Enterprise Data domains
• Exposure to large-scale finance, billing, or enterprise integration transformation programmes.
• Experience building operational dashboards and platform-level reporting for engineering and business teams.

📌 Senior / Lead Data Engineer – Service Layer / QTC Integration Platform (India)
🏢 TALPRO INDIA PRIVATE
📍 India