Job Summary

Software Engineering Lead - Data Platform

We are seeking a highly skilled Software Engineering Lead - Data Platform to lead the development of Caterpillar's next-generation Digital Manufacturing Data Platform. This platform enables large-scale data ingestion, transformation, and analytics across manufacturing, supply chain, and engineering ecosystems.

The ideal candidate will bring deep expertise in data engineering, large-scale data ingestion, AWS-based architectures, and Snowflake, along with robust leadership and software engineering discipline.
Key Responsibilities
Leadership Delivery
Lead and mentor a team of data engineers and platform developers
Drive Agile execution and ensure predictable, high-quality delivery
Establish engineering best practices, code quality, and CI/CD standards
Data Platform Architecture
Architect scalable and secure data platforms on AWS
Design robust data ingestion frameworks for batch and near real-time pipelines
Define best practices in data modeling, governance, and metadata management
Data Engineering Ingestion




Lead design and development of scalable ingestion pipelines (structured and unstructured data)
Build and optimize Snowflake-based data platforms for performance and cost
Enable ingestion of diverse sources (databases, APIs, files, streaming data)
Cloud Platform Engineering
Leverage AWS services (S3, Glue, Lambda, EMR, Redshift, etc.) for end-to-end pipelines
Implement CI/CD pipelines using Azure DevOps / Jenkins
Ensure system scalability, resiliency, and operational readiness
Software Engineering Excellence
Enforce software engineering principles (modular design, code quality, testing, version control)
Drive automation and continuous improvement
Promote reusable frameworks for ingestion and transformation
Stakeholder Collaboration
Partner with product managers, SMEs, and business stakeholders
Translate business needs into scalable data solutions
Must-Have Skills
10+ years of

📌 Lead Software Engineer Data Engineer Bengaluru
🏢 Caterpillar
📍 Bengaluru