Job Summary

Software Engineering Lead - Data Platform

We are seeking a highly skilled Software Engineering Lead - Data Platform to lead the development of Caterpillar's next-generation Digital Manufacturing Data Platform. This platform enables large-scale data ingestion, transformation, and analytics across manufacturing, supply chain, and engineering ecosystems.

The ideal candidate will bring deep expertise in data engineering, large-scale data ingestion, AWS-based architectures, and Snowflake, along with robust leadership and software engineering discipline.
Key Responsibilities
Leadership Delivery
- Lead and mentor a team of data engineers and platform developers
- Drive Agile execution and ensure predictable, high-quality delivery
- Establish engineering best practices, code quality, and CI/CD standards
Data Platform Architecture
- Architect scalable and secure data platforms on AWS
- Design robust data ingestion frameworks for batch and near real-time pipelines
- Define best practices in data modeling, governance, and metadata management
Data Engineering Ingestion




- Lead design and development of scalable ingestion pipelines (structured and unstructured data)
- Build and optimize Snowflake-based data platforms for performance and cost
- Enable ingestion of diverse sources (databases, APIs, files, streaming data)
Cloud Platform Engineering
- Leverage AWS services (S3, Glue, Lambda, EMR, Redshift, etc.) for end-to-end pipelines
- Implement CI/CD pipelines using Azure DevOps / Jenkins
- Ensure system scalability, resiliency, and operational readiness
Software Engineering Excellence
- Enforce software engineering principles (modular design, code quality, testing, version control)
- Drive automation and continuous improvement
- Promote reusable frameworks for ingestion and transformation
Stakeholder Collaboration
- Partner with product managers, SMEs, and business stakeholders
- Translate business needs into scalable data solutions
Must-Have Skills
- 10+ years of

📌 Lead Software Engineer- Data Engineer (Bengaluru)
🏢 Caterpillar
📍 Bengaluru