Description
Leads the design, development, deployment, and maintenance of data and analytics platforms. Develops reliable, scalable, and effective data pipelines and data processing solutions that enable data to be effectively processed, stored, governed, and made available to analysts and other data consumers. Collaborates with business stakeholders, IT experts, data scientists, architects, and subject-matter experts to deliver enterprise data and analytics solutions aligned with business and technical requirements.
Key Responsibilities
- Design, develop, and automate distributed data ingestion and transformation solutions using data from relational, event-based, semi-structured, and unstructured sources.
- Build reliable, scalable, and efficient ETL/ELT data pipelines using appropriate tools, technologies, and scripting languages.
- Design and implement data quality, validation, monitoring, and alerting frameworks to identify and resolve data integrity issues.
- Implement data governance practices covering metadata, data access, retention, compliance, and security.
- Design and implement physical data models, including database structures, indexing, and table relationships, to support performance and scalability.
- Develop and operate large-scale data storage and processing solutions across cloud and distributed data platforms, including data lakes, warehouses, and lakehouse environments.
- Optimize data pipelines, Spark workloads, databases, and cloud infrastructure for performance, reliability, scalability, and cost efficiency.
- Integrate data from a variety of enterprise applications and source systems and support real-time and event-driven data processing.
- Develop automation for common and repeatable data preparation, integration, deployment, and platform-management activities to minimize manual and error-prone processes.
- Implement CI/CD and DevOps practices to support automated deployment, testing, and release management.
- Participate in troubleshooting
📌 Data Scientist (Pune)
🏢 Cummins
📍 Pune