Purpose of the Role The role holder will be responsible for designing, implementing, and maintaining data pipelines and automation solutions on the Azure cloud infrastructure. The role will involve developing data pipelines, managing databases, and creating data warehouses and data lakes to support various project requirements. The role holder will work closely with business stakeholders, vendor teams, and other technical colleagues to deliver robust data-driven solutions.

Solid proficiency in Python, PySpark, Scala, PowerShell, and database management is essential for success in this role.

Job Responsibility

- Designing data pipelines on Azure cloud infra
- Developing automations using Python, PySpark, Scala and PowerShell etc.
- Designing data pipelines, data flows, database for various project requirement
- Database management for SQL and NO-SQL databases
- Developing and maintaining data warehouse and data lake in Azure
- Developing automated solution for data loads with secure pipelines
- Work with business and vendor team to deliver data drives solution for projects
- Creating documentation for all standard process
- Data Standardization & transformations, Data Sanitization and clean up

Stakeholder Profiles & Nature of Interactions

Internal

IT Operations Team: Coordination with the IT operations team to ensure smooth deployment, monitoring, and management of data engineering solutions on Azure cloud infrastructure.

Security and Compliance Team: Working closely with security and compliance teams to ensure that data engineering solutions comply with regulatory requirements and security standards, including data encryption, access controls, and data governance.

External

Consultants:



Engaging with consultants or external experts for specialized knowledge or assistance in designing and implementing complex data engineering solutions on Azure cloud infrastructure.

Regulatory Bodies: Interacting with regulatory bodies or industry organizations to stay informed about regulatory requirements, best practices, and emerging trends in data management, privacy, and security.

Desired Candidate Profile

- Qualification: - B.E./B.Tech
- Experience: - 3 to 8years
- Degree in Engineering, Computer Science or other technical discipline
- 4+ years of relevant work experience as a data engineer and solving business challenges is preferred
- Strong understanding of Azure services like Azure data factory, Synapse, Databricks, Azure functions etc. and their configurations
- Required Data Engineer with good exposure on data technologies
- Candidate should have knowledge on Big data, RDBMS, Data warehouse
- Ability to develop analytical approaches to answer high-level questions and develop insightful recommendations
- Exceptional attention to detail and ability to work effectively with minimal supervision
- Should have certification in Azure

Skills & Competencies

- Experience working on Linux and Windows based infrastructure
- Microsoft Azure cloud technologies knowledge must have
- Expertise in Azure data factory, Synapse, Databricks, Azure functions, Azure Data lake and Data warehouse etc.
- RDBMS - MSSQL, MySQL, Oracle
- Worked on/with SAP/CRM/SAP-BW/pre-formatted data
- Strong Proficiency in programming in Python, PySpark, PowerShell, Shell script
- Hadoop basic understanding, understanding of file system, Storage technologies
- Understanding of JSON, YAML and XML

📌 Senior Manager-E-Services ( Spares Parts ) (Thane)
🏢 Tata Motors
📍 Thane