Must Have Skills:

- Hands-on experience in designing, building, and optimizing data pipelines, data models and PySpark for large-scale data processing and analysis
- Extensive experience and deep expertise in data modeling and data model concepts, particularly with large datasets, ensuring the design and implementation of productive, scalable, and high-performing data models.
- Strong software engineer, you take pride in what youre developing with a robust testing ethos.
- String proficiency in Python programming, with a focus on data processing and analysis
- Extensive experience in designing, building, and optimizing ADB/ PySpark pipelines and architectures, with a strong focus on supporting both batch and real-time data workflows.
- Hand-on Experience with cloud technologies and big data concepts, especially in Azure Databricks, Azure Data Lake and Azure Data Factory
- In-Depth Knowledge of PySpark,



including experience with PySpark performance tuning techniques to optimal processing efficiency
- Strong SQL skills for querying and manipulating large datasets, with experience in optimizing complex queries for performance

Good to Have Skills:

- Experience in automating data workflows and implementing CI/CD pipelines for data applications, ensuring efficient deployment and monitoring
- Any experience / knowledge in PowerShell, bash, Perl scripting is desirable.
- Any Knowledge in BI Reporting tool (eg. Power BI) is a plus
- Experience with Agile Methodology, Scrum Methodology is a plus
- Experience in the financial sector is a plus, but not essential
- Experience with DevOps tools such as GitLab and CI/CD pipelines is a plus

📌 Azure Data Engineer (6 years To 12 years) (Pune)
🏢 Tata Consultancy Services
📍 Pune