Role description:
- Job Description - Senior Data Engineer (Azure Databricks | Fabric | PySpark)
Position
Senior Data Engineer
Experience
5+ Years
Primary (Must-Have) Skills
- Azure Databricks
- PySpark
- Python
- Microsoft Fabric
- Azure Data Factory (ADF)
- Azure Synapse Analytics
- Informatica
- ETL/Data Engineering
- SQL
- Data Quality Frameworks
- DataOps
- CI/CD Pipeline implementation
- Solution Design & Architecture
- Strong communication and stakeholder management skills
Secondary Skills
- Apache Spark
- Azure Cloud Services
- Data Modeling
- Performance Optimization
- Azure DevOps
- Git
- Agile/Scrum methodology
- JavaScript (basic knowledge is acceptable)
Roles & Responsibilities
- Design, develop, and optimize scalable data pipelines using Azure Databricks, PySpark, and Python.
- Build and maintain ETL/ELT workflows using Azure Data Factory, Synapse, Informatica, and Microsoft Fabric.
- Develop robust data engineering solutions to support business intelligence and analytics initiatives.
- Ensure data quality, integrity, governance,
and monitoring across data platforms.
- Implement DataOps best practices and automate deployments using CI/CD pipelines.
- Translate functional and business requirements into technical solutions and development tasks.
- Collaborate with business stakeholders, product owners, architects, and cross-functional teams.
- Troubleshoot production issues, optimize performance, and ensure high availability of data solutions.
- Participate in solution design discussions, technical reviews, and estimation activities.
- Follow coding standards, documentation practices, and industry best practices.
Positive-to-Have Skills
- Domain experience in Manufacturing or Retail
- Experience with Azure DevOps
- Knowledge of Delta Lake and Lakehouse architecture
- Experience with Data Warehousing concepts
- Exposure to Power BI
- Experience with cloud migration projects
- Certification in Azure Data Engineering or Databricks
- Knowledge of AI/ML d
📌 Lead I (Hyderabad)
🏢 UST
📍 Hyderabad