19 Nov
|
Brillio
|
Bengaluru
Job Summary
Senior Lead Data Engineer
Primary Skills
- AKS
- Event Hub
- Azure DevOps
- Cosmos DB
- Azure Functions
Specialization
- Azure Data Engineering Advanced: Senior Data Engineer
Job Requirements
- Experience: 5-8 years
- Locations: Bangalore | Hyderabad | Pune | Chennai | Gurgaon
- Employment Type: Full-time
What You'll Do
- Lead the technical implementation of data modernization and migration projects, moving legacy SQL databases, ETL workloads, and data processing solutions to Microsoft Fabric.
- Analyze existing SQL, ETL, and data integration workloads to identify migration strategies, dependencies, complexity, performance challenges, and optimization opportunities.
- Design and develop scalable and reliable data pipelines using Microsoft Fabric Data Factory, notebooks, and related Fabric services.
- Build modern data engineering solutions using Fabric Lakehouse, Warehouse and OneLake.
- Develop and optimize data transformation and processing logic using SQL, Python and PySpark.
- Design reusable engineering components and establish standards around code organization, naming conventions, logging, monitoring, and deployment practices.
- Troubleshoot complex pipeline failures, data quality issues, and production performance bottlenecks.
- Implement efficient ingestion patterns, including incremental data loads and Change Data Capture where required.
- Work with engineering and analytics teams to ensure reliable integration between data platforms and reporting solutions.
- Conduct code reviews, mentor other data engineers, and drive engineering best practices across the team.
What We're Looking For
Must-Have Skills
- 5-8 years of overall experience in Data Engineering.
- At least 3 years of hands-on experience in Azure Data Engineering and data migration/modernization projects.
- Strong hands-on expertise with Microsoft Fabric Data Engineering
- Experience with Microsoft Fabric Data Factory, Lakehouse, Warehouse and OneLake.
- Advanced SQL skills for data transformation, migration, integration, and performance optimization.
- Strong programming experience in Python and PySpark.
- Hands-on experience migrating legacy ETL and SQL-based workloads to Azure/Fabric environments.
- Experience with Azure DevOps for CI/CD and deployment automation.
- Working knowledge of Git and collaborative source-control practices.
- Good understanding of Microsoft Fabric workspaces and modern data engineering best practices.
- Solid troubleshooting and problem-solving capabilities in production data environments.
Good to Have
- Experience with data modeling, dimensional modeling, and data partitioning strategies.
- Knowledge of data quality, security, and data governance practices within Azure environments.
- Experience with monitoring, logging, and performance tuning of large-scale data pipelines.
- Hands-on knowledge of Change Data Capture (CDC) and incremental ingestion patterns.
- Familiarity with Power BI and working closely with BI/analytics teams.
- Experience providing technical guidance or mentoring junior/mid-level Data Engineers.
Education Preferred Certifications
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline.
- Microsoft Azure/Data Engineering certification is preferred.
- Microsoft Fabric certification, such as Microsoft Certified: Fabric Analytics Engineer Associate, is an added advantage.
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
📌 Sr Fabric Lead Data Engineer945 (Bengaluru)
🏢 Brillio
📍 Bengaluru