22 Oct
|
RapidBrains
|
Bangalore Metropolitan Area
22 Oct
RapidBrains
Bangalore Metropolitan Area
Job Title : Senior Data Engineer
Experience: 6+ Years
Location: Bangalore, Pune, Chennai, Hyderabad
Employment Type: Fulltime
We are looking for an experienced Senior Data Engineer to design, build, and optimize scalable data pipelines within Microsoft Fabric Lakehouse environments. This role is ideal for professionals with robust expertise in enterprise data ingestion, ELT processes, Medallion architecture, and high-volume data processing. You will play a key role in developing robust ingestion and transformation pipelines, optimizing Lakehouse performance, implementing CDC and incremental loading strategies, and delivering scalable data solutions for enterprise environments
Key Responsibilities
Design and implement enterprise data ingestion pipelines into Microsoft Fabric Lakehouse environments.
Configure, optimize, and manage Fivetran connectors for Oracle, SQL Server, and other enterprise data sources.
Build scalable ELT pipelines and manage large-volume data ingestion and backfill operations.
Develop Bronze-to-Silver transformation pipelines following Medallion Architecture best practices.
Implement Change Data Capture (CDC) and incremental data loading strategies.
Optimize Lakehouse performance, storage architecture, and query execution.
Develop monitoring and alerting for record counts, load duration, pipeline failures, and data quality.
Support Dev, Test, and Production deployment processes.
Collaborate with cross-functional teams to deliver secure, scalable, and reliable data solutions.
Required Qualifications
6+ years of experience in Data Engineering.
Hands-on experience with Microsoft Fabric or Azure Synapse Analytics / Azure Data Factory.
Robust expertise in Fivetran or similar ELT tools.
Experience processing high-volume datasets (hundreds of millions of records).
Solid proficiency in SQL, Python, and data modeling.
Solid understanding of Medallion Architecture and Lakehouse concepts.
Experience building scalable, enterprise-grade data pipelines.
📌 Senior Data Engineer Bangalore Metropolitan Area
🏢 RapidBrains
📍 Bangalore Metropolitan Area