13 Nov
|
netweb software
|
Vadodara
13 Nov
netweb software
Vadodara
Job highlights
- Build and manage Fabric Data Factory pipelines for batch,scheduled,and incremental data ingestion
- . Bachelor or Master degree in Computer Science,Information Technology,Engineering,Data Science,or a related field
- Preferred Qualifications . Microsoft Certified: Fabric Data Engineer Associate (DP-700) is preferred
- Required Qualifications .
_Title":"Data Engineer", "Is_Locked":false,"City": "Vadodara", "Industry": "Technology", "Job_Description": " We are looking for a Data Engineer with strong hands-on experience in Microsoft Fabric to design, build, optimize, and support scalable data engineering solutions. The role focuses on data ingestion and transformation, Lakehouse and Warehouse development, data pipelines, and reliable delivery of curated, analytics-ready data. The successful candidate will work closely with data analysts, Power BI developers , application teams, and business stakeholders to translate business needs into secure, maintainable, high-performing data solutions.
Requirements Key Responsibilities
Microsoft Fabric & Data Platform Development
Design, develop, and maintain end-to-end data engineering solutions using Microsoft Fabric.
Build and manage Fabric Data Factory pipelines for batch, scheduled, and incremental data ingestion.
Develop and maintain Fabric Lakehouse and Warehouse solutions, using OneLake for centralized data storage and management.
Use Fabric notebooks, Spark, SQL analytics, and Dataflows Gen2 to process and transform data.
Apply Medallion Architecture (Bronze, Silver, and Gold) where appropriate for data organization, quality, and reuse.
Data Ingestion, Transformation & Modeling
Develop robust ETL/ELT processes to ingest data from databases, REST APIs, files, and cloud/SaaS applications.
Implement incremental loads, change data capture (CDC), slowly changing dimensions (SCD), partitioning, and other suitable processing strategies.
Develop and optimize analytical data models, including dimensional models and star/snowflake schemas.
Use SQL, Python, PySpark, and Spark SQL to transform, validate, and prepare data for reporting and analytics.
Implement data quality checks, validation, reconciliation, error handling, and appropriate recovery mechanisms.
Performance, Reliability & Operations
Monitor pipeline execution and data processing; troubleshoot failures and perform root-cause analysis.
Optimize Spark jobs, SQL queries, pipeline design,
and storage usage for performance, scalability, and cost efficiency.
Implement logging, monitoring, alerting, and production-support practices to improve platform reliability.
Support issue resolution and continuous improvement across data pipelines and platform workloads.
Security, Governance & Deployment
Apply appropriate security controls, workspace access, identity and access management, and data access practices with platform and security teams.
Follow data governance, documentation, and engineering standards; support data lineage and governance requirements where applicable.
Use Git-based source control and support CI/CD and deployment processes for Fabric solutions through Azure DevOps or GitHub.
Document data architecture, pipelines, data models, dependencies, operational procedures, and technical decisions.
Collaboration & Business Alignment
Work with Power BI teams to provide curated, trusted, analytics-ready datasets and support semantic-model requirements.
Collaborate with analysts, data scientists, application teams, and business stakeholders to understand requirements and deliver practical solutions.
Contribute to design reviews, estimation, sprint planning, testing, release activities, and continuous improvement in an Agile delivery setting.
Required Qualifications
Experience & Education
3+ years of experience in data engineering, with mandatory hands-on experience developing solutions in Microsoft Fabric.
Bachelor or Master degree in Computer Science, Information Technology, Engineering, Data Science, or a related field; equivalent professional experience may be considered.
Experience building end-to-end data pipelines and supporting data solutions through development, deployment, and production.
Microsoft Fabric Skills
Hands-on experience with Fabric Data Factory / Data Pipelines, Lakehouse, Warehouse, OneLake, notebooks, Spark, SQL analytics, and Dataflows Gen2.
Understanding of Microsoft Fabric architecture, workspaces, and end-to-end data processing patterns.
Ability to develop,
troubleshoot, and optimize production-ready data pipelines in Microsoft Fabric.
Familiarity with Fabric data agents and their potential use in data discovery or conversational access to data.
Core Data Engineering Skills
Strong SQL skills, including complex queries, joins, CTEs, window functions, and query optimization.
Strong programming knowledge in Python and/or PySpark, with practical experience in data transformation and processing.
Solid understanding of ETL/ELT concepts, batch and incremental processing, data warehousing, and dimensional modeling.
Experience with data quality, validation, reconciliation, error handling, and reliable pipeline design.
Working understanding of cloud security, identity, access management, and secure handling of enterprise data.
DevOps & Engineering Practices
Experience with Git and source control, and familiarity with Azure DevOps or GitHub.
Understanding of CI/CD pipelines, automated deployment, workplace management, and release practices.
Experience with monitoring, logging, troubleshooting, and production support.
Preferred Qualifications
Microsoft Certified: Fabric Data Engineer Associate (DP-700) is preferred; Microsoft Certified: Fabric Analytics Engineer Associate (DP-600) is also relevant.
Experience migrating data solutions from Azure Synapse Analytics, Azure Data Factory, Databricks, or traditional data warehouses to Microsoft Fabric.
Experience with Power BI semantic models and DAX.
Knowledge of Microsoft Purview, data governance, data lineage, and compliance practices.
Exposure to real-time analytics, Eventstream, or Real-Time Intelligence in Microsoft Fabric.
Experience integrating REST APIs and SaaS data sources.
Familiarity with Microsoft Azure AI Foundry / Microsoft Foundry and its data or AI-related capabilities.
Experience working in Agile/Scrum teams.
Preferred Candidate Profile
Strong analytical thinking, problem-solving, and attention to data quality.
Able to work independently while collaborating effectively with cross-functional teams.
Clear communication and technical documentation skills.
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
📌 Data Engineer (Vadodara)
🏢 netweb software
📍 Vadodara