Job Overview:

We are seeking an experienced Senior Data Engineer to build and optimize scalable data platforms using Microsoft Fabric, Databricks (and/or Snowflake). The role focuses on designing reliable data pipelines, lakehouse and warehouse models, and semantic layers that enable enterprise analytics, BI, and AI/Gen AI use cases.

You will work closely with analytics, BI, and data science teams to deliver high-quality, performant, and governed data solutions, while driving best practices in data engineering, optimization, and platform design.

Key Responsibilities

- Design, build, and maintain end-to-end data solutions on Microsoft Fabric, Databricks, including Pipelines, Notebooks, Lakehouse, Data Warehouse, and Semantic Models.
- Implement scalable data ingestion, transformation, and loading (ETL/ELT) using Fabric Pipelines and PySpark.
- Develop robust data models and schemas optimized for analytics, reporting, and AI-driven consumption.
- Create and maintain semantic models to support Power BI and enterprise BI solutions.
- Engineer high-performance data solutions that meet requirements for throughput, scalability, quality, and security.
- Author productive PySpark and SQL code for large-scale data transformation, data quality management, and business rule processing.
- Build reusable framework components for metadata-driven pipelines and automation.
- Optimize Lakehouse and Data Warehouse performance, including partitioning, indexing, Delta optimization, and compute tuning.
- Develop and maintain stored procedures and advanced SQL logic for operational workloads.
- Design and prepare feature-ready datasets for AI and GenAI applications.
- Collaborate with data scientists and ML engineers to productionize AI pipelines.
- Implement data governance and metadata practices required for responsible AI.
- Leverage Fabric, Databricks capabilities to orchestrate and monitor AI-related data workflows.
- Apply data governance,



privacy, and security standards across all engineered assets.
- Implement monitoring, alerting, and observability best practices for pipelines and compute workloads.
- Drive data quality initiatives, including validation frameworks, profiling, and anomaly detection.
- Partner with analytics, BI, data science, and product teams to understand requirements and translate them into technical solutions.
- Mentor junior engineers and contribute to engineering standards and patterns.
- Participate in architecture reviews, technical design sessions, and roadmap planning.
- Develop and deliver dashboards and reports using Power BI and Tableau.

Required Skills:

- Bachelor s or Master s degree in Computer Science, Engineering, Data Science, or equivalent experience.
- 7+ years of professional experience in data engineering or a related discipline.
- Mandatory hands-on expertise with Microsoft Fabric, Databricks or Snowflake including:
- Pipelines
- Notebooks
- Lakehouse
- Data Warehouse
- Semantic Models

- Advanced proficiency in PySpark for distributed data processing.
- Strong command of SQL for analytical and operational workloads.
- Experience developing and optimizing stored procedures and complex SQL transformations.
- Understanding of GenAI architectures, vectorization, embedding pipelines, and data preparation for LLM use cases.
- Strong knowledge of data modeling, ETL/ELT patterns, and modern data lakehouse principles.
- Experience designing and optimizing large-scale data pipelines in cloud environments (Azure preferred).
- Excellent problem-solving skills with the ability to analyze complex data workflows.
- Proficiency in creating interactive dashboards and reports using Power BI and Tableau.

Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Senior Data Engineer (Uttar Pradesh)
🏢 Sparix Global
📍 Uttar Pradesh