Key Responsibilities
Design, develop, and optimize data solutions using Microsoft Fabric and PySpark.
Analyze and tune Spark workloads for performance, scalability, and cost efficiency.
Develop and maintain Lakehouse architectures for enterprise analytics platforms.
Optimize data ingestion, transformation, and processing pipelines.
Perform Spark code reviews and recommend performance improvements.
Implement partitioning, caching, indexing, and file optimization techniques.
Monitor and troubleshoot Fabric workloads, Spark jobs, and resource utilization.
Collaborate with data architects, data scientists, and business stakeholders.
Design robust ETL/ELT frameworks using Fabric Data Factory and Notebooks.
Ensure data quality, governance, security, and compliance standards.
Implement best practices for distributed computing and big data processing.
Create technical documentation, optimization reports, and performance metrics dashboards.

Required Skills

Microsoft Fabric
Microsoft Fabric
Fabric Data Engineering
Fabric Lakehouse
Fabric Data Factory




Fabric Notebooks
Fabric Real-Time Analytics
Fabric OneLake
Fabric Warehouse

PySpark &
Spark Optimization
PySpark Development
Apache Spark Architecture
Spark SQL
Spark DataFrames
Spark Structured Streaming
DAG Optimization
Catalyst Optimizer
Adaptive Query Execution (AQE)
Broadcast Joins
Partitioning &
Bucketing
Shuffle Optimization
Caching &
Persistence Techniques

Data Engineering
ETL/ELT Development
Data Modeling
Data Lake Architecture
Medallion Architecture
Delta Lake
Data Quality Frameworks

Programming
Python
PySpark
SQL
Scala (Preferred)

Databases
SQL Server
Azure SQL
PostgreSQL
Snowflake (Preferred)

Cloud &
Analytics
Azure Data Services
Azure Data Lake Storage (ADLS)
Power BI
Azure Monitor
Log Analytics

DevOps &
Automation
Git
Azure DevOps
CI/CD Pipelines
Infrastructure as Code (Preferred)

📌 Ms Fabric + Pyspark Tuning S Bengaluru (India)
🏢 Tata Consultancy Services
📍 India