Job Summary :

We are seeking a highly skilled Senior Data Engineer to design, build, and optimize our enterprise data platform and analytics solutions. In this role, you will serve as a key hands-on engineering track lead responsible for transforming complex business requirements into scalable, production-grade data pipelines and Lakehouse architectures.

Key Responsibilities :

- Data Pipeline &
- Lakehouse Engineering : Design, implement, and maintain high-performance batch and real-time streaming data pipelines using Python, PySpark, and Spark SQL on Azure Databricks and Microsoft Fabric. Build multi-tier Medallion architectures (Bronze, Silver, Gold) using Delta Lake principles.
- Microsoft Fabric Platform Delivery : Leverage Fabric capabilities (OneLake, Direct Lake, Lakehouses, and Data Factory Gen2) to streamline zero-copy ingestion, optimize semantic models, and eliminate redundant ETL pipelines.
- API &
- Systems Integration : Build resilient API connectors, CDC workflows, and data integration pipelines to ingest data from heterogeneous sources.
- AI &
- Analytics Enabling : Structure, clean, and optimize lakehouse data layers for downstream Power BI reporting as well as AI/ML applications, ensuring data is clean, validated, and structured for AI grounding.
- Data Quality &
- Governance :



Implement robust data validation, access controls, data lineage, and metadata management using Unity Catalog, Fabric Governance, and automated testing frameworks.
- Technical Mentorship &
- CI/CD : Perform thorough code reviews, enforce engineering best practices, optimize query performance, and manage automated deployment pipelines.

Technical Skills :

- Core Development : Advanced hands-on proficiency in Python, PySpark, and SQL (Spark SQL / T-SQL).
- Databricks Platform : Robust hands-on experience with Azure Databricks (Delta Lake, Unity Catalog, Delta Live Tables, Auto Loader) and Genie.
- Microsoft Fabric : Demonstrated experience building pipelines and workspace items within Microsoft Fabric.
- Cloud Infrastructure : Solid expertise across Azure Data Lake Storage Gen2 (ADLS Gen2), Azure Data Factory (ADF), Azure Key Vault, and Azure SQL/Synapse.
- AI Data Concepts : Strong conceptual and practical understanding of data engineering requirements for AI/ML workloads including vector embeddings, RAG data ingestion, and feature stores.

Education :

- Bachelor's or Master's degree in Computer Science, Information Technology, or a related quantitative field.

📌 Straive - Senior Data Engineer - Azure Databricks (India)
🏢 Straive
📍 India