17 Nov
|
Azistinc
|
India
We're Hiring | Senior Data Engineer – Microsoft Azure Data Platform | Remote | Full-time | 6–10 Years Experience
- At AZIST, we work at the intersection of innovation and enterprise. We're looking for a seasoned Senior Data Engineer with deep expertise in the Microsoft Azure data ecosystem — including Azure Databricks, Azure Synapse Analytics, and Microsoft Fabric. If you're passionate about designing robust data platforms, building enterprise-grade pipelines with PySpark, and driving data governance and analytics at scale — this role is for you.
What's in it for you
- Industry-best compensation package + performance bonus tied to real outcomes
- Fully remote, your home is your office
- Medical insurance covering you and your family
- Onsite travel opportunities to client locations as needed
- Hands-on work with the latest Microsoft Azure data technologies, including Azure Databricks and Microsoft Fabric
- High visibility, real ownership, and fast growth opportunities
What you'll do
- Architect, design, and maintain scalable data pipelines and ETL/ELT workflows using Azure Data Factory, Synapse Pipelines, and Fabric Data Factory
- Build and optimize large-scale data processing and transformation workloads using PySpark on Azure Databricks, leveraging Delta Lake, Unity Catalog, Auto Loader, and Databricks Workflows
- Design and implement up-to-date lakehouse solutions on Microsoft Fabric using OneLake, Lakehouses, Warehouses, Data Pipelines, Notebooks, and Direct Lake semantic models
- Lead the design and management of data storage solutions across Azure Data Lake Storage (ADLS Gen2), Azure Synapse Analytics, Azure SQL, and Cosmos DB
- Own analytics and telemetry solutions using Azure Data Explorer and Kusto Query Language (KQL)
- Design and develop Python and PySpark scripts for data processing, transformation, and ML model integration
- Design and govern data models and warehouse structures optimized for analytics and reporting at scale. Own data governance, lineage tracking, and privacy compliance including GDPR and contact permission management
- Manage secure access and credentials through Azure Key Vault and Managed Identity
- Design and manage CI/CD pipelines in Azure DevOps (multi-stage YAML pipelines, Repos, Boards) to automate testing and deployment of ADF, Synapse, Databricks, and Fabric artifacts across dev, test, and production environments
- Lead code reviews, mentor junior data engineers, and drive data engineering best practices
- Participate in sprint planning, story delivery, and Agile ceremonies using ADO Boards
- Adhere to and champion Microsoft engineering standards including OneBranch, CloudBuild, LiveSite DRI, and incident management practices
- Drive AI and Copilot driven data initiatives with working knowledge of Copilot Studio and Azure AI Foundry
What you bring
- 5–10 years of data engineering experience within the Microsoft Azure ecosystem
- Deep hands-on expertise with Azure Data Factory for pipeline orchestration and ETL/ELT workflows
- Strong proficiency with Azure Data Lake Storage (ADLS Gen2), Azure Synapse Analytics, Azure SQL, and Cosmos DB
- Hands-on experience with Azure Synapse Analytics — dedicated and serverless SQL pools, Spark pools, Synapse Pipelines, and performance tuning (distribution, partitioning, indexing)
- Strong hands-on experience with Azure Databricks — Delta Lake, Unity Catalog, cluster and job optimization, Databricks Workflows, and Delta Live Tables
- Advanced PySpark proficiency for distributed data processing — DataFrame API, Spark SQL, joins and window functions, partitioning, and performance tuning
- Hands-on experience with Microsoft Fabric — OneLake, Lakehouse, Warehouse, Data Factory pipelines, Notebooks, Direct Lake, and Fabric capacity and workspace management
- Proven working knowledge of Azure Data Explorer and Kusto Query Language (KQL) for analytics and telemetry
- Proven Power BI report development experience including DAX for calculated measures and advanced reporting
- Deep understanding of data modeling and data warehouse design principles
- Proven experience designing and managing Azure DevOps CI/CD pipelines (multi-stage YAML), Repos, Boards, and Artifacts, including automated deployment of ADF, Synapse, Databricks (Asset Bundles), and Fabric (Git integration and deployment pipelines)
- Familiarity with Infrastructure as Code (Bicep or Terraform), Git branching strategies, and automated testing for data pipelines
- Demonstrated ability to lead technical teams, conduct code reviews, and mentor data engineers
- Deep understanding of Agile/Scrum delivery: sprint planning, ADO work items, story delivery, and retrospectives
- Working knowledge of AI/Copilot tools: Copilot Studio and Azure AI Foundry
Added Advantage — candidates with any of the following will stand out:
- Hands-on ML model development or integration
- Experience with event driven architecture using Azure Service Bus or Event Hubs
- Data lineage and cataloging experience: Microsoft Purview or similar
- Real time data processing experience using Azure Stream Analytics or Event Hubs
- Spark Structured Streaming experience on Azure Databricks or Microsoft Fabric
- Relevant certifications: Microsoft Certified: Fabric Data Engineer Associate (DP-700) or Databricks Certified Data Engineer Associate / Professional
Azist Inc. is an IT services and consulting firm founded in 2011, with offices in Virginia (USA) and Goa (India). Built by industry veterans with decades of experience at globally recognized technology organizations, Azist delivers end-to-end enterprise solutions spanning AI/ML, cloud infrastructure, data science, custom software development, software testing, and process management. With over 200 client engagements, 15+ years in business, and a 98% client retention rate.
📌 Senior Data Engineer (India)
🏢 Azistinc
📍 India