- Design scalable enterprise data platforms using Azure and Databricks.
- Architect Lakehouse solutions using Delta Lake and Unity Catalog.
- Build metadata-driven data engineering frameworks.
- Develop scalable data pipelines using PySpark and Spark SQL.
- Optimize Spark jobs for performance and cost.
- Design dimensional and normalized data models.
- Build and deploy GenAI/LLM-powered applications.
- Guide AI-assisted software engineering using Copilot, Claude Code, Gemini Code, or Codex.
- Review AI-generated code and establish engineering best practices.
- Implement CI/CD using Azure DevOps, Git, or Jenkins.
- Collaborate with business stakeholders and delivery teams.
- Mentor engineers and provide technical leadership.
Mandatory Skills
- Azure Databricks
- PySpark
- Spark SQL
- Azure Data Factory
- Azure Data Lake Storage (ADLS Gen2)
- Delta Lake
- Unity Catalog
- Microsoft Fabric
- Azure SQL
- Python
- SQL
- Data Lakehouse Architecture
- Data Modeling
- GenAI / LLM
- Azure Cloud
- Git
- Azure DevOps / Jenkins
Valuable to Have
- Delta Live Tables
- Databricks Workflows
- SQL Warehouse
- Mosaic AI
- AI/BI Genie
- Azure Purview
- Event Hub
- Cosmos DB
- Azure Data Explorer
- Neo4j
- Elasticsearch
- Vector Databases
- MCP
- Claude Code
- GitHub Copilot
- Gemini Code
- Codex
- Azure Infrastructure & Security
Education
Bachelor's Degree (B.E./B.Tech) in Computer Science, IT, or related discipline.
Preferred Candidate Profile
- 8+ years of Data Engineering experience.
- 4+ years of Azure Databricks experience.
- Strong expertise in PySpark and Spark optimization.
- Experience designing enterprise-scale Lakehouse architectures.
- Hands-on experience with Azure cloud services.
- Exposure to GenAI and Agentic AI development.
- Excellent communication and stakeholder management skills.
- Architecture and solution design experience is preferred.