Description

JD for Databricks Data Engineer

- Design, develop, and maintain scalable data pipelines using Azure Databricks, PySpark, and SQL.
- Build and optimize ETL/ELT workflows for ingesting, transforming, and serving large-scale structured and unstructured datasets.
- Develop and manage data transformation models using DBT (Data Build Tool).
- Implement and monitor data quality frameworks, ensuring data accuracy, completeness, consistency, and reliability.
- Optimize Spark workloads for performance, scalability, and cost efficiency.
- Design and maintain data models to support analytics, reporting, and AI/ML initiatives.
- Collaborate with business stakeholders to understand processes and translate requirements into technical solutions.
- Implement best practices for data governance, security, testing, monitoring, and CI/CD.
- Support the development of Agentic AI and AI-powered analytics solutions by enabling high-quality and accessible data assets.
- Troubleshoot production issues and drive root cause analysis for data-related incidents.
- Work independently to identify improvement opportunities and deliver innovative solutions.

[hr align="center" size="2" width="100%"] Required Skills & Qualifications

Technical Skills

- Strong proficiency in Python for data engineering and automation.
- Advanced knowledge of SQL for data manipulation, optimization, and performance tuning.
- Hands-on experience with Azure Databricks and Apache Spark (PySpark).
- Experience developing and maintaining DBT models and testing frameworks.
- Robust understanding of Data Quality principles and tools.
- Experience building modern data lakehouse architectures.
- Hands-on experience with Azure Data Services such as:
- Azure Data Lake Storage (ADLS)
- Azure Data Factory (ADF)
- Azure Key Vault
- Azure DevOps





- Experience with workflow orchestration and scheduling tools.
- Knowledge of version control systems such as Git.

Agentic AI Exposure

- Understanding of Agentic AI concepts, autonomous workflows, and AI-powered data solutions.
- Exposure to LLM-based applications, AI agents, retrieval systems, or GenAI platforms is a plus.

Enable Skills-Based Hiring No ECMS REQ IDAdditional Details

- ECMS REQ ID : 584205
- Project Location 1 : Pan India
- Project Location 2 : (No Value)
- Project Location 3 : (No Value)
- Project Location 4 : (No Value)
- Delivery SPOC : Nishit Kumar
- Relevant Experience : 6+ Years
- Mandatory skills : • Strong proficiency in Python for data engineering and automation.• Advanced knowledge of SQL for data manipulation, optimization, and performance tuning.• Hands-on experience with Azure Databricks and Apache Spark (PySpark).• Experience developing and maintaining DBT models and testing frameworks.• Strong understanding of Data Quality principles and tools.• Experience building modern data lakehouse architectures.
- Desired skills : • Strong proficiency in Python for data engineering and automation.• Advanced knowledge of SQL for data manipulation, optimization, and performance tuning.• Hands-on experience with Azure Databricks and Apache Spark (PySpark).• Experience developing and maintaining DBT models and testing frameworks.• Strong understanding of Data Quality principles and tools.• Experience building modern data lakehouse architectures.
- Domain (Industry) : Oil & Gas
- BGC (Before onboarding / After onboarding) : PostOB
- Total Experience (Ex. 5-7 Years) : 6+ Years
- BGC Details : Infosys Standard Checklist
- BGC Vendor : Any NASSCOM Vendor
- Mode of Interview : Face to Face
- WFO / WFH / Hybrid : Hybrid
- Please enter shift timings : UK Shift
- Shift Timings : Afternoon

📌 584205 | Databricks Data Engineer | IC | Pan India (Bengaluru)
🏢 Trigent Software
📍 Bengaluru