Role description 5 days working from Client office inManyata Tech Park Bengaluru About the Role We are seeking hands on Data Engineers to design build and operate secure data pipelines and analytical platforms on Azure with Databricks Youll work inside a cleanroom controlled environment to ingest transform and publish high quality datasets for analytics ML and reportingensuring governance privacy and performance at scale Key Responsibilities 1Data Pipelines ETLELT oDesign and implement batchstreaming pipelines using Azure Databricks PySpark and Delta Lake oBuild ingestion frameworks for diverse sources files APIs databases applying robust data quality checks and observability logging metrics s 2Lakehouse Architecture oModel curated layers BronzeSilverGold with Delta tables optimize for ACID Z Ordering partitioning and costperformance oImplement schema evolution CDC and data versioning 3Azure Data Services oIntegrate Azure Data Lake Storage ADLS Gen2 Azure Data FactorySynapse pipelines Azure Key Vault Event HubService Bus as applicable oManage secrets identities Managed Identity and role based access Azure RBAC 4Operational Excellence oOwn CICD for Databricks ReposJobs infra automation TerraformBicep and environment promotion oEnsure SLAsSLOs implement cost optimization auto scaling and cluster policies 5Data Quality Governance oDefine DQ rules eg Great Expectations Deequ and lineage adhere to privacy and compliance standards in cleanroom operations oWork closely with InfoSec to maintain air gappedcontrolled workflows and review access requests 6Collaboration Documentation oPartner with AnalyticsML teams to productionize feature stores write clear runbooks data dictionaries and operating procedures Must Have Qualifications 2 to 6 years in Data Engineering with strong PySparkSpark experience Hands on with Azure Databricks clusters jobs notebooks repos and Delta Lake Practical experience with ADLS Gen2 Azure Data FactorySynapse Azure DevOpsGitHub Actions Solid SQL performance tuning window functions and Python dataETL utilities Experience in production data pipelines scheduling monitoring ing Understanding of data modeling dimensional wide tables partitioning and performance optimization Comfort working onsite five days in a cleanroom with strict governance Positive to Have Preferred Streaming Structured Streaming KafkaEvent Hub Orchestration AirflowAzure Data Factory advanced patterns Testing Quality Great ExpectationsDeequ unitintegration tests for data pipelines Security Key Vault Private Endpoints VNET network rules familiarity with PII handling Infra as Code TerraformBicep for Azure resources Databricks workspacecluster policies Cost Observability Photon auto optimize Delta cache Lakehouse performance tuning Exposure to ML feature pipelinesMLflow and feature stores nice to have Evaluation Rubric high level DatabricksSpark 30 Coding quality performance Delta features Azure Data Stack 25 ADLS ADFSynapse security identities Data Modeling Quality 20 BronzeSilverGold DQ rules lineage OpsCICDCost 15 Automation monitoring cost controls Cleanroom Readiness 10 Compliance discipline documentation Skills Mandatory Skills Azure Data Factory Databricks About LTIMindtree LTIMindtree is a global technology consulting and digital solutions company that enables enterprises across industries to reimagine business models accelerate innovation and maximize growth by harnessing digital technologies As a digital transformation partner to more than 700 clients LTIMindtree brings extensive domain and technology expertise to help drive superior competitive differentiation customer experiences and business outcomes in a converging world Powered by 87 000 talented and entrepreneurial professionals across more than 40 countries LTIMindtree a Larsen Toubro Group company solves the most complex business challenges and delivers transformation at scale For more information please visit https www ltimindtree com Please also note that neither LTIMindtree nor any of its authorized recruitment agencies partners charge any candidate registration fee or any other fees from talent candidates towards appearing for an interview or securing employment internship Candidates shall be solely responsible for verifying the credentials of any agency consultant that claims to be working with LTIMindtree for recruitment Please note that anyone who relies on the representations made by fraudulent employment agencies does so at their own risk and LTIMindtree disclaims any liability in case of loss or damage suffered as a consequence of the same Recruitment Fraud Alert - https www ltimindtree com recruitment-fraud-alert

📌 Specialist - Data Engineering (Karnataka)
🏢 LTIMindtree
📍 Karnataka