21 Nov
|
Ness Digital Engineering (India) Private
|
Hyderabad
21 Nov
Ness Digital Engineering (India) Private
Hyderabad
Key Responsibilities :- Design and implement enterprise data warehouses and modern data platforms on Azure and Databricks.- Define data architecture standards, patterns and best practices, including lakehouse design on Delta Lake.- Build and optimise data solutions using Azure Synapse Analytics (Dedicated and Serverless SQL Pools), Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS Gen2).- Develop large-scale data processing on Databricks using Spark (PySpark/Scala), Delta Lake and MLflow.- Design data models using 3NF, dimensional, Data Vault and canonical approaches to suit business and reporting needs.- Write and tune complex SQL for performance and scalability.- Plan and lead migration from traditional platforms (Oracle, Teradata, SQL Server, Informatica) to modern cloud-based ELT/ETL frameworks.- Set up data governance, metadata management, data quality and security controls, using tools such as Azure Purview.- Design and build real-time and streaming data pipelines using Kafka, Event Hub or similar technologies where required.- Guide and mentor data engineers, and review designs, code and pipelines.- Work with business, analytics,
AI/ML and infrastructure teams to understand data needs and deliver the right solutions.- Prepare architecture documents, data models, data flow diagrams and technical standards.Must Have :- Proven experience as a Data Architect or Senior Data Engineer/Lead designing and implementing enterprise data warehouses and modern data platforms.- 8 - 12 years of experience in Data Engineering/Architecture, with at least 5+ years in Azure & Databricks ecosystem.- Hands-on expertise with Azure services: Azure Synapse Analytics (Dedicated SQL Pools, Serverless Pools), Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), Azure Purview.- Strong expertise with Databricks: Delta Lake, Lakehouse, Spark (PySpark/Scala), MLflow.- Very solid SQL development and optimization skills.- Strong data modeling (3NF, Dimensional, Data Vault, Canonical Models, etc.).- Knowledge of traditional data platforms (Oracle, Teradata, SQL Server, Informatica, etc.) along with modern ELT/ETL frameworks.- Solid understanding of data governance, metadata management, data quality, and security.- Experience in building real-time/streaming data pipelines (Kafka, Event Hub, etc.) is a plus. (ref:hirist.tech)
📌 Ness Technologies - Data Architect - Azure/Databricks (Hyderabad)
🏢 Ness Digital Engineering (India) Private
📍 Hyderabad