25 Sep
|
Innova ESI
|
Noida
–
Role- Databricks Architect
Experience: 15+ years overall, with 10+ years of relevant experience
Location- Noida / Gurugram / Hyderabad
Primary Skills: Python, SQL, Azure Data Technologies (ADT), Databricks, Data Architecture
Job Summary
Genpact is looking for an experienced Databricks Architect to design, architect, and lead the implementation of scalable, secure, and high-performance data platforms using Databricks, Python, SQL, and Azure data technologies. The ideal candidate will have strong hands-on architecture experience, excellent understanding of modern data engineering patterns, and the ability to work with business and technical stakeholders to translate requirements into robust data solutions.
Key Responsibilities
- Define and implement end-to-end data architecture using Databricks and Azure data services.
- Design scalable data engineering and analytics platforms using Databricks, Delta Lake, Spark, and related technologies.
- Develop and review solutions using Python, PySpark, SQL, and distributed data-processing frameworks.
- Design modern Lakehouse architectures, including Bronze, Silver, and Gold data layers.
- Architect data ingestion, transformation, integration, and orchestration pipelines.
- Establish standards for data modeling, data governance, security, performance, and scalability.
- Work with Azure data services such as Azure Data Lake Storage, Azure Data Factory, Azure Synapse, Azure Key Vault, and related services.
- Provide technical leadership to data engineers and development teams.
- Conduct architecture reviews and ensure adherence to enterprise architecture and engineering standards.
- Optimize Databricks workloads for performance, reliability, scalability, and cost.
- Implement CI/CD, version control, infrastructure automation, and DevOps practices for data platforms.
- Collaborate with product owners, business stakeholders, architects, and engineering teams to understand requirements and develop technical solutions.
- Define and maintain technical documentation, architecture diagrams, design standards, and implementation guidelines.
- Troubleshoot complex data-platform and performance issues and provide architectural recommendations.
- Mentor senior engineers and contribute to technical strategy and capability development.
Required Skills
- 15+ years of overall IT experience, with 10+ years of relevant experience in data engineering, data architecture, or cloud data platforms.
- Robust hands-on experience with Databricks and Apache Spark.
- Robust programming skills in Python/PySpark.
- Advanced SQL skills, including query optimization and complex data transformations.
- Strong knowledge of Azure Data Technologies (ADT) and cloud data architecture.
- Experience designing and implementing Lakehouse/Data Lake architectures.
- Strong knowledge of Delta Lake, medallion architecture, data pipelines, and distributed processing.
- Experience with Azure Data Factory or equivalent orchestration technologies.
- Strong understanding of data modeling, ETL/ELT, batch and streaming architectures.
- Experience with cloud security, identity/access management, encryption, networking, and governance.
- Experience with CI/CD and DevOps practices for data engineering platforms.
- Strong understanding of performance tuning and cost optimization in Databricks.
- Excellent architecture, communication, stakeholder-management, and problem-solving skills.
Good to Have
- Databricks certifications such as Databricks Certified Data Engineer or Databricks Certified Data Engineer Professional.
- Experience with Unity Catalog, Delta Live Tables/Lakeflow, Workflows, and Databricks SQL.
- Experience with Kafka/Event Hubs and real-time/streaming data pipelines.
- Knowledge of Terraform and Infrastructure as Code.
- Experience with Power BI or other BI/analytics platforms.
- Experience implementing enterprise-level data governance and data quality frameworks.
- Knowledge of MLOps or integration with AI/ML platforms.
- Experience working in large global organizations and client-facing environments.
📌 Databricks Architect || 15 Year+ || Only Immediate Joiner || Hyderabad (Noida)
🏢 Innova ESI
📍 Noida