18 Sep
|
Neurealm
|
Chennai
We are looking for an experienced Data Solution Architect to design, architect, and implement large-scale data platforms and analytics solutions. The ideal candidate will have solid hands-on experience in Databricks, Azure, and Google Cloud Platform (GCP), along with the ability to create animated/visual data stories for technical and business audiences.
Candidate will play a key role in defining data strategy, optimizing data pipelines, ensuring platform scalability, and developing reusable frameworks. ?
Required Skills & Experience
- 10–12+ years overall in data engineering, analytics, or architecture roles.
- 2–4+ years strong hands-on experience in Databricks (mandatory).
- Deep experience with Azure and GCP cloud ecosystems.
- Strong expertise in Spark, PySpark, SQL, Python.
- Proficiency in data modeling, ETL frameworks, and distributed data systems.
- Experience with streaming technologies (Kafka, Event Hub, Pub/Sub).
- Ability to design and present animated architecture and data flows.
- Strong communication and stakeholder management skills.
Key Responsibilities
1. Architecture & Solution Design
- Design end-to-end data lakehouse, data warehouse, and analytics architectures using Databricks, Azure, and GCP.
- Build scalable and cost-optimized architectures for ingestion, transformation, streaming, and ML workloads.
- Define and enforce architecture standards, patterns, and governance models.
- Translate business requirements into logical and physical data models.
2. Databricks Expertise
- Architect and optimize Databricks Lakehouse solutions using Delta Lake, Unity Catalog, Databricks SQL, MLflow, Auto Loader, and DLT.
- Implement advanced transformations using PySpark, Spark SQL, and notebooks.
- Design job orchestration using Databricks Workflows or other orchestration tools.
3. Cloud Platform Ownership
Azure
- Azure Data Factory, Azure Data Lake Storage (ADLS), Azure Synapse, Azure Functions, Event Hub, Azure DevOps, Azure Kubernetes Service (AKS).
- Architecture of secure and scalable data systems on Azure.
GCP
- BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, Cloud Composer.
- Hands-on experience designing lakehouse/analytics solutions on GCP.
4. Data Engineering & Integration
- Lead development of batch and real-time data pipelines using Spark, ADF, Dataflow, or Databricks Workflows.
- Implement ETL/ELT frameworks, CI/CD, and reusable components.
6. Security & Governance
- Implement data governance frameworks including Unity Catalog, encryption, and compliance.
- Ensure best practices for cost optimization, monitoring, and performance tuning.
📌 Data Architect (Chennai)
🏢 Neurealm
📍 Chennai