04 Oct
|
IndiHire Consultants
|
Bengaluru
04 Oct
IndiHire Consultants
Bengaluru
Role & responsibilities
:
We are looking for an experienced Data Architect to lead the architecture and technical direction of a new real-time data streaming, processing, and integration platform.The role will be responsible for designing scalable, resilient, secure, and low-latency data architectures using modern data engineering and streaming technologies. The ideal candidate should have strong hands-on experience with Apache Flink, Apache Kafka, Debezium, Change Data Capture (CDC), ClickHouse, and data orchestration frameworks such as Apache Airflow. The platform will ingest real-time data from multiple source systems, process and transform streaming data, and make it available through analytical engines, APIs, dashboards, and future downstream integrations.
The Data
Architect will work closely with engineering, application, infrastructure, product, and business teams to establish architecture standards, technical patterns, and engineering practices for the platform.
Key Responsibilities
Data Architecture & Solution Design
- Define the overall architecture for the real-time data streaming and integration platform.
- Design end-to-end data architectures covering ingestion, CDC, streaming, processing, transformation, storage, analytics, APIs, and downstream integrations.
- Define scalable, fault-tolerant, highly available, and low-latency data processing architectures.
- Establish architectural patterns for batch, streaming, event-driven, and hybrid data workloads.
- Evaluate technical requirements and translate them into scalable data architecture and solution designs.
- Define data flow, system interactions, interfaces, dependencies, and technology boundaries.
- Conduct architecture reviews and provide technical recommendations and trade-offs.
- Create architecture diagrams, technical design documents, data flow diagrams, and implementation guidelines.
Preferred candidate profile
:
We are looking for an experienced Data Architect to lead the architecture and technical direction of a new real-time data streaming, processing,
and integration platform.The role will be responsible for designing scalable, resilient, secure, and low-latency data architectures using modern data engineering and streaming technologies. The ideal candidate should have strong hands-on experience with Apache Flink, Apache Kafka, Debezium, Change Data Capture (CDC), ClickHouse, and data orchestration frameworks such as Apache Airflow. The platform will ingest real-time data from multiple source systems, process and transform streaming data, and make it available through analytical engines, APIs, dashboards, and future downstream integrations.
The Data
Architect will work closely with engineering, application, infrastructure, product, and business teams to establish architecture standards, technical patterns, and engineering practices for the platform.
Key Responsibilities
Data Architecture & Solution Design
- Define the overall architecture for the real-time data streaming and integration platform.
- Design end-to-end data architectures covering ingestion, CDC, streaming, processing, transformation, storage, analytics, APIs, and downstream integrations.
- Define scalable, fault-tolerant, highly available, and low-latency data processing architectures.
- Establish architectural patterns for batch, streaming, event-driven, and hybrid data workloads.
- Evaluate technical requirements and translate them into scalable data architecture and solution designs.
- Define data flow, system interactions, interfaces, dependencies, and technology boundaries.
- Conduct architecture reviews and provide technical recommendations and trade-offs.
- Create architecture diagrams, technical design documents, data flow diagrams,
and implementation guidelines.
Performance & Reliability
- Evaluate system performance across Kafka, Flink, CDC, orchestration, and analytical platforms.
- Identify architectural bottlenecks and recommend improvements.
- Define strategies for high availability, fault tolerance, disaster recovery, and scalability.
- Establish SLAs/SLOs for critical data pipelines and services.
- Drive performance tuning and capacity planning.
Technical Leadership
- Provide technical leadership and mentoring to Data Engineers.
- Conduct design and code reviews where appropriate.
- Establish development standards and reusable engineering patterns.
- Lead technical POCs and evaluate new technologies.
- Collaborate with engineering and infrastructure teams on deployment and operational architecture.
- Support production troubleshooting and complex technical issues.
- Drive continuous modernization and improvement of the data platform.
Required Skills & Experience
- 8+ years of experience in Data Engineering, Data Architecture, or related roles.
- Strong experience designing real-time and streaming data platforms.
- Strong hands-on expertise with Apache Flink.
- Strong experience with Apache Kafka.
- Hands-on experience with Debezium and Change Data Capture (CDC).
- Strong experience with analytical databases; ClickHouse experience is highly preferred.
- Robust experience with Apache Airflow or equivalent orchestration technologies.
- Strong understanding of:
- Distributed systems
- Event-driven architecture
- Streaming architecture
- Batch and real-time processing
- Data integration patterns
- Strong SQL and data modeling skills.
- Experience designing cloud-based data platforms.
- Experience with data quality, observability, monitoring, governance, and security.
- Experience with REST APIs and downstream integrations.
- Strong architecture documentation and communication skills.
- Ability to translate business and technical requirements into scalable architecture
📌 Data Architect (Bengaluru)
🏢 IndiHire Consultants
📍 Bengaluru