We are looking for a highly skilled Senior Open Source Data Engineer to design, build, and scale a up-to-date enterprise data platform using open-source technologies. The ideal candidate will have extensive experience in distributed data architectures, real-time data processing, data governance, and large-scale data platform engineering. This role will drive the design and implementation of secure, scalable, and high-performance data solutions supporting analytics, business intelligence, and operational systems across global environments.
Key Responsibilities
Data Platform Architecture & Engineering
- Design and implement scalable Data Lake architectures using Medallion Architecture (Bronze, Silver, Gold layers).
- Build and maintain end-to-end batch and real-time data ingestion pipelines.
- Architect scalable, distributed data platforms supporting enterprise-wide data requirements.
- Ensure seamless data flow from source systems to analytics, APIs, and downstream applications.
- Support multi-region and globally distributed data environments.
Open Source Technology Leadership
- Lead implementation and optimization of open-source data platforms including:
- Data Ingestion: Debezium, Apache Airflow
- Streaming & Processing: Apache Kafka (KRaft), Apache Flink
- Storage: Hadoop, Apache Iceberg
- Query & Data Access: Trino, Apache Gravitino
- Metadata & Governance: Apache Atlas
- Visualization: Apache Superset
- Drive containerized deployments using Docker and Kubernetes.
- Promote open standards and extensible architectures while minimizing dependence on proprietary technologies.
Data Modeling & Governance
- Design robust and scalable data models for complex business domains.
- Implement enterprise-wide data governance frameworks.
- Establish data quality, metadata management, lineage tracking, and access control mechanisms.
- Ensure compliance with organizational security and data protection standards.
- Mainta
📌 Senior Data Engineer (Hyderabad)
🏢 Innova Solutions
📍 Hyderabad
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.