Roles & Responsibilities

- Design, develop, and maintain scalable, reliable, and high-performance data pipelines.
- Build and manage ETL/ELT workflows using DBT and Apache Airflow.
- Develop and optimize complex SQL queries for large-scale data processing.
- Design and implement robust data models while ensuring data quality and consistency.
- Develop data processing solutions using PySpark, Apache Spark, or Apache Flink.
- Design, implement, and manage Data Lake architectures using Apache Iceberg.
- Monitor and optimize database performance, storage, and query efficiency.
- Collaborate with Product Managers, Data Scientists, Analysts, and Engineering teams to deliver business-driven data solutions.
- Implement CI/CD pipelines, testing, monitoring, logging, and documentation best practices.
- Troubleshoot production issues and ensure the reliability, scalability, and security of data platforms.
- Continuously improve data engineering processes, architecture, and performance.

Preferred candidate profiles

- Minimum 5+ years of experience as a Data Engineer or in a similar data engineering role.
- Strong hands-on experience with DBT, Apache Airflow, PySpark, and SQL.




- Experience in designing and implementing Data Lake solutions using Apache Iceberg.
- Strong understanding of ETL/ELT, Data Warehousing, and up-to-date data architectures.
- Proficiency in Python for data engineering and automation (Preferred).
- Experience with Apache Spark or Apache Flink for distributed data processing.
- Working knowledge of cloud platforms, preferably AWS (S3, Athena, MWAA, EKS, ElastiCache, SageMaker).
- Familiarity with Kubernetes, YARN, Parquet, ORC, and Iceberg file formats is an added advantage.
- Good understanding of CI/CD, version control, testing, monitoring, and DevOps practices.
- Strong analytical, problem-solving, and debugging skills.
- Excellent communication and stakeholder management abilities.
- Experience working in Agile/Scrum development environments.
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
- Candidates with experience building scalable, secure, and high-performance data platforms will be preferred.

📌 Data Engineer (Bengaluru)
🏢 Kapi Technologies
📍 Bengaluru