Role: Confluent Kafka Data Engineer

Location: Bengaluru/ Chennai/ Hyderabad/ Pune/ Gurugram/ Mumbai (Hybrid)

Job Type: Permanent

Key Responsibilities:

- Design, develop, and maintain robust, scalable data pipelines using Azure Data Factory (ADF) and Databricks
- Implement batch and real-time data ingestion frameworks leveraging Confluent Kafka
- Build and optimize data models, transformations, and ETL/ELT pipelines using SQL and Spark
- Develop and manage data storage solutions in Azure Data Lake Storage (ADLS)
- Ensure high data quality, integrity, and governance standards
- Support data integration across systems including core banking and CRM
- Optimize performance of large-scale datasets
- Collaborate with stakeholders to translate requirements into solutions
- Ensure compliance with banking regulations and data privacy
- Troubleshoot production issues

Required Skills & Experience

- Solid expertise in SQL (complex queries, tuning, modeling)
- Hands-on experience with Databricks, Spark,



PySpark
- Experience with Confluent Kafka and event-driven architecture
- Experience with Azure Data Factory (ADF)
- Experience with Azure Data Lake Storage (ADLS)
- Understanding of data warehousing concepts
- Familiarity with CI/CD and DevOps for data platforms

Domain Knowledge (Preferred)

- Experience in banking or financial services
- Understanding of core banking systems
- Knowledge of payments, lending, wealth management
- Awareness of regulatory frameworks (KYC, AML)

Good to Have

- Experience with Python or Scala
- Exposure to data governance tools
- Knowledge of APIs and microservices
- Experience with BI tools like Power BI

Key Success Indicators

- Delivery of scalable pipelines
- Improved data reliability
- Reduced data latency
- Alignment with business requirements

📌 Confluent Kafka Data Engineer (Bengaluru)
🏢 Insight International (UK
📍 Bengaluru