Key Responsibilities:
• Design, develop, and maintain robust, scalable data pipelines using Azure Data Factory (ADF) and Databricks
• Implement batch and real-time data ingestion frameworks leveraging Confluent Kafka
• Build and optimize data models, transformations, and ETL/ELT pipelines using SQL and Spark
• Develop and manage data storage solutions in Azure Data Lake Storage (ADLS)
• Ensure high data quality, integrity, and governance standards
• Support data integration across systems including core banking and CRM
• Optimize performance of large-scale datasets
• Collaborate with stakeholders to translate requirements into solutions
• Ensure compliance with banking regulations and data privacy
• Troubleshoot production issues
Required Skills & Experience
• Strong expertise in SQL (complex queries, tuning, modeling)
• Hands-on experience with Databricks, Spark,
PySpark
• Experience with Confluent Kafka and event-driven architecture
• Experience with Azure Data Factory (ADF)
• Experience with Azure Data Lake Storage (ADLS)
• Understanding of data warehousing concepts
• Familiarity with CI/CD and DevOps for data platforms
Domain Knowledge (Preferred)
• Experience in banking or financial services
• Understanding of core banking systems
• Knowledge of payments, lending, wealth management
• Awareness of regulatory frameworks (KYC, AML)
Valuable to Have
• Experience with Python or Scala
• Exposure to data governance tools
• Knowledge of APIs and microservices
• Experience with BI tools like Power BI
Key Success Indicators
• Delivery of scalable pipelines
• Improved data reliability
• Reduced data latency
• Alignment with business requirements
📌 Confluent Kafka Data Engineer (Bengaluru)
🏢 Insight International (UK
📍 Bengaluru
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.