Roles and responsibilities

As a Data Engineer specializing in Data Integration, you will design and build solutions to transfer data from operational and external environments to the business intelligence workplace. You will utilize various tools to create and implement Extract, Transform, and Load (ETL) processes, ensuring the seamless flow of data throughout the business intelligence solution's lifecycle. Your primary responsibilities will include:
Design and develop scalable data integration pipelines to ingest data from multiple sources (core systems, third-party systems, and streaming platforms) into the data warehouse.
Build and maintain batch and near real-time ETL/ELT pipelines using tools such as AWS Glue, AWS DMS or Apache Kafka.
Implement data transformation, cleansing, and validation logic to ensure high data quality and consistency.
Optimize data pipelines and queries for performance, scalability, and cost efficiency, especially within Amazon Redshift.
Monitor, troubleshoot, and resolve data pipeline issues, ensuring high availability and reliability of data systems.
Collaborate with stakeholders to understand data requirements and deliver robust data solutions.
Maintain proper documentation and follow best practices in data engineering,



including logging, error handling, and version control.

Required Skills & Qualifications
2+ years of experience in Data Engineering or related roles
Strong proficiency in Advanced SQL (joins, window functions, query optimization, CTE function)
Hands-on experience with Python for data processing
Experience building ETL/ELT pipelines using tools like AWS Glue or similar frameworks
Working knowledge of streaming systems such as Apache Kafka
Familiarity with AWS services such as Amazon S3, Amazon Redshift, and AWS DMS
Understanding of data modeling concepts (OLAP vs OLTP, star schema, normalization)
Experience working with different types of databases, such as Relational (SQL) Databases, NoSQL Databases, Data Warehouses (OLAP) etc with a transparent understanding of their use cases, performance characteristics, and data modeling approaches.

Valuable to Have
Experience with Apache Spark / PySpark
Knowledge of data quality frameworks and monitoring tools
Understanding of cloud cost optimization techniques
Some exposure in LLM training, hosting, modelling creation of vector database etc.

📌 Data Engineer Bengaluru
🏢 Vistaar Finance
📍 Bengaluru