What Youll Be Doing (position summary):

The Data Engineer is accountable for designing, developing, and optimizing scalable data ingestion and transformation pipelines that integrate data from multiple source systems into Databricks for reporting and analytics. This role supports the implementation of Medallion architecture principles to ensure data is standardized, governed, and business-ready across Bronze, Silver, and Gold layers. The role contributes to high-quality, reliable, and efficient data solutions by combining strong technical expertise, automation, testing, and cross-functional collaboration.

At CIBC India we enable the work environment most optimal for you to thrive in your role. Details on your work arrangement (including on-site and remote work) will be discussed at the time of your interview.

How Youll Succeed (responsibilities):

Data Pipeline Development:

- Design, build, and optimize scalable ETL/ELT pipelines to ingest data from Oracle, SQL MI, MongoDB, and file-based feeds into Databricks to support timely and reliable downstream consumption.
- Develop robust transformation logic using SQL, Python, Scala, Spark, and Databricks to ensure efficient processing of structured and semi-structured data.
- Implement reusable and maintainable data engineering components to improve development consistency, operational efficiency, and solution scalability.
- Troubleshoot and resolve pipeline issues to minimize disruption and maintain data availability for reporting and analytics.

Medallion Architecture Implementation:

- Lead the implementation of Medallion architecture principles across Bronze, Silver, and Gold layers to support scalable, governed, and business-aligned data processing.
- Establish ingestion and transformation patterns that ensure raw, cleansed, and curated data layers are clearly defined and consistently managed.
- Apply best practices in layer design, transformation sequencing, and data organization to improve data usability and platform performance.
- Support reporting and analytical use cases by ensuring data assets are structured in accordance with architectural standards.

Data Modeling and Transformation:

- Develop and maintain effective data models within the Silver layer to provide standardized, cleansed, and business-ready datasets.
- Define data relationships, transformation rules, and normalization approaches to support accurate integration and downstream analytics.
- Ensure transformation logic is well documented, maintainable, and aligned to business and reporting requirements.
- Collaborate with stakeholders to refine data structures that improve accessibility, consistency, and analytical value.

Data Quality, Testing, and Governance:

- Develop automated tests and validation checks for data pipelines and transformations to ensure data reliability, completeness, and correctness.
- Implement data quality controls, reconciliation processes, and cleansing routines to maintain data integrity throughout ingestion and transformation.




- Apply data governance best practices to ensure data is managed in a controlled, consistent, and auditable manner.
- Identify and address data issues proactively to reduce defects and improve stakeholder confidence in delivered data products.

Automation, Monitoring, and Operational Excellence:

- Automate pipeline workflows and operational processes to improve reliability, repeatability, and efficiency.
- Monitor pipeline execution, performance, and data freshness to identify issues and ensure high service availability.
- Analyze performance bottlenecks and implement enhancements to improve throughput, scalability, and stability.
- Maintain operational support practices that enable rapid diagnosis and resolution of production issues.
- Stakeholder Collaboration and Documentation
- Work closely with data analysts, data scientists, and business stakeholders to understand requirements and translate them into effective data engineering solutions.
- Provide technical input and recommendations on data ingestion, transformation, modeling, and architecture decisions.
- Develop and maintain documentation for pipelines, data flows, models, controls, and design decisions to support maintainability and knowledge transfer.
- Contribute to continuous improvement initiatives by identifying opportunities to enhance engineering practices, tools, and delivery standards.

CROSS-FUNCTIONAL RELATIONSHIPS:

- This job works closely with data analysts and data scientists to understand analytical and reporting requirements and deliver business-ready datasets.
- This job collaborates with business stakeholders to clarify source data requirements, data definitions, and expected reporting outcomes.
- This job interacts with source system owners and database teams to support data extraction, issue resolution, and changes impacting ingestion processes.
- This job partners with platform, cloud, and DevOps teams to support Databricks, Azure Data Factory, automation, deployment, and environment stability.
- This job collaborates with governance, risk, and control partners to support data quality, data management, and compliance expectations.

COMPLIANCE REQUIREMENTS/RESPONSIBILITIES:

- As an employee of CIBC, the incumbent must comply with all applicable CIBC and Line of Business policies, standards, guidelines, and controls.

JOB DIMENSIONS:

- Responsible for the delivery of testing services for assigned projects/initiatives, supporting Capital Markets ASG group, focusing primarily on Middleware/API and database testing.
- Supports achievement of business unit goals through effective quality engineering and testing practices

Experience:



5+ years of experience

Who You Are (skills/qualifications):

Bachelors degree in Computer Science, Engineering, Information Systems, or related field.

Must Have Skills:

- 5+ years of hands-on experience in data engineering roles, with strong experience in ETL/ELT pipeline development and cloud-based data platforms.
- Strong proficiency in SQL and Python or Scala for data transformation, orchestration, and performance tuning.
- Hands-on experience with Databricks, Spark, Azure Data Factory, and Medallion architecture implementation.
- Strong knowledge of source database technologies including Oracle, SQL MI, and MongoDB.
- Experience with file-based ingestion patterns involving CSV, Parquet, JSON, and similar formats.
- Robust data modeling expertise, particularly in designing standardized and business-ready datasets within the Silver layer.
- Understanding of data warehousing, reporting, and downstream analytics concepts.
- Knowledge of automated testing practices for data pipelines and transformations.
- Strong analytical and problem-solving skills with the ability to diagnose issues and implement effective solutions.
- Strong verbal and written communication skills with the ability to work effectively with technical and business stakeholders.
- Ability to work independently, manage priorities, and lead technical initiatives.
- Strong attention to detail and commitment to data quality, integrity, and consistency.

Good to Have:

- Preferred: experience with cloud platforms such as Azure, AWS, or GCP.
- Preferred: knowledge of CI/CD tools and practices.
- Preferred: exposure to data governance and security frameworks.

About CIBC:

CIBC is a leading North American bank, with a history of more than 150 years of living our purpose - to help make your ambition a reality. Client relationships are at the centre of our strategy, and our growth is enabled by a highly engaged and connected global team. From digital banking to community impact, were proud to drive progress for our clients and our communities. Recognized for our culture, award-winning innovation and community investment, CIBC continues to evolve by putting people and progress at the centre of everything we do.

About CIBC India:

CIBC India is a technology and operations hub in Hyderabad, where youll be part of our highly engaged and global team, and play a central role in supporting our growth. Whether youre driving innovation in digital banking or streamlining client onboarding, youll be part of a culture recognized for excellence and investment in people. At CIBC India, people and progress are at the centre of what we do: youll develop skills, collaborate with industry leaders, and see your ideas realized in a culture that values progress and belonging. We provide cutting-edge resources, ongoing learning, and a supportive environment where people come first. If youre ready to create meaningful change and build your future, CIBC India is where your ambition meets opportunity.

📌 Senior Data Engineer (Hyderabad)
🏢 CIBC India
📍 Hyderabad