Job Description

Key Responsibilities:

n

n
- Design, build, and maintain CI/CD pipelines using Azure DevOps for automated testing, container image builds, and deployment
n
- Containerise the data quality platform using Docker and manage deployment of container images across Business environments using Kubernetes
n
- Spin up and configure new container instances when onboarding current Business areas onto the platform
n
- Develop and maintain automated testing frameworks covering unit, integration, and end-to-end testing
n
- Design, build, and expose RESTful APIs to integrate the platform with upstream and downstream systems
n
- Collaborate with the Data Architecture team to ensure platform components are modular, platform-agnostic, and deployable across cloud and on-premise environments
n
- Work with Azure services including Azure Container Registry, Azure Kubernetes Service, Azure Key Vault, Azure Blob Storage, and Azure DevOps
n
- Develop and optimise data pipelines using PySpark and SQL Server
n
- Support the integration of Databricks workloads into the platform architecture
n
- Ensure platform security standards are met including secret management, image vulnerability scanning, and access controls
n
- Contribute to technical documentation, runbooks, and onboarding guides for Business teams
n

n

Required Skills & Experience:

n

n
- 8–12 years of experience in data engineering or platform engineering roles
n
- Strong hands-on experience with Azure services — AKS, ACR, Key Vault,



Blob Storage, Azure DevOps
n
- Proven experience containerising applications using Docker and orchestrating with Kubernetes
n
- Solid experience building and maintaining CI/CD pipelines in Azure DevOps
n
- Experience designing and building RESTful APIs
n
- Proficiency in PySpark and SQL Server
n
- Experience with Databricks — job orchestration, cluster management, notebook development
n
- Strong understanding of software testing practices — unit testing, integration testing, end-to-end testing
n
- Experience with branching strategies, versioning, and release management
n
- Strong problem-solving skills and ability to work independently in a fast-paced environment
n

n

Preferred Skills:

n

n
- Understanding of Data Quality concepts — profiling, validation rules, DQ scoring, observability
n
- Experience with data quality tooling such as Informatica, Great Expectations, or similar
n
- Familiarity with Flask or similar Python web frameworks
n
- Experience with modular, microservice-style architecture design
n

n

What We're Looking For:

n

Beyond technical skills, we are looking for someone who takes ownership, thinks like a platform engineer, and understands that the best infrastructure is the kind that Business teams never have to think about. The ideal candidate is equally comfortable writing a Dockerfile, debugging a failing pipeline, and discussing deployment governance with a non-technical stakeholder.

📌 Senior Data Engineer (Bengaluru)
🏢 TEKsystems
📍 Bengaluru