Role Summary

We are seeking an experienced Senior Data Engineer to design, build and support up-to-date data platforms, data products and data pipelines across our application landscape. The successful candidate will be responsible for data ingestion, transformation, curation, orchestration, quality controls, cloud data engineering and end-to-end platform reliability.

This role requires strong hands-on engineering capability with accountability for delivering secure, scalable, resilient and well-governed data solutions that support reporting, analytics, operational processes and strategic business outcomes.

Data Engineering & Platform Delivery

- Design, develop and support scalable data products, data pipelines and data platform capabilities.
- Build and maintain batch, streaming and event-driven data processing solutions.
- Develop and optimise ETL/ELT pipelines using SQL, Python and modern transformation frameworks.
- Support ingestion, transformation, staging, curation and consumption layers across enterprise data platforms.
- Ensure solutions are secure, resilient, performant and aligned with enterprise engineering standards.

dbt, SQL & Data Transformation

- Develop, test and maintain dbt models for data transformation, business logic and reusable data assets.
- Implement SQL-based transformation patterns with appropriate data modelling and performance optimisation.
- Create modular, maintainable and well-documented transformation logic to support downstream consumption.
- Apply testing, reconciliation and validation controls to improve trust in data products.

Workflow Orchestration & DAG Development





- Develop, maintain and support Apache Airflow / Cloud Composer DAGs for workflow orchestration.
- Define task dependencies, scheduling, retries, monitoring and alerting for data workflows.
- Improve pipeline reliability through automation, observability and operational controls.
- Support incident investigation and root cause analysis for failed or delayed data workflows.

Cloud Data Engineering

- Build and support cloud-native data solutions using GCP or equivalent cloud platforms.
- Work with technologies such as BigQuery, Dataflow, Pub/Sub, Cloud Storage and Cloud Composer where applicable.
- Contribute to platform automation using Terraform or Infrastructure as Code tooling.
- Support CI/CD pipelines, deployment automation and DevOps practices for data engineering workloads.

Data Quality, Governance & Controls

- Implement data quality checks covering completeness, accuracy, consistency, freshness and uniqueness.
- Support metadata, lineage, data ownership, policy tagging and governance requirements.
- Embed monitoring, observability and reconciliation controls across critical data flows.
- Ensure data solutions comply with information security, privacy and regulatory control expectations.





Collaboration & Technical Leadership

- Work closely with Product Owners, Architects, Analysts, Engineers and Business Stakeholders to translate requirements into technical solutions.
- Contribute to solution design, code reviews, engineering standards and continuous improvement initiatives.
- Mentor and support engineers within the team, promoting good engineering practices and knowledge sharing.
- Provide support for production services, incident resolution and operational stability.

Leadership & Stakeholder Management

- Lead DevOps engineers, platform engineers, and infrastructure specialists.
- Act as the primary escalation point for platform and environment-related issues.
- Work closely with Engineering Leads, Architects, Security, Networks, and Infrastructure teams.
- Provide technical guidance, mentoring, and operational leadership.

Required Skills & Experience

Technical Skills

- Strong experience in Data Engineering and enterprise-scale data platform delivery.
- Advanced SQL development and data modelling skills.
- Strong Python programming experience for data engineering use cases.
- Hands-on experience with dbt for data transformation and model development.
- Experience developing and maintaining Apache Airflow / Cloud Composer DAGs.
- Experience with ETL/ELT design, data ingestion, transformation and curation patterns.
- Experience with cloud data platforms such as GCP, AWS or Azure.
- Experience with CI/CD, Git-based development, deployment automation and DevOps practices.

📌 Data Engineer (Hyderabad)
🏢 Lloyds Technology Centre
📍 Hyderabad