Role & responsibilities

- Design, develop, and maintain scalable ETL/ELT data pipelines to ingest, transform, and load data from diverse sources

- Write and optimize advanced SQL queries for data extraction, transformation, validation, and reporting

- Build and maintain data models, warehouses, and semantic layers to support analytics, reporting, and downstream consumption

- Work with cloud data platforms and services to enhance data processing, storage, and scalability

- Automate and orchestrate end-to-end data workflows using tools such as Airflow, dbt, Cloud Composer, or Data Factory

- Collaborate with cross-functional stakeholders Data Architects, Data Scientists, Analysts, and business teams to translate requirements into technical solutions

- Perform data validation, reconciliation, and quality checks to ensure accuracy, consistency, and governance across pipelines

- Document data pipelines, schemas, and technical processes to support knowledge sharing and maintainability

- Troubleshoot pipeline failures, performance bottlenecks, and data quality anomalies

- Mentor junior engineers and contribute to best practices, code reviews, and continuous improvement (scope matched to seniority)

Preferred candidate profile

- 410 years of relevant experience in Data Engineering, Data Pipeline Development, or a related field

- Advanced SQL complex joins, CTEs, window functions, and query performance optimization

- Hands-on proficiency in Python and/or PySpark for data transformation, automation, and processing

- Proven experience building and maintaining ETL/ELT pipelines end-to-end

- Hands-on expertise in at least one cloud data platform (AWS,



GCP, or Azure) and its associated warehouse/lake service (Snowflake, BigQuery, Databricks, or Microsoft Fabric)

- Experience with workflow orchestration tools (Airflow, dbt, Cloud Composer, Data Factory, or similar)

- Strong problem-solving skills and the ability to collaborate effectively with technical and non-technical stakeholders

Positive to have:

- Marketing/Media Data Engineering &
- MLOps:

Data mart and harmonization design across disparate sources, MLOps and model deployment, model monitoring, cost optimization

- Unstructured Data &

- GenAI-Adjacent Engineering (GCP):

Document AI, Vertex AI, embeddings and vector search, semantic data modeling, Agentic AI/LangChain exposure

- Cloud-Native Pipeline Engineering (AWS): S3, Lambda, SNS, Step Functions, NumPy/Pandas, independent end-to-end ownership as an individual contributor

- Databricks DataOps &

- BI Enablement:

Databricks and PySpark at scale, data validation and reconciliation, BI tool exposure (Power BI, Tableau, or Looker), Git/CI-CD discipline

- Microsoft Fabric Platform Engineering: Data Factory/Dataflows Gen2, Medallion architecture (Bronze/Silver/Gold, Delta Lake, OneLake), Power BI DAX/Direct Lake, real-time streams (Eventstreams/KQL), Purview governance

- Team &

- Delivery Leadership:

Mentoring, code reviews, sprint/delivery ownership, cross-functional and business stakeholder management

Qualifications

- Bachelor's or Master's degree in a quantitative field (Computer Science, Engineering, or related) or equivalent practical experience

- Strong analytical, problem-solving, and communication skills.

📌 Data Engineer (Bengaluru)
🏢 Latentview
📍 Bengaluru