16 Sep
|
waters
|
Bengaluru
Overview
The Senior Data Engineer (ETL) will lead the design and development of scalable ETL pipelines for Waters’ Enterprise Data Lakehouse. This role requires a strong technical foundation, hands-on experience with Azure databricks, Pyspark, SQL, Python & GITHUB alongwith a strong understanding of standard data engineering practices, and the ability to work cross-functionally with business and technical teams
Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL/ELT processes.
- Develop and optimize data processing solutions using Databricks and Apache Spark.
- Build data pipelines using PySpark, Python, and SQL.
- Work with Databricks notebooks, workflows/jobs, clusters, and Delta Lake.
- Implement incremental data processing and performance optimization techniques.
- Manage source code using GitHub, including branching, pull requests, code reviews, and repository management.
- Implement and maintain CI/CD pipelines for data engineering workloads using GitHub-based development practices.
Qualifications
- Bachelor’s degree in computer science,
Software Engineering, Information Systems, or a related field.
- Typically, 4-7 years of relevant experience in Data Engineering, ETL, DWH, or data analytics decision support role, with some senior roles requiring total experience between 4 to 7 years.
- Robust handson experience with Databricks platform.
- Expert proficiency in SQL and Python (advanced querying, stored procedures, performance tuning).
- Extensive experience with ETL tools and platforms (e.g., SSIS, Azure Data Factory, Azure DataBricks).
- Deep understanding of data warehousing concepts (dimensional modelling, Kimball/Inmon methodologies).
- Familiarity with Big Data technologies (e.g., Spark, Hadoop, Kafka) for processing large
- Experience with version control systems (e.g., Git).
- Strong technical & analytical skills required, including a thorough understanding of how to interpret customer business needs and translate them i
📌 Data Platforms Senior Engineer – Databricks (Bengaluru)
🏢 waters
📍 Bengaluru