Job Title: Senior Data Engineer
Location: Noida
Employment Type: Full time
Experience Required: 8 - 12 Years
About the Role/Project
The role involves maintaining and developing data pipelines, serverless migration, and CI/CD process ownership. The solution product line encompasses several key areas, including marketing analytics, category optimisation, pricing optimisation, customer experience improvement, and sales performance enhancement.
Key Responsibilities
Adapt existing pipelines and code for serverless environments, ensuring stable execution
Maintain and enhance CI/CD processes, including proposing architectural improvements
Monitor pipelines and repositories, review merge requests, and oversee code changes
Develop and maintain data pipelines while understanding underlying business logic
Required Skills & Qualifications
8 years of experience in Data Engineering
Strong experience with Databricks and Delta Lake
Strong experience designing Lakehouse Architecture
Solid experience with PySpark and Spark SQL
Experience building configuration-driven data pipelines.
Robust Python programming skills with experience developing reusable Python libraries/packages.
Experience with Delta table optimisation and maintenance.
Recent project experience on the Databricks ecosystem is mandatory.
Experience with serverless architectures
Understanding of pricing or financial data domains
Preferred / Good to Have
Experience with Azure Data Platform.
Knowledge of CI/CD for data pipelines.
Performance tuning and optimisation of large-scale data pipelines.