About the Role We are looking for a hands-on Data Engineer (7+years) to build and optimize scalable data pipelines and analytical datasets on the Databricks platform. You will work closely with Analytics/BI, Product, and Business teams to enable data-driven decision-making. Retail / eCommerce domain exposure is a strong plus, along with the ability to translate business needs into reliable and performant data solutions.
Key Responsibilities:
• Design, develop, and maintain robust ETL/ELT pipelines using Databricks (Spark) and Python (PySpark). • Develop and optimize complex transformations using SQL (joins, window functions, CTEs, query tuning). • Build curated datasets and data models to support reporting, dashboards, and advanced analytics use cases. • Implement pipeline reliability best practices: data quality checks, monitoring, alerting, and reconciliation. • Optimize Databricks workloads for performance and cost (cluster sizing, partitioning strategies, caching, file formats). • Work with structured and semi-structured data (JSON, CSV, Parquet/Delta) and handle schema evolution.



• Collaborate with stakeholders to understand business KPIs and deliver data solutions aligned to retail/eCommerce metrics (sales, orders, returns, inventory, customer cohorts). • Follow engineering best practices for version control (Git), documentation, reusable code patterns, and testing. • Good to have: Support or migrate Alteryx workflows into Python/Databricks pipelines.
Must-Have Skills & Qualifications :
• 7-13 years of experience in Data Engineering / Data Warehousing / Big Data. • Robust hands-on experience with Databricks (Jobs/Workflows, notebooks, cluster concepts, Spark tuning fundamentals). • Strong programming skills in Python (PySpark preferred). • Excellent SQL skills, including performance tuning and writing complex analytical queries. • Experience building scalable pipelines and working with large datasets in distributed environments. • Strong understanding of data

📌 Process Manager (Navi Mumbai)
🏢 eClerx
📍 Navi Mumbai