- The candidate will be required on Clients payroll.
- Must Have Skills- PySpark, Azure Databricks, SQL, Python
- 6+ Years
- Requires robust SQL, working knowledge of PySpark/Python, and experience with data pipelines, Azure Data Factory, Databricks, and basic ML workflows.
- Involves stakeholder communication, CI/CD monitoring, and continuous improvement through debugging, pattern identification, and data-driven insights.
- Support and monitor weekend batch loads, ensuring timely and accurate execution
- Perform outlier detection and root cause analysis for issues such as demand forecast drops or anomalies
- Translate findings into transparent insights and communicate them effectively to leads and stakeholders
- Monitor scheduled jobs and data pipelines to ensure successful execution
- Validate dashboard outputs and analyze data trends, anomalies, and inconsistencies
- Support basic model validation and output analysis (e.g., forecast vs actuals,
error trends)
- Write and optimize SQL queries and PySpark transformations for debugging and analysis
- Identify patterns in recurring failures and recommend improvements
- Work with data science and engineering teams to debug model or data-related issues
- Escalate issues when required, with clear analysis and supporting insights
- Monitor workflows in Azure Data Factory (ADF) and Databricks
- Contribute to CI/CD validation and release monitoring
- Maintain documentation for issues, RCA findings, and fixes.
Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.