21 Nov
|
Publicis Sapient
|
Bengaluru
21 Nov
Publicis Sapient
Bengaluru
ROLE SUMMARY : We are looking for a hands-on Data Engineering Manager with solid expertise in Python, PySpark, SQL, and Apache Airflow. The role will lead a team of approximately 8 - 12 engineers, own delivery of scalable data platforms and pipelines, provide technical direction, and remain hands-on across architecture, development, performance optimization, data quality, and production support.KEY RESPONSIBILITIES : - Lead the architecture, design, and delivery of scalable batch and near-real-time data pipelines using Python and PySpark.- Provide technical direction for Apache Airflow orchestration, including DAG standards, scheduling, dependencies, monitoring, retries, and recovery.- Guide development of optimized SQL for ingestion, transformation, reconciliation, controls, and analytical workloads.- Manage and mentor 8 - 12 data engineers while remaining hands-on with design reviews, code reviews, debugging, and complex problem solving.- Own delivery planning, estimation, execution, quality, risks, dependencies, release readiness, and production stability.- Drive data-quality controls, lineage, observability, security, auditability, and operational support standards.- Optimize Spark jobs, SQL queries, partitioning, caching, joins, resource utilization, cloud performance, and cost.- Partner with Product Owners, Architects, Governance, DevOps, QA, and client stakeholders to deliver measurable business outcomes.REQUIRED SKILLS : - 10+ years of data or software engineering experience, including proven technical, delivery,
and people leadership.- Strong hands-on Python with OOP, modular architecture, testing, exception handling, and reusable framework development.- Deep PySpark expertise across DataFrames, Spark SQL, distributed processing, partitioning, joins, caching, and performance tuning.- Advanced SQL including CTEs, window functions, complex joins, validation, reconciliation, data modelling, and query optimization.- Strong Apache Airflow expertise covering DAG architecture, operators, sensors, dependencies, scheduling, monitoring, and troubleshooting.- Experience with relational databases, data lakes or warehouses, Parquet, schema evolution, and high-volume data processing.- Experience with Git, CI/CD, Linux, test automation, deployment governance, and production support practices.- Strong Agile delivery, stakeholder management, communication, risk management, and team-development skills.PREFERRED SKILLS : - Financial Services / Capital Markets experience, including reconciliation, controls, risk, or regulatory data.- Azure or AWS data services, Databricks, Snowflake, Docker, Kubernetes, or Kafka exposure.- Knowledge of data governance, metadata management, security, observability, and cloud cost optimization.IDEAL CANDIDATE PROFILE : Hands-on Data Engineering Manager who can lead teams and delivery while architecting reliable pipelines, optimizing distributed workloads, strengthening data quality, and owning production outcomes.ROLE POSITIONING : Technical Leadership + Data Engineering Delivery + People Management (ref:hirist.tech)
📌 Manager - Python/Data Engineering (Bengaluru)
🏢 Publicis Sapient
📍 Bengaluru