We are seeking a highly skilled Team Lead - Data Engineering with 7-9 years of experience in designing and developing scalable data solutions using Databricks, PySpark, and Python. The candidate will be responsible for leading a small team, driving technical delivery, and building high-performance data pipelines for enterprise analytics platforms.
Responsibilities
- Lead the design, development, and deployment of data engineering solutions using Databricks and PySpark.
- Develop scalable ETL/ELT pipelines and data transformation frameworks.
- Write effective Python and SQL code for data processing and analytics.
- Work closely with stakeholders to understand business requirements and translate them into technical solutions.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Ensure data quality, governance, and security standards are followed.
- Review code, mentor team members, and provide technical guidance.
- Support production deployments, incident troubleshooting,
and root cause analysis.
- Participate in Agile ceremonies and drive sprint deliverables.
Technical Requirements
- 7-9 years of overall IT experience.
- Strong hands-on expertise in Databricks and PySpark.
- Proficiency in Python programming and SQL.
- Experience building enterprise-scale data pipelines and data lake solutions.
- Strong understanding of distributed computing concepts and Spark architecture.
- Experience in performance tuning and optimization.
- Excellent stakeholder management and team coordination skills.
Education
- MBA
- MCA
- ME
- MSc
- MTech
- Bachelor of Engineering
- BCA
- BSc
- BTech
Preferred Skills
- Technology->Data Engineering->Databricks
- Technology->AI-Data science->PYTHON
- Technology->Big Data - Data Processing->PySpark