Job Title:

Technical Lead - Python/PySpark Data EngineeringLocation:

BangaloreExperience:

5-10 YearsJob DescriptionWe are looking for an experienced

Technical Lead

with strong expertise in

Python, PySpark, Databricks, AWS Cloud, SQL, and Apache Kafka

to lead the design, development, and implementation of scalable data engineering solutions. The ideal candidate will have extensive hands-on experience in big data processing, cloud-based data platforms, and real-time streaming architectures.Primary SkillsPythonPySpark / Apache SparkDatabricksAWS Cloud ServicesSQL / Spark SQLApache KafkaSecondary SkillsApache AirflowOracle DatabasePL/SQLUnix Shell ScriptingAutoSysTest-Driven Development (TDD)GitHubKey ResponsibilitiesDesign, architect, and develop scalable data processing applications using Python, PySpark, Databricks, and AWS.Build and optimize batch and streaming data pipelines using Apache Kafka and Airflow.Lead end-to-end implementation of Databricks solutions, including architecture, development, deployment, and performance tuning.Develop and maintain complex ETL/ELT solutions for large-scale data platforms.Work closely with business stakeholders, vendors, and cross-functional teams to deliver high-quality solutions.Support production systems, troubleshoot critical issues,



and provide timely resolutions.Perform data modeling, database design, and performance optimization.Mentor junior developers and provide technical leadership to Agile teams.Ensure adherence to best practices in software development, testing, and deployment.Required Qualifications5-10 years of overall IT experience with strong data engineering expertise.Solid hands-on experience in Python, PySpark, Spark SQL, Databricks, and AWS.Minimum 2+ years of experience with Apache Kafka streaming solutions.Experience with Airflow, Oracle, PostgreSQL, and NoSQL databases.Knowledge of APIs, data integration frameworks, and real-time data streaming.Experience working in Agile environments and version control systems.Excellent analytical, problem-solving, and communication skills.Ability to work independently in a fast-paced, deadline-driven environment.Preferred QualificationsExperience in Banking or Capital Markets domain.Experience leading Agile development teams.Exposure to large-scale enterprise data platforms and cloud migration projects.Note:

This role requires approximately

90% hands-on development, design, and architecture work

and

10% team mentoring and technical leadership responsibilities .Provide your feedback on BizChat

📌 AWS Data Engineer/ Data Bricks/Pyspark (Mumbai)
🏢 TCS
📍 Mumbai