AWS Data Engineer / Databricks / PySpark Technical Lead Experience:

5-10 Years Location:

Bangalore (Whitefield) Job Summary We are seeking an experienced

AWS Data Engineer / Databricks / PySpark Technical Lead

with strong expertise in designing, developing, and implementing large-scale data solutions on AWS and Databricks platforms. The ideal candidate should have extensive hands-on experience with Python, PySpark, Spark SQL, Kafka, Airflow, and cloud-based data engineering solutions. Key Responsibilities Design, develop, and optimize scalable data pipelines using

Python, PySpark, Spark SQL, Databricks, and AWS Services . Lead end-to-end architecture, design, development, and implementation of big data solutions. Develop and maintain real-time and batch data processing workflows using

Apache Kafka

and

Airflow . Build and support data integration solutions using APIs, streaming platforms, and cloud-native services. Perform data modeling, database design, performance tuning, and optimization activities. Support production systems, troubleshoot critical issues, and provide timely resolutions. Collaborate with business stakeholders, cross-functional teams, and vendors to deliver high-quality solutions. Mentor junior team members and provide technical leadership in Agile development environments.



Participate in code reviews, architecture discussions, and best practice implementation. Required Skills Strong experience in

Python, PySpark, Spark SQL, Databricks, AWS Cloud Services . Minimum

2+ years of Apache Kafka

streaming experience. Hands-on experience with

Apache Airflow . Experience working with

Oracle, PostgreSQL, and NoSQL databases . Positive knowledge of

SQL, PL/SQL, Unix Shell Scripting, and Autosys . Experience with

GitHub, Agile methodologies, and CI/CD practices . Strong analytical, troubleshooting, and problem-solving skills. Excellent communication and stakeholder management skills. Preferred Qualifications Experience in

Banking or Capital Markets domain . Proven ability to lead Agile development teams and deliver projects within tight deadlines. Experience in designing enterprise-scale data platforms and cloud migration initiatives. Note:

This role requires approximately

90% hands-on development, design, and architecture work , along with

10% team mentoring and technical leadership responsibilities . Provide your feedback on BizChat

Make this JD more conciseAdd key performance indicators for this roleCreate interview questions based on this JD

📌 AWS Data Engineer/ Data Bricks/Pyspark (Bengaluru)
🏢 Tata Consultancy Services
📍 Bengaluru