Job Title Data Engineer Location TechM Blr ITPL Years of Experience 7 10 Years Job Summary We are seeking a highly skilled Data Engineer with a strong background in PySpark to join our dynamic team The ideal candidate will have 7 10 years of experience in data engineering with a proven track record of designing building and maintaining scalable data pipelines You will play a crucial role in transforming raw data into actionable insights enabling data driven decision making across the organization Skills Python Pyspark Hadoop HiveQL Devops Responsibilities Design develop and maintain robust data pipelines using PySpark to process large datasets efficiently Collaborate with data scientists and analysts to understand data requirements and deliver high quality data solutions Implement data integration processes from various sources ensuring data quality and consistency Optimize existing data workflows and troubleshoot performance issues to enhance data processing efficiency Work with cloud based data platforms and tools to manage and store data securely Document data engineering processes and maintain clear communication with stakeholders regarding project status and challenges Stay updated with the latest industry trends and technologies in data engineering and big data Mandatory Skills Strong expertise in PySpark for data processing and transformation Proficient in SQL and experience with relational databases Experience with data warehousing solutions and ETL processes Familiarity with cloud platforms such as AWS Azure or Google Cloud Solid understanding of data modeling and data architecture principles Ability to work in a fast paced environment and manage multiple priorities effectively Preferred Skills Experience with Apache Kafka or similar streaming technologies Knowledge of machine learning concepts and frameworks Familiarity with containerization technologies like Docker Experience with version control systems such as Git Robust analytical and problem solving s