17 Nov
|
Infosys
|
Bengaluru
Job Description:
- We are looking for an experienced Senior PySpark Developer with 8 years of experience in Data Engineering and Big Data technologies
- The ideal candidate should possess strong expertise in PySpark Python Spark SQL ETL development Data Warehousing and Cloud platforms AWS Azure GCP
- The candidate will be responsible for designing developing and optimizing large scale data processing solutions
Key Responsibilities:
- Key Responsibilities
- Design and develop scalable data pipelines using PySpark and Apache Spark
- Build and optimize ETL ELT frameworks for processing large volumes of structured and unstructured data
- Develop high performance Spark applications using Spark SQL DataFrames and RDDs
- Work closely with data architects and business teams to define data requirements
- Implement data quality validation and governance processes
- Optimize Spark jobs for performance memory management and scalability
- Integrate data from various sources including databases APIs cloud storage and streaming platforms
- Support production deployments and resolve performance bottlenecks
- Mentor junior developers and participate in code reviews
Technical Requirements:
- Mandatory Skills
- 8 years of IT experience
- Robust hands on experience in PySpark and Python
- Expertise in Apache Spark Spark SQL DataFrames and ETL Development
- Strong SQL and database concepts
- Experience with Hadoop Ecosystem Hive HDFS YARN
- Experience in Data Warehousing and Data Modeling
- Hands on experience with Unix Linux
Preferred Skills:
Technology->Data Engineering->PySpark
Skills: hdfs , Hive, Yarn, Apache Spark, Pyspark, Hadoop Ecosystem, Etl Development, Data Warehousing, Sql, Spark SQL, Linux, Unix, Python
Experience: 0.00-0.00 Years
📌 Senior PySpark Developer (Bengaluru)
🏢 Infosys
📍 Bengaluru