Key Responsibilities:

- Design develop and maintain ETL ELT pipelines using PySpark and Databricks
- Build scalable data processing solutions for large datasets
- Develop and optimize Spark jobs for performance and efficiency
- Implement data ingestion pipelines from various structured and unstructured data sources
- Work with Delta Lake Data Lake and cloud storage solutions
- Collaborate with business stakeholders data analysts and architects to understand data requirements
- Ensure data quality governance and security standards are followed
- Troubleshoot performance issues and optimize data workflows
- Participate in code reviews and follow DevOps best practices
- Required Skills
- Technical Skills
- Strong experience with PySpark and Apache Spark




- Hands on experience with Databricks
- Proficiency in Python programming
- Experience in building ETL pipelines and data transformations
- Robust SQL knowledge and query optimization skills
- Experience with Azure Data Factory ADF AWS Glue or similar tools
- Knowledge of Delta Lake Medallion Architecture and Data Lakes
- Experience with Git CI CD pipelines and Agile methodologies

Preferred Skills:

Technology->Data Engineering->Databricks,Technology->Big Data - Data Processing->PySpark

📌 Pyspark databricks (India)
🏢 Infosys
📍 India