We are looking for an experienced Data Engineer with strong hands-on experience in Python, PySpark, Apache Spark and SQL to develop and maintain scalable data processing solutions.
Key Responsibilities
- Develop and maintain data processing pipelines using Python and PySpark.
- Work with large datasets using Apache Spark and Spark SQL.
- Write and optimize complex SQL queries.
- Perform data extraction, transformation and loading (ETL/ELT).
- Work on Spark transformations, actions, joins and aggregations.
- Optimize Spark jobs for better performance and scalability.
- Ensure data quality and reliability of data pipelines.
- Troubleshoot and resolve data processing issues.
Required Skills
- Solid hands-on experience in Python and PySpark.
- Good understanding of Apache Spark architecture and concepts.
- Strong SQL skills, including joins, CTEs and window functions.
- Experience in DataFrames, transformations and actions.
- Hands-on experience in developing ETL/ELT pipelines.
- Basic working knowledge of Databricks.
Preferred Skills
- Databricks notebooks, clusters and workflows.
- Delta Lake.
- Spark performance tuning and optimization.
- Azure or AWS cloud platforms.
- Experience with production data pipelines.
Ideal Candidate
A Data Engineer with approximately 5+ years of experience, strong practical knowledge of PySpark, Apache Spark and SQL, and basic exposure to Databricks.
📌 Data Engineer (Bengaluru)
🏢 Perint Infosystems India Private
📍 Bengaluru
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.