We are looking for an experienced Senior Data Engineer with strong hands-on expertise in Python, PySpark, and SQL. The candidate will be responsible for designing, developing, and maintaining scalable data processing solutions and data pipelines.

The ideal candidate should have strong programming and data engineering fundamentals, excellent SQL skills, and experience working with large datasets using PySpark.

Key Responsibilities

- Design, develop, and maintain scalable data pipelines and data processing solutions.
- Develop efficient data transformation and processing applications using Python and PySpark.
- Write complex and optimized SQL queries for data extraction, transformation, and analysis.
- Work with large and structured/unstructured datasets.
- Develop ETL/ELT workflows and ensure data quality and reliability.
- Perform data cleansing, transformation, validation, and reconciliation.
- Optimize PySpark jobs and SQL queries for performance and scalability.
- Troubleshoot data pipeline failures and resolve data quality issues.
- Collaborate with data analysts, developers, architects,



and business stakeholders to understand data requirements.
- Implement appropriate error handling, logging, and monitoring mechanisms.
- Participate in code reviews and follow established coding and data engineering standards.
- Create and maintain technical documentation for data pipelines and processes.
- Support production deployments and resolve data-related issues.
- Contribute to continuous improvement and automation of data engineering processes.

Must-Have Skills

- 46 years of experience in Data Engineering.
- Strong hands-on experience with Python.
- Strong hands-on experience with PySpark.
- Excellent knowledge of SQL.
- Experience developing and optimizing data processing pipelines.
- Strong understanding of ETL/ELT concepts.
- Experience working with large volumes of data.
- Good understanding of data transformation and data quality processes.
- Strong analytical and problem-solving skills.
- Ability to work effectively in a collaborative development workplace

📌 Senior Data Engineer (Bengaluru)
🏢 Orcapod Consulting Services
📍 Bengaluru