Job Title: Python PySpark

Experience: 6+ years

Job Type: India, Full-Time, Remote

Innovya Technologies is a fast‑growing software consulting firm specializing in business automation and up-to-date digital solutions. We enable organizations to accelerate value realization from their technology and transformation initiatives through expert, context‑driven product development and high‑quality engineering services.

Innovya fosters a high‑energy, collaborative environment built on continuous learning, accountability, and disciplined execution. We value individuals who are results‑oriented, growth‑focused, and motivated to make meaningful contributions. If you thrive in a supportive, fast‑paced setting and enjoy solving complex challenges, Innovya offers an ideal place to grow and make an impact.

About the Role:

We are looking for an experienced PySpark Developer with deep expertise data streaming development and deployment to join our dynamic team. A Python developer who will play a key role in maintaining, enhancing, and modernizing critical high volume and real time data pipelines. The ideal candidate will have a robust background in building scalable, fault-tolerant, and high-performance ELT applications, with a keen interest in AI enabled development.

You will work closely with cross-functional teams to design, analyse, build and deploy complex systems, providing technical direction and expertise to ensure the delivery of robust, efficient, and scalable solutions.

Key Responsibilities:

- Design, develop, and optimize scalable data pipelines using PySpark
- Process and analyze large datasets in distributed environments
- Collaborate with data engineers, analysts, and stakeholders to deliver data solutions
- Write efficient, reusable,



and reliable PySpark code
- Perform data cleansing, transformation, and validation
- Optimize Spark jobs for performance and cost efficiency
- Integrate data from various sources (databases, APIs, streaming platforms, etc.)
- Troubleshoot and debug data pipeline issues
- Maintain documentation for data workflows and processes

Required Skills:

- 6+ years of total experience in Python development, with at least 5 years in Apache Spark (PySpark).

· Hands-on experience with Python and PySpark in production data engineering environments.

· Use Spark DataFrames for large-scale data transformation and processing.

· Experience with complex aggregations, window functions, conditional aggregations, and performance tuning in Spark.

· Good understanding of Spark execution concepts such as partitions, shuffles, join behavior, and optimization techniques.

· Experience with API integrations, including pagination, request handling, response transformation, and fault tolerance.

· Working knowledge of AWS, preferably with services such as S3, Lambda, Glue, EMR, Athena, Step Functions, or similar.

· Experience handling JSON and schema-based parsing in distributed data pipelines.

· Familiarity with SQL-based data extraction and transformation.

· Ability to write production-quality, maintainable, and testable code.

· Good analytical and problem-solving skills, with the ability to understand complex data flows and business rules.

· Experience identifying bugs, troubleshooting processing issues, performing root cause analysis, and implementing fixes in distributed systems.

· Ability to create and maintain unit tests to improve code quality and reduce regressions.

· Preferred - 2+ years of DataBricks and Data Lake development experience

📌 Data Engineer (Raipur)
🏢 Innovya Technologies
📍 Raipur