About the Opportunity
A fast-scaling technology services firm operating in the Data Engineering and Cloud Analytics space, we partner with enterprises to build scalable data pipelines, automate ETL workflows, and unlock real-time insights from structured and unstructured datasets. Our Python PySpark developers architect and deploy high-performance data solutions on cloud platforms—enabling faster decision-making, predictive modeling, and regulatory compliance for global clients.

Role & Responsibilities

- Design, develop, and optimize PySpark ETL pipelines for large-scale batch and streaming data processing.

- Collaborate with data scientists and analysts to ingest, transform, and model data for ML training and reporting use cases.

- Implement data quality checks, error handling, and logging frameworks within Spark jobs for production-grade reliability.

- Integrate PySpark workflows with cloud data lakes (AWS S3, Azure Data Lake), metastores (Hive, Glue), and orchestration tools (Airflow, Luigi).





- Performance-tune Spark applications—partitioning, caching, broadcasting, and resource allocation—to reduce job runtimes and costs.

- Write clean, modular, testable Python code with unit/integration tests and document architecture decisions for team scalability.

Skills & Qualifications

Must-Have

- PySpark

- Python

- Apache Spark

- ETL Development

- SQL

- Data Lake Architecture

- AWS S3

- Apache Airflow

Preferred

- Azure Data Lake

- Databricks

- Delta Lake

Benefits & Culture Highlights

- In office collaborative workspace in major Indian tech hubs with modern infrastructure and R&D; labs.

- Opportunities to upskill in cloud-native data platforms and GenAI-powered data pipelines.

- Fast-track career growth with cross-functional exposure to data science, ML engineering, and cloud architecture teams.

📌 Python Pyspark Developer (Thiruvananthapuram)
🏢 Important Group
📍 Thiruvananthapuram