Key Responsibilities

Design, develop, and maintain scalable data pipelines using PySpark.

Process and transform large datasets for analytics and reporting.

Build ETL/ELT workflows for structured and unstructured data.

Optimize Spark jobs for performance, scalability, and reliability.

Work with data lakes, data warehouses, and distributed computing frameworks.

Collaborate with data analysts, data scientists, and business stakeholders.

Implement data quality checks and monitoring solutions.

Troubleshoot production issues and improve existing data processes.

Participate in code reviews and follow software engineering best practices.

Technical Skills

Solid programming experience in Python.

Hands-on experience with PySpark and Apache Spark.

Knowledge of SQL and database concepts.

Experience with ETL/ELT development.

Understanding of distributed data processing.

Familiarity with data modeling and data warehousing concepts.

Experience with Git and CI/CD practices.

📌 Data Engineer Hyderabad (India)
🏢 JAV Consultancy Services
📍 India