Data Engineer (PySpark, SQL, Azure) - 8+ (Remote)
? Location: Remote (India)
? Type: Contract Full time (Immediate Joiners or 15 Days Notice Only)
⏳ Experience: Mid-Level to Senior (8+ Years)
Position Overview
We are seeking a highly skilled Data Engineer to design, build, and optimize next-generation data architecture and pipelines. This role requires expert-level proficiency in PySpark and SQL to build scalable, high-performance ETL/ELT processes that transform raw data into actionable insights.
Key Responsibilities
• Pipeline Development: Architect, develop, and maintain robust, fault-tolerant ETL/ELT pipelines for ingesting data from diverse sources.
• PySpark Expertise: Write and optimize complex data transformation jobs using PySpark and the Spark DataFrame API for large-scale data processing.
• SQL Mastery: Utilize Advanced SQL for complex querying, data manipulation, and performance tuning.
• Architecture & Automation: Design optimal data models (Dimensional Modeling, Snowflake Schema) and implement orchestration tools (e.g., Airflow, ADF).
• Quality & Collaboration:
Implement data validation routines and collaborate closely with Data Scientists, Analysts, and stakeholders.
Required Technical Skills
• PySpark: Expert-level, hands-on experience in developing and optimizing large-scale data processing applications.
• SQL: Mastery of Advanced SQL (window functions, complex joins, query performance tuning) across platforms like Snowflake, Redshift, PostgreSQL.
• Programming & Big Data: Strong proficiency in Python/Pandas and solid understanding of distributed systems architecture.
• ETL/ELT: Proven experience in designing and maintaining robust data pipelines.
Preferred Qualifications
• Hands-on experience with major Cloud Platforms (specifically Azure data-related services).
• Familiarity with workflow orchestration tools (Apache Airflow) and streaming data (Kafka, Spark Structured Streaming).
? How to Apply: Send your resume to
[email protected].
DataEng
📌 Data Engineer (India)
🏢 Indivize Technologies
📍 India