Data Engineer (Pune)

Data Engineer (Pune)

03 Sep
|
Acme Services Private
|
Pune

03 Sep

Acme Services Private

Pune

Job Description:

- Design, develop, and maintain scalable ETL/ELT data pipelines using Python and PySpark.

- Develop PySpark applications for processing and transforming large volumes of structured and semi-structured data.

- Write effective Python scripts and reusable functions for data processing, validation, and automation.

- Develop complex SQL queries for data extraction, transformation, validation, and analysis.

- Work with Apache Spark, Spark SQL, DataFrames, Hive, and Hadoop for distributed data processing.

- Perform data cleansing, transformation, validation, reconciliation, and quality checks.

- Optimize Spark/PySpark jobs using techniques such as partitioning, caching, repartitioning, and broadcast joins.

- Develop and maintain data pipelines for ingestion from multiple source systems.

- Implement error handling, retry mechanisms, and monitoring to ensure reliable data pipelines.

📌 Data Engineer (Pune)
🏢 Acme Services Private
📍 Pune

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: data engineer (pune) / pune

Subscribe to this job alert:

Get the latest job offers by email for: data engineer (pune) / pune