Job Description – Data Engineer Position-Data Engineer Experience-5 – 10 Years
LOCATION: NEWZEALAND (ON-SITE OPPORTUNITY)
Job Summary We are seeking an experienced
Data Engineer
with strong expertise in building scalable and high-performance data platforms. The ideal candidate will have hands-on experience with
Apache Airflow, Apache Spark, PySpark, Scala, Databricks, Docker, and Advanced SQL , with a proven track record of developing and optimizing large-scale data pipelines and ETL processes.
Key Responsibilities Design, develop, and maintain scalable data pipelines and data processing frameworks. Build and orchestrate ETL/ELT workflows using
Apache Airflow . Develop data engineering solutions using
Apache Spark, PySpark, and Scala . Leverage
Databricks
for large-scale data processing, optimization, and analytics. Write, optimize, and troubleshoot complex SQL queries involving: Inner, Left, Right, and Full Joins Self Joins CTEs and Subqueries Window Functions Query Performance Tuning Develop, deploy, and manage containerized applications using
Docker . Collaborate with cross-functional teams to gather requirements and deliver robust data solutions. Implement data quality checks, monitoring, and performance tuning. Ensure data governance, security, and compliance standards are maintained. Troubleshoot and resolve production issues related to data pipelines and processing jobs.




Mandatory Skills Apache Airflow
– DAG creation, workflow orchestration, scheduling, and monitoring. Apache Spark
– Distributed data processing and performance optimization. PySpark
– ETL development, transformations, and data engineering. Scala Programming
– Strong hands-on development experience with Spark/Scala applications. Databricks
– Notebooks, workflows, Delta Lake, and cluster management. Advanced SQL Strong expertise in Joins (Inner, Outer, Left, Right, Full, Self)
Window Functions Query Optimization Data Modeling Concepts Performance Tuning Docker Containerization Image Creation and Management Docker Compose Deployment and Troubleshooting
Python Programming Git Version Control
Preferred Skills Delta Lake Azure, AWS, or GCP Kafka or Streaming Technologies CI/CD Pipelines (Azure Dev Ops, Jenkins, Git Hub Actions) Data Warehousing Concepts Linux/Unix Administration Lakehouse Architecture
Qualifications Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field. 5–10 years of experience in Data Engineering and Big Data technologies. Solid expertise in Spark ecosystem and distributed computing. Experience building enterprise-scale data platforms and ETL solutions. Excellent analytical, problem-solving, and communication skills.

📌 Data Engineer (India)
🏢 Talentgigs
📍 India