Job Description – Data EngineerPosition-Data EngineerExperience-5 – 10 YearsLOCATION: NEWZEALAND (ON-SITE OPPORTUNITY)Job SummaryWe are seeking an experienced Data Engineer with solid expertise in building scalable and high-performance data platforms. The ideal candidate will have hands-on experience with Apache Airflow, Apache Spark, PySpark, Scala, Databricks, Docker, and Advanced SQL , with a proven track record of developing and optimizing large-scale data pipelines and ETL processes.Key ResponsibilitiesDesign, develop, and maintain scalable data pipelines and data processing frameworks.Build and orchestrate ETL/ELT workflows using Apache Airflow .Develop data engineering solutions using Apache Spark, PySpark, and Scala .Leverage Databricks for large-scale data processing, optimization, and analytics.Write, optimize, and troubleshoot complex SQL queries involving:Inner, Left, Right, and Full JoinsSelf JoinsCTEs and SubqueriesWindow FunctionsQuery Performance TuningDevelop, deploy, and manage containerized applications using Docker .Collaborate with cross-functional teams to gather requirements and deliver robust data solutions.Implement data quality checks, monitoring, and performance tuning.Ensure data governance, security,



and compliance standards are maintained.Troubleshoot and resolve production issues related to data pipelines and processing jobs.Mandatory SkillsApache Airflow – DAG creation, workflow orchestration, scheduling, and monitoring.Apache Spark – Distributed data processing and performance optimization.PySpark – ETL development, transformations, and data engineering.Scala Programming – Strong hands-on development experience with Spark/Scala applications.Databricks – Notebooks, workflows, Delta Lake, and cluster management.Advanced SQLStrong expertise in Joins (Inner, Outer, Left, Right, Full, Self)Window FunctionsQuery OptimizationData Modeling ConceptsPerformance TuningDockerContainerizationImage Creation and ManagementDocker ComposeDeployment and TroubleshootingPython ProgrammingGit Version ControlPreferred SkillsDelta LakeAzure, AWS, or GCPKafka or Streaming TechnologiesCI/CD Pipelines (Azure DevOps, Jenkins, GitHub Actions)Data Warehousing ConceptsLinux/Unix AdministrationLakehouse ArchitectureQualificationsBachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.5–10 years of experience in Data Engineering and Big Data technologies.Strong expertise in Spark ecosystem and distributed computing.Experience building enterprise-scale data platforms and ETL solutions.Excellent analytical, problem-solving, and communication skills.

📌 Data Engineer (Ernakulam)
🏢 Talentgigs
📍 Ernakulam