Role Description

We are seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines and data processing solutions in a cloud-based environment. The ideal candidate will have strong hands-on experience with Google Cloud Platform (GCP), Apache Airflow DAGs, Java, Apache Spark, and SQL. The Data Engineer will work closely with data architects, software engineers, analysts, and business stakeholders to build reliable and high-performance data solutions. This role requires robust programming, data engineering, troubleshooting, and problem-solving skills, with a focus on developing production-grade batch and data processing pipelines. Key Responsibilities Design, develop, test, and maintain scalable data pipelines and ETL/ELT workflows. Develop and manage Apache Airflow DAGs for workflow orchestration, scheduling, monitoring, and dependency management. Build and optimize Apache Spark jobs using Java for large-scale data processing and transformation. Develop data processing solutions using GCP services and cloud-native technologies.



Write complex and optimized SQL queries for data extraction, transformation, validation, and analysis. Work with large and complex datasets across structured and semi-structured data sources. Implement data ingestion, transformation, cleansing, and integration pipelines. Monitor production pipelines and troubleshoot failures, performance issues, and data-quality problems. Optimize Spark jobs, SQL queries, and data pipelines for performance, scalability, and cost efficiency. Implement appropriate error handling, logging, ing, retry mechanisms, and operational monitoring. Develop reusable and maintainable data engineering components following coding and design best practices. Collaborate with application developers, data analysts, data scientists, and other engineering teams. Participate in code reviews and contribute to development standards and best practices. Ensure data pipelines meet requirements for relia

📌 Lead I (Thiruvananthapuram)
🏢 UST
📍 Thiruvananthapuram