Are you a Senior Data Engineer with 8+ years of experience in building scalable data platforms and pipelines? Apply now for a remote long-term contract opportunity with a leading client in London.

Requirements

- 8+ years of experience in Data Engineering

- Solid expertise in Python and extensive experience with PySpark and distributed data processing

- Expert-level knowledge of AWS services including EMR, Glue, Lambda, S3, Athena, Redshift, ECS/EKS, Step Functions, EventBridge, IAM, CloudWatch, and SNS/SQS

- Hands-on experience building data lakes and modern data platforms

- Strong understanding of ETL/ELT frameworks and data modelling

- Strong SQL skills and query optimization experience

- Experience with NoSQL databases, particularly DynamoDB, including partitioning, indexing, and data access patterns

- Hands-on experience with OpenSearch/Elasticsearch, including designing indexes, search schemas, query optimisation, and search performance tuning

- Working knowledge of Java and understanding of object-oriented programming and software engineering best practices

- Experience implementing analytics and monitoring dashboards

- Strong problem-solving and analytical skills

- Ability to work in a fast-paced, cloud-native environment

- Excellent communication and stakeholder management skills

- Strong ownership and accountability mindset with ability to mentor and guide team members

- (Desirable) Experience with streaming platforms such as Kafka or Kinesis

- (Desirable)



Experience with Airflow or other workflow orchestration tools

- (Desirable) Experience with Docker and Kubernetes

- (Desirable) Exposure to Data Lake and Lakehouse architecture

- (Desirable) Experience in Agile/Scrum development environments

- (Desirable) Experience with Apache Kafka, Apache Airflow, Terraform, Databricks, Delta Lake, Snowflake, CI/CD pipelines, or Machine Learning data pipelines

- (Desirable) Bachelor's or Master's degree in Computer Science, Engineering, or a related field

- Language(s): English (fluent)

- Design, develop, and maintain large-scale data pipelines using Python and PySpark

- Build and optimise batch and near real-time data processing solutions

- Design scalable data architectures on AWS

- Develop ETL/ELT workflows for ingesting, transforming, and publishing datasets

- Work with structured and semi-structured data from multiple sources

- Design and maintain NoSQL database solutions using DynamoDB

- Implement search and analytics solutions using OpenSearch

- Optimise data processing jobs for performance, scalability, and cost efficiency

- Collaborate with product teams, architects, and software engineers to define data requirements

- Monitor, troubleshoot, and resolve data platform and pipeline issues

- Implement data quality, governance, security, and observability best practices

- Participate in code reviews and mentor junior engineers

📌 Senior Data Engineer (India)
🏢 Nova Hunte
📍 India