We are looking for an experienced Data Engineer to design build and maintain scalable data pipelines and infrastructure that power analytics reporting and machine learning initiatives across the organization. The ideal candidate has a strong foundation in data modelling distributed systems and cloud-based data platforms with a track record of delivering reliable high-performance data solutions.
Key Responsibilities:
- Design build and optimize scalable ETL/ELT pipelines to ingest transform and load data from diverse sources (databases APIs streaming platforms third-party systems).
- Develop and maintain data warehouse/data lake architectures ensuring data quality consistency and reliability
- Collaborate with data scientists analysts and product teams to understand data requirements and deliver well-structured accessible datasets
- Build and maintain batch and real-time streaming data pipelines using tools like Apache Spark Kafka or Airflow
- Implement data quality checks monitoring and alerting to ensure pipeline reliability and data integrity
- Optimize database and query performance for large-scale datasets
- Design and maintain data models (conceptual logical physical) and schemas that support analytics and application needs
- Work with cloud platforms (AWS/Azure/GCP) to manage data infrastructure storage and compute resources
- Implement and enforce data governance security and compliance best practices
- Participate in code reviews CI/CD pipeline development and infrastructure-as-code practices
- Document data pipelines architecture and processes for team knowledge sharing
- Troubleshoot and resolve production data pipeline issues in a timely manner
Required Skills & Qualifications
- Bachelors degree in Computer Science Engineering or a related field
- 8 years of hands-on experience as a Data Engineer or in a similar role