This role offers an exciting prospect to work on diverse projects, collaborating with cross functional teams to design, build, and optimize data pipelines and infrastructure.
Responsibilities
Design, develop, and maintain scalable data pipelines and ETL processes leveraging AWS services such as S3,
Glue, EMR, Lambda, and Redshift.
Collaborate with data scientists and analysts to understand data requirements and implement solutions that support analytics and machine learning initiatives.
Optimize data storage and retrieval mechanisms to ensure performance, reliability, and cost-effectiveness.
Implement data governance and security best practices to ensure compliance and data integrity.
Troubleshoot and debug data pipeline issues, providing timely resolution and proactive monitoring.
Stay abreast of emerging technologies and industry trends, recommending innovative solutions to enhance data engineering capabilities.