The AWS Data Engineer will design and implement robust data solutions using AWS services, ensuring seamless integration, storage, processing, and analysis of data. The role involves collaborating with cross-functional teams to support business objectives and optimize data workflows.
- Design, build, and manage scalable and reliable data pipelines on AWS.
- Extract, transform, and load (ETL) data from various sources into AWS storage solutions like S3, Redshift, and RDS.
- Implement data lake and warehouse solutions for structured and unstructured data.
- Write and optimize SQL queries and scripts for data transformation and aggregation.
- Monitor data workflows for performance, availability, and accuracy.
- Optimize data pipelines for cost-efficiency and scalability.
- Work closely with data scientists, analysts,
and other engineers to understand data needs and ensure end-to-end solutions.
- Assist in the integration of third-party tools and APIs.
- Ensure data security by implementing IAM policies, encryption, and other AWS security best practices.
- Automate repetitive tasks to improve efficiency and reduce errors.
- Robust programming skills in Python.
- Familiarity with workflow orchestration tools like Apache Airflow or AWS Step Functions.
- Strong problem-solving and analytical skills.
- Effective communication to collaborate with diverse teams.
- Attention to detail and a commitment to delivering high-quality solutions.