- Design, implement, and manage scalable data pipelines and analytics solutions using AWS services (Glue, Redshift) and Azure tools (Data Factory, Synapse).
1. Cloud IoT Services Expertise:
- Work with IoT cloud services, including AWS IoT Core and Azure IoT Hub, to collect, process, and analyze IoT data efficiently.
1. Data Integration:
- Develop solutions for integrating data from diverse sources, ensuring real-time or batch processing as per business requirements.
1. Performance Optimization:
- Optimize performance of cloud-based data solutions to handle large-scale datasets effectively.
1. Troubleshooting and Debugging:
- Identify and resolve issues in data pipelines, ensuring high availability and reliability of cloud data platforms.
1. Collaboration:
- Work closely with data analysts,
data scientists, and business stakeholders to meet data needs and deliver insights.
1. Documentation and Standards Compliance:
- Maintain comprehensive documentation of data architecture, processes, and workflows while ensuring compliance with best practices and security standards.
Requirements:
- Solid expertise in AWS Glue, AWS Redshift, Azure Data Factory, and Azure Synapse Analytics.
- Hands-on experience with IoT cloud platforms, such as AWS IoT Core and Azure IoT Hub.
- Proficiency in building and managing data pipelines.
- Knowledge of cloud security practices and cost optimization strategies.
- Excellent problem-solving and communication skills.