Roles and Responsibilities:
Data Pipeline and Analytics Development:
Design, implement, and manage scalable data pipelines and analytics solutions using AWS services (Glue, Redshift) and Azure tools (Data Factory, Synapse).
Cloud IoT Services Expertise:
Work with IoT cloud services, including AWS IoT Core and Azure IoT Hub, to collect, process, and analyze IoT data efficiently.
Data Integration:
Develop solutions for integrating data from diverse sources, ensuring real-time or batch processing as per business requirements.
Performance Optimization:
Optimize performance of cloud-based data solutions to handle large-scale datasets effectively.
Troubleshooting and Debugging:
Identify and resolve issues in data pipelines, ensuring high availability and reliability of cloud data platforms.
Collaboration:
Work closely with data analysts, data scientists,
and business stakeholders to meet data needs and deliver insights.
Documentation and Standards Compliance:
Maintain comprehensive documentation of data architecture, processes, and workflows while ensuring compliance with best practices and security standards.
Requirements:
Solid expertise in AWS Glue, AWS Redshift, Azure Data Factory, and Azure Synapse Analytics.
Hands-on experience with IoT cloud platforms, such as AWS IoT Core and Azure IoT Hub.
Proficiency in building and managing data pipelines.
Knowledge of cloud security practices and cost optimization strategies.
Excellent problem-solving and communication skills.