This role involves designing, developing, and maintaining cloud-native data solutions while working in an Agile environment.
Key Responsibilities
- Contemporary Data Platform Development:
- Build data lake components on cloud-based platforms.
- Design and develop data marts for business analysts and data scientists.
- Data Engineering & Pipelines:
- Design data pipelines to integrate structured, semi-structured, and unstructured data from multiple sources.
- Implement ETL/ELT processes to transform and cleanse data.
- Ensure data quality and transformation rules align with enterprise standards.
- Work with Medallion architecture and implement best practices for data modeling.
- Agile & DevOps Practices:
- Deliver solutions using Agile methodologies in a CI/CD-driven environment.
- Work on containerized solutions (Azure Kubernetes) and scheduling tools like Azure Scheduler.
- Follow secure coding practices and authentication/authorization protocols.
Candidate Qualifications
- Education: Bachelor’s degree in Computer Science or equivalent.
- Experience:
- 4 - 8 years of experience in data engineering or data application development (ETL/ELT/BI).
- 2+ years of experience in cloud-based data platform development.
- Expertise in building Azure-based data pipelines, including:
- Azure Data Factory / Synapse
- DataBricks / Synapse Spark Pool
- Cosmos DB
- Azure Data Lake Storage (ADLS)
- Dedicated SQL Pool / Azure SQL
- Azure Logic Apps
- Hands-on experience with data transformation & cleansing using Spark, Python, R, SQL.
- Strong understanding of CI/CD, test-driven development, and domain-driven design.