15 Oct
|
Emagine
|
Bengaluru
Summary The main purpose of the Data Engineer role is to contribute to the Core Engineering team by developing scalable and maintainable data platforms that enhance the decision-making process and support technological advancements within the organization.
Main Responsibilities
- Collaborate with analysts, architects, and stakeholders to understand requirements and produce maintainable solutions.
- Analyse raw data against requirements and provide early input for JIRA tickets, along with design and development activities.
- Design, develop, and maintain scalable, robust data pipelines and workflows to support data ingestion, transformation, and storage.
- Architect and optimize data models, ELT processes, and data architecture to ensure efficiency, quality, and security.
- Lead initiatives to improve existing data infrastructure, including migration, automation, and performance tuning.
- Monitor data quality and pipeline performance, troubleshoot issues, and minimize downtime.
- Ensure compliance with data governance, security policies, and privacy standards.
- Collaborate within a multidisciplinary engineering squad and provide technical guidance where necessary.
- Write and maintain documentation and operational runbooks.
- Support and improve the deployment processes.
Key Requirements
- Robust analytical skills in working with both structured & unstructured datasets.
- In-depth knowledge of data modelling, data architecture, and ETL/ELT design principles.
- Expert-level proficiency in SQL.
- Proven track record developing data platforms on Cloud databases, ideally Snowflake.
- Hands-on experience working with AWS, particularly data services such as RDS, S3, Glue, etc.
- Proficiency in Python programming, including writing tests and maintaining code quality.
Nice to Have
- Experience with DBT tooling.
- Experience processing raw data at scale (e.g., scripting, web scraping, API calls).
- Advocate of Continuous Integration/Continuous Deployment (CI/CD) and Test-Driven Development (TDD).
Other Details Team Structure: Multidisciplinary engineering squad.
Technology Stack: AWS technologies and other third-party systems.
📌 Data Engineer - Junior (Bengaluru)
🏢 Emagine
📍 Bengaluru