Job Summary

What you will do: Data Engineering Platform Development. Design, develop, and maintain scalable batch and real-time data pipelines. Build and optimize ETL/ELT workflows using modern data engineering frameworks.

Develop and manage data lake, data warehouse, lake house architecture and snowflakes. Implement data quality checks, validation frameworks, and monitoring solutions. Optimize data processing jobs for performance, scalability, and cost efficiency.

Collaborate with analytics, BI, and AI teams to deliver trusted datasets.

Cloud Infrastructure

Build and maintain cloud-native data platforms on AWS, Azure, or GCP.

Implement

Infrastructure as Code (IaC) using Terraform or equivalent tools. Support CI/CD implementation for data engineering workloads. Ensure security, compliance, and governance standards across data platforms.

Collaboration Delivery

Work directly with stakeholders to understand business requirements and translate them into technical solutions. Participate in architecture discussions, design reviews, and sprint planning. Perform code reviews and mentor junior engineers. Contribute to technical documentation, standards, and reusable assets.

Innovation Practice Building

Evaluate and recommend modern data engineering tools and frameworks. Contribute to internal accelerators, reusable components, and best practices. Support proof-of-concepts and emerging technology initiatives in analytics, AI, and data platforms.

Responsibilities

- Design, develop, and maintain scalable batch and real-time data pipelines.

- Build and optimize ETL/ELT workflows using modern data engineering frameworks.

- Develop and manage data lake, data warehouse, lake house architecture and snowflakes.

- Implement data quality checks, validation frameworks, and monitoring solutions.

- Optimize data processing jobs for performance, scalability, and cost efficiency.

- Collaborate with analytics, BI, and AI teams to deliver trusted datasets.





- Build and maintain cloud-native data platforms on AWS, Azure, or GCP.

- Implement Infrastructure as Code (IaC) using Terraform or equivalent tools.

- Support CI/CD implementation for data engineering workloads.

- Ensure security, compliance, and governance standards across data platforms.

- Work directly with stakeholders to understand business requirements and translate them into technical solutions.

- Participate in architecture discussions, design reviews, and sprint planning.

- Perform code reviews and mentor junior engineers.

- Contribute to technical documentation, standards, and reusable assets.

- Evaluate and recommend modern data engineering tools and frameworks.

- Contribute to internal accelerators, reusable components, and best practices.

- Support proof-of-concepts and emerging technology initiatives in analytics, AI, and data platforms.

Requirements

- Must have 4 to 5 years of experience in Data Engineering.

- Strong expertise in Python, PySpark, and SQL.

- Hands-on experience with Spark-based data processing at production scale.

- Experience with cloud platforms such as AWS, Azure, or GCP.

- Strong understanding of Data Lakes, Data Warehouses, and Lakehouse architectures.

- Experience with orchestration tools such as Airflow, Azure Data Factory, or similar.

- Experience with data modeling, performance tuning, and pipeline optimization.

- Knowledge of CI/CD, Git, and DevOps practices.

- Strong communication and stakeholder management skills.

- Certifications: Databricks Certified Data Engineer Associate
- Databricks Certified Data Engineer Qualified
- AWS Certified Data Engineer - Associate
- Microsoft Certified: Azure Data Engineer Associate (DP-203)
- Google Professional Data Engine.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Sr Data Engineer (Ahmedabad)
🏢 Creole Studios
📍 Ahmedabad