Architect- Data Engineering (Architect - 2 )

Location: Bangalore

Mode: Hybrid

Notice period: Max 30 days (Immediate / short notice preferred)

Experience: Architect: 14+ Years

:

Role Summary

We are looking for an experienced Architect - Data Engineering with 14+ years of experience and robust expertise in AWS, Databricks, PySpark, and Python . The ideal candidate will lead the design, development, and optimization of scalable data platforms and pipelines, provide technical leadership to a team of data engineers, and collaborate with business and technology stakeholders to deliver secure, reliable, and high-performing data solutions.

Key Skills

- AWS Cloud Services (S3, Glue, EMR, Redshift, Lambda, IAM, EC2)
- Databricks
- PySpark
- Apache Airflow
- ETL/ELT Pipeline Development
- Data Modeling & Data Architecture
- SQL & Performance Optimization
- Python

Must-Have Skills

- Experience with Databricks for data engineering and analytics
- Strong expertise in AWS Cloud Services (S3, Glue, EMR, Lambda, Redshift, IAM, EC2)
- Expert-level proficiency in PySpark for large-scale data processing
- Hands-on experience in designing and developing ETL/ELT pipelines
- Advanced SQL and Data Modeling skills
- Strong programming experience in Python





Good-to-Have Skills

- Big Data technologies such as Spark Streaming, Hadoop, and Kafka
- CI/CD and DevOps practices
- Infrastructure as Code ( Terraform, CloudFormation )
- Docker and Kubernetes
- Agile methodologies
- Exposure to Snowflake, Azure Data Factory, or GCP Data Services

Roles & Responsibilities

- Lead the design, development, and implementation of scalable data engineering solutions
- Define architecture standards and best practices for data platforms
- Review technical designs and ensure adherence to engineering standards
- Drive technical innovation and continuous improvement initiatives
- Mentor and guide a team of Data Engineers
- Foster a collaborative and high-performance engineering culture
- Build and maintain data pipelines using AWS, Databricks, PySpark, and Airflow
- Develop high-performance ETL/ELT processes for structured and unstructured data
- Design and optimize data models to support reporting, analytics, and machine learning use cases
- Ensure high availability, reliability, and scalability of data platforms

Regards Vibha Patel [email protected]

📌 Architect - Data Engineering (Bengaluru)
🏢 Quantum Integrators
📍 Bengaluru