Architect- Data Engineering (Architect - 2 )
Location: Bangalore
Mode: Hybrid
Notice period: Max 30 days (Immediate / short notice preferred)
Experience: Architect: 14+ Years

Job Description:

Role Summary
We are looking for an experienced Architect - Data Engineering with 14+ years of experience and strong expertise in AWS, Databricks, PySpark, and Python . The ideal candidate will lead the design, development, and optimization of scalable data platforms and pipelines, provide technical leadership to a team of data engineers, and collaborate with business and technology stakeholders to deliver secure, reliable, and high-performing data solutions.

Key Skills
AWS Cloud Services (S3, Glue, EMR, Redshift, Lambda, IAM, EC2)
Databricks
PySpark
Apache Airflow
ETL/ELT Pipeline Development
Data Modeling & Data Architecture
SQL & Performance Optimization
Python

Must-Have Skills
Experience with Databricks for data engineering and analytics
Solid expertise in AWS Cloud Services (S3, Glue, EMR, Lambda, Redshift, IAM, EC2)
Expert-level proficiency in PySpark for large-scale data processing
Hands-on experience in designing and developing ETL/ELT pipelines
Advanced SQL and Data Modeling skills




Strong programming experience in Python

Good-to-Have Skills
Big Data technologies such as Spark Streaming, Hadoop, and Kafka
CI/CD and DevOps practices
Infrastructure as Code ( Terraform, CloudFormation )
Docker and Kubernetes
Agile methodologies
Exposure to Snowflake, Azure Data Factory, or GCP Data Services

Roles & Responsibilities
Lead the design, development, and implementation of scalable data engineering solutions
Define architecture standards and best practices for data platforms
Review technical designs and ensure adherence to engineering standards
Drive technical innovation and continuous improvement initiatives
Mentor and guide a team of Data Engineers
Foster a collaborative and high-performance engineering culture
Build and maintain data pipelines using AWS, Databricks, PySpark, and Airflow
Develop high-performance ETL/ELT processes for structured and unstructured data
Design and optimize data models to support reporting, analytics, and machine learning use cases
Ensure high availability, reliability, and scalability of data platforms

Regards
Vibha Patel

📌 Architect - Data Engineering (Bengaluru)
🏢 Quantum Integrators
📍 Bengaluru