Lead Data Engineer

Primary Skills
- Athena, Step Functions, Spark - Pyspark, ETL Fundamentals, SQL (Basic + Advanced), Glue, Python, Lambda, Data Warehousing, EBS /EFS, AWS EC2, Lake Formation, Aurora, S3, Modern Data Platform Fundamentals, PLSQL, Data Modelling Fundamentals, Cloud front

Specialization
- AWS Data EngineerIng Basic: Senior Data Engineer

Job requirements

Job Description – AWS Senior Data Engineer

Company: Brillio Technologies
Role: Senior Data Engineer
Experience: 5–10 Years
Location: India (Hybrid/Remote)
Employment Type: Full-time

Role Overview

We are looking for a Senior Data Engineer with solid Databricks and Big Data expertise to design, build, and optimize scalable data pipelines and platforms. The ideal candidate will have hands-on experience with Spark, PySpark, AWS ecosystem, and modern data architectures, along with exposure to APIs and real-time data processing.

Key Responsibilities
- Design and build scalable data pipelines using Spark / PySpark on Databricks
- Develop and optimize ETL/ELT workflows for large-scale data processing




- Work with AWS EMR, S3, and Hadoop ecosystem for distributed data processing
- Build and maintain data lake and data warehouse solutions
- Develop and integrate APIs for data ingestion and consumption
- Implement data processing workflows using Airflow / Autosys
- Optimize data performance using partitioning, caching, and query tuning
- Handle structured and semi-structured data from multiple sources
- Ensure data quality, governance, and reliability of pipelines
- Collaborate with cross-functional teams (Analytics, Product, Engineering)

Technical Skills (Must-Have)
- Strong expertise in:
- Apache Spark / PySpark
- Databricks (mandatory)
- SQL (PostgreSQL or similar)
- Hands-on experience with:
- AWS EMR, S3
- Hadoop, Hive ecosystem
- Programming skills:
- Python (must-have)
- Scala (good exposure)
- Strong knowledge of:
- UNIX / Shell scripting
- ETL pipelines and data engineering fundamentals

Additi

📌 Sr Data Specialist (Bengaluru)
🏢 Brillio
📍 Bengaluru