Lead Data Engineer Primary Skills - Athena, Step Functions, Spark - Pyspark, ETL Fundamentals, SQL (Basic + Advanced), Glue, Python, Lambda, Data Warehousing, EBS /EFS, AWS EC2, Lake Formation, Aurora, S3, Modern Data Platform Fundamentals, PLSQL, Data Modelling Fundamentals, Cloud front Specialization - AWS Data EngineerIng Basic: Senior Data Engineer Job requirements - AWS Senior Data Engineer Company: Brillio Technologies Role: Senior Data Engineer Experience: 5-10 Years Location: India (Hybrid/Remote) Employment Type: Full-time Role Overview We are looking for a Senior Data Engineer with strong Databricks and Big Data expertise to design, build, and optimize scalable data pipelines and platforms. The ideal candidate will have hands-on experience with Spark, PySpark, AWS ecosystem, and up-to-date data architectures, along with exposure to APIs and real-time data processing.

Key Responsibilities - Design and build scalable data pipelines using Spark / PySpark on Databricks

- Develop and optimize ETL/ELT workflows for large-scale data processing
- Work with AWS EMR, S3, and Hadoop ecosystem for distributed data processing
- Build and maintain data lake and data warehouse solutions
- Develop and integrate APIs for data ingestion and consumption
- Implement data processing workflows using Airflow / Autosys
- Optimize data performance using partitioning, caching, and query tuning




- Handle structured and semi-structured data from multiple sources
- Ensure data quality, governance, and reliability of pipelines
- Collaborate with cross-functional teams (Analytics, Product, Engineering) Technical Skills (Must-Have) - Strong expertise in:
- Apache Spark / PySpark
- Databricks (mandatory)
- SQL (PostgreSQL or similar) - Hands-on experience with:
- AWS EMR, S3
- Hadoop, Hive ecosystem - Programming skills:
- Python (must-have)
- Scala (good exposure) - Strong knowledge of:
- UNIX / Shell scripting
- ETL pipelines and data engineering fundamentals Additional Skills - Experience with:
- Elasticsearch (data storage & retrieval)
- Workflow orchestration tools (Airflow, Autosys) - Exposure to:
- API development & integration
- Version control tools (Git / SVN) - Basic knowledge of HTML (for web/data tasks) Experience Required - Minimum 5+ years of Data Engineering experience
- Strong hands-on experience in Databricks + Big Data stack
- Proven experience working on large-scale distributed systems Education - Bachelor's or Master's degree in Computer Science, Engineering, or related field Why Join Brillio - Work on cloud-native, large-scale data platforms
- Exposure to cutting-edge Databricks & AWS ecosystems
- Collaborative and high-performance engineering culture

📌 Sr Data Specialist133 (Bengaluru)
🏢 Brillio
📍 Bengaluru