Experience Band: 6 – 10 years

Location: Bangalore.

Band: Sr. Consultant and Manager

Mandatory Skills: Python, PySpark, SQL and Databricks.

Responsibilities
Development experience as a data engineer with focus on core tools and technologies like AWS Lambda, Glue, S3, Pyspark, SQL and Databricks. Experience in Redshift, Athena and Airflow is added advantage.
Design, develop, and optimize ETL/ELT data pipelines using Glue / Lamda with Databricks.
Robust SQL and Python/PySpark skills for data transformation and analysis.
Work with structured, semi-structured, and unstructured data sources.
Troubleshoot and optimize data workflows for scalability and performance.
Solid experience with optimized performance-oriented ELT/ETL design and implementations in context of large and complex datasets using PySpark, SQL, Databricks, Glue or Lamda
Design and build high-performance and scalable data pipelines adhering to delta/ medallion/ data-lake house, data warehouse & data marts standards for optimal storage, retrieval, and processing of data.
Databricks:



Hands-on experience with Databricks for data processing, analytics, and Serverless SQLWH, Unity Catalogue, optimization of code and cluster.
Develop data profiling and data quality methodologies and embed them into the processes involved in transforming data across the systems.
Experience in Agile Development and code deployment using Github & CI-CD pipelines.
Ability to work with business owners to define key business requirements and convert to technical specifications
Experience with security models and development on large data sets
Responsible for system testing, ensuring effective resolution of defects, timely discussion around business issues and appropriate management of resources relevant to data and integration

Secondary skills:
Project management – status reports, stand up, knowledge on Agile methodology
Stake holder management
Mentor data engineers and analysts
Provide technical

📌 Databricks Data Engineer Bengaluru
🏢 Trigent Software
📍 Bengaluru