Experience Band: 6 – 10 years

Location: Bangalore.

Band: Sr. Consultant and Manager

Mandatory Skills: Python, PySpark, SQL and Databricks.

Responsibilities

- Development experience as a data engineer with focus on core tools and technologies like AWS Lambda, Glue, S3, Pyspark, SQL and Databricks. Experience in Redshift, Athena and Airflow is added advantage.
- Design, develop, and optimize ETL/ELT data pipelines using Glue / Lamda with Databricks.
- Strong SQL and Python/PySpark skills for data transformation and analysis.
- Work with structured, semi-structured, and unstructured data sources.
- Troubleshoot and optimize data workflows for scalability and performance.
- Strong experience with optimized performance-oriented ELT/ETL design and implementations in context of large and complex datasets using PySpark, SQL, Databricks, Glue or Lamda
- Design and build high-performance and scalable data pipelines adhering to delta/ medallion/ data-lake house, data warehouse & data marts standards for optimal storage, retrieval, and processing of data.
- Databricks: Hands-on experience with Databricks for data processing, analytics, and Serverless SQLWH, Unity Catalogue, optimization of code and cluster.
- Develop data profiling and data quality methodologies and embed them into the processes involved in transforming data across the systems.
- Experience in Agile Development and code deployment using Github & CI-CD pipelines.
- Ability to work with business owners to define key business requirements and convert to technical specifications




- Experience with security models and development on large data sets
- Responsible for system testing, ensuring effective resolution of defects, timely discussion around business issues and appropriate management of resources relevant to data and integration

Secondary skills:

- Project management – status reports, stand up, knowledge on Agile methodology
- Stake holder management
- Mentor data engineers and analysts
- Provide technical direction and design reviews
- A determined focus on the user and user experience when problem- solving
- Team player and able to collaborate, proactively communicates and shares understanding and experience within a team
- Clear communicator and able to work in a multicultural/lingual environment
- Uses initiative and owns their deliverables end-to-end
- Brings ideas to the table, is inquisitive and excited by new technology
- Used to working in a versatile, independent manner on a mixture of small unstructured and large structured items
- Provides regular, considered feedback and regularly strives to improve their own ways of working

Preferred Qualifications / Certifications

- Bachelor's degree in computer science, information technology, management information systems or equivalent work experience
- Experience working in regulated environments and with internal systems quality policies and procedures.
- Experience in development and deployment on cloud infrastructure.
- Pharmaceutical or healthcare industry experience

AWS and Databricks certification is good to have.

📌 Databricks Data Engineer - Deloitte (Bengaluru)
🏢 Trigent Software
📍 Bengaluru