Its a 1 round of interview for Big 4 which happen Face-to-face on 21st Nov(Sat)

( Bangalore) after offer will released are you ok with this?

Senior Associate_ Databricks Senior Data Engineer_Data and Analytics_Advisory_Bangalore

You will work on:

- Build reliable, scalable batch and streaming pipelines using PySpark and Databricks Workflows/Jobs.
- Implement Delta Lake best practices: schema enforcement/evolution, ACID transactions, time travel, OPTIMIZE/ZORDER, and VACUUM.
- Develop Delta Live Tables (DLT) or Structured Streaming pipelines with Auto Loader for ingestion.
- Write robust modular code in Python (packaging, logging, configuration, error handling).
- Write productive and scalable SQL queries for data extraction and reporting.
- Optimize Spark jobs (partitions, joins, bucketing, caching, AQE, broadcast hints, file sizing).
- Familiarity with SQL Warehouses, Photon, DBR versions, cluster policies, and pools.
- Workflows/Jobs orchestration: tasks, dependencies, parameters, retries, alerts, schedules.




- Unity Catalog: permissions, data lineage, audit, external locations, shares; catalog-first design.
- Ensure idempotent, repayable, and incremental pipelines with proper checkpoints and watermarks.
- Implement data quality checks (expectations tests in DLT/Great Expectations), unit/integration tests for pipelines.
- Ensure data quality, integrity, and governance across the pipeline.
- Work with CI/CD tools to automate deployment and testing of data workflows.
- Work with stakeholders to define data SLAs, schemas, and interface contracts.
- Document pipelines, data sets (data dictionary, lineage), and usage guidelines.

Primary Skills:

- Databricks Platform
- PySpark
- SQL
- Data Modelling
- Python (Scripting, Data Manipulation)
- Azure ADF, ADLS

Secondary Skills:

- CI/CD (Git, Jenkins, Azure DevOps, etc.)

📌 Data Engineer (Bengaluru)
🏢 Advent Talent Solutions
📍 Bengaluru