Key Responsibilities
ETL Pipeline Development: Design, build, and maintain batch and streaming data pipelines using Apache Spark, PySpark, and Delta Lake.
Lakehouse Architecture: Implement medallion architecture (Bronze, Silver, Gold layers) to process data incrementally using tools like Delta Live Tables (DLT).
Orchestration & Automation: Schedule and monitor data workflows using Databricks Jobs and Workflows.
Data Governance & Security: Enforce data access policies, lineage, and security compliance utilizing Unity Catalog.
Performance Tuning: Optimize Spark clusters, query performance, and infrastructure costs within cloud settings
📌 Opening For Data Engineer Databricks Noida
🏢 EXL
📍 Noida