09 Sep
|
InterSources
|
India
09 Sep
InterSources
India
Job Title: Senior Databricks Data Engineer
Company: InterSources Inc.
Location: Remote Job
Shift - EST
POSITION SUMMARY:
Overview:
We are seeking a Senior Databricks Data Engineer to design, build, and operate a Data & AI platform with a robust foundation in the Medallion Architecture (raw/bronze, curated/silver, and mart/gold layers). This platform will orchestrate complex data workflows and scalable ELT pipelines to integrate data from enterprise systems such as PeopleSoft, D2L, and Salesforce, delivering high-quality, governed data for machine learning, AI/BI, and analytics at scale.
You will play a senior technical role in guiding engineering standards, reusable patterns, platform reliability, and production readiness across the Data & AI platform.
You will play a critical role in engineering the infrastructure and workflows that enable seamless data flow across the enterprise, ensure operational excellence, and provide the backbone for strategic decision-making, predictive modeling, and innovation.
Responsibilities:
1.
Data & AI Platform Engineering (Databricks-Centric):
Provide senior-level technical leadership in the design, optimization, and standardization of Databricks engineering patterns across the Data & AI platform.
Design, implement, and optimize end-to-end data pipelines on Databricks, following the Medallion Architecture principles.
Build robust and scalable ETL/ELT pipelines using Apache Spark and Delta Lake to transform raw (bronze) data into trusted curated (silver) and analytics-ready (gold) data layers.
Operationalize Databricks Workflows for orchestration, dependency management, and pipeline automation.
Apply schema evolution and data versioning to support agile data development.
2. Platform Integration & Data Ingestion:
Lead the design of reusable ingestion frameworks and integration patterns that support scalable, reliable, and governed data onboarding across enterprise systems.
Connect and ingest data from enterprise system
📌 Senior Databricks Data Engineer (India)
🏢 InterSources
📍 India