17 Nov
|
Impressico Business Solutions
|
Noida
17 Nov
Impressico Business Solutions
Noida
Job highlights
- - Hands-on experience working with multiple databases. .
Close Data Engineer Databricks
Impressico is looking for data engineers to work on building data ingestion and transformation pipelines, and their orchestration, on the Databricks Lakehouse Platform for its clients projects.
- Create and maintain optimal data pipeline architectures on the Databricks Lakehouse Platform
- Optimize data delivery and Spark workloads for greater scalability, performance and cost efficiency
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using PySpark, Spark SQL and Delta Lake on Databricks
- Work with internal and external stakeholders to assist with data-related technical issues and support data infrastructure needs
- enabling and running data ingestions from different sources like DB, streaming, files, SFTP using Auto Loader, Structured Streaming and Delta Live Tables.
- performing thorough testing and validation in order to support the accuracy of data transformations and data verification to be used by downstream applications
- Recommend different ways to constantly improve data reliability, quality and governance (Unity Catalog).
Job Specification
Coding in Python
- Coding in PySpark Python
- Advance SQL querying (Spark SQL / Databricks SQL)
- Hands-on experience in the following: Databricks Workspace, Delta Lake, Delta Live Tables (DLT), Auto Loader, Unity Catalog, Databricks Workflows/Jobs, Databricks SQL and Spark Structured Streaming.
- Orchestration experience of data pipelines using Databricks Workflows including automation
- CICD (Databricks Asset Bundles, Git integration / Repos)
- Hands-on experience in ETL/ELT tools, frameworks, processes including Medallion (Bronze/Silver/Gold) architecture
- Hands-on experience working on a data warehouse / lakehouse
- Hands-on experience working with multiple databases
This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
📌 Data Engineer Databricks (Noida)
🏢 Impressico Business Solutions
📍 Noida