Seeking a Specialist with 5 to 7 years of experience in GCP BigQuery and Databricks to design, develop and optimize scalable data solutions within the GCP Databricks ecosystem.
Design, build and maintain scalable data pipelines and ETL/ELT processes using GCP BigQuery and Databricks.
Develop and optimize Databricks notebooks and jobs leveraging Apache Spark and PySpark for productive data processing.
Implement data ingestion transformation and storage solutions using GCP native tools.
Ensure data quality validation and reconciliation across pipelines.
Collaborate with cross-functional teams to align data solutions with business requirements.
Monitor and troubleshoot pipeline performance and optimize for scalability and reliability.
Maintain compliance with data governance security and best practices within cloud settings.
Stay current with emerging technologies and best practices in cloud data engineering and Databricks implementations.
Responsibilities
Lead the design and implementation of data workflows and pipelines on GCP and Databricks platforms.
Develop and optimize Spark jobs notebooks and workflows for processing large enterprise datasets.
Apply data engineering best practices to enhance performance scalability and reliability of data solutions.
Collaborate with data scientists analysts and architects to support analytics AI and machine learning workflows.
Participate in code and design reviews to ensure high-quality deliverables.
Support production data pipelines including monitoring incident resolution and continuous improvement.
Mentor and guide junior team members on technology evaluations design and implementation.
Integrate Databricks pipelines with upstream and downstream systems and cloud services.
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
📌 Specialist Data Engineering Kolkata
🏢 LTM
📍 Kolkata
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.