Sr. Data Engineer - SQL - ETL-PYTHON-SNOWFLAKE- AZURE

5 to 9 years - Gurugram

Are you ready to elevate your career by joining a dynamic environment where your expertise in Databricks, Azure, and data engineering can create a substantial impact? We are looking for a Senior Data Engineer to be an integral part of our Technology & Analytics Practice. If you are passionate about crafting scalable data solutions and enjoy solving intricate business challenges, then this role is perfect for you. This hybrid opportunity offers both the excitement of working remotely and collaborating in-person across PAN India.

Responsibilities

- Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.

- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness

- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.

- Diagnose system performance issue related to data processing and implement solution to address them.

- Collaborate with other teams to ensure successful integration of data pipelines into larger system architecture requirement.

- Maintain integrity and quality across all pipelines and environments.

- Understand and follow secure coding practice to make sure code is not vulnerable

In this role,



you will report to the Data Engineering Team Lead.

Skills

- Should have Strong background in Software development with experience in ingest, transform and store data from large datasets using Pyspark in Azure Databricks with strong knowledge on distributed computing concepts.

- Must have hands on experience in designing and developing ETL Pipelines in Pyspark in Azure Databricks with robust python scripting exposure like list comprehensions, Dictionary variables etc.

- Must have minimum 5 years of exposure and good proficiency in data warehousing concepts.

- Proficient in SQl and database Design concepts.

EXL Service

10 Exchange Place, Suite 2200, Jersey City, NJ T: +1.(phone hidden) www.exlservice.com

- Hands on experience with good proficiency in Delta table and delta file

operations like merge, Insert override, Partition overrides etc.

- Hands on Experience in CICD in Azure DevOps/Harness and ADF/Stone branch for orchestration.

- Knowledge of Azure cloud computing platform with Azure Synapse, ADLS.

- Knowledge on GitHub and build management.

- Passion for data and experience working within a data driven organization

- Excellent presentation, communication (oral & written), and relationship building skills, across all levels of management

📌 Sr. Data Engineer - Snowflake - SQL - Pyspark (Noida)
🏢 Crescendo Global Leadership Hiring India
📍 Noida