04 Oct
|
Duobridge
|
India
Job Summary:
We are looking for a Senior Data Engineer with solid Python and data engineering experience to build scalable data platforms and pipelines using Azure, Databricks, and Snowflake .
Key Responsibilities:
- Design and develop scalable ETL/ELT data pipelines.
- Build data ingestion frameworks using Python, APIs, and distributed processing .
- Develop Databricks pipelines using PySpark and Delta Lake .
- Work with Snowflake, Azure Data Factory, ADLS Gen2, and Azure Synapse .
- Design Medallion Architecture, CDC, data models, and data quality frameworks.
- Develop REST/GraphQL API integrations with authentication, retry, and error handling.
- Optimize pipeline performance, reliability, and cloud costs.
- Support batch, real-time, and event-driven data processing.
- Collaborate with AI engineers, architects,
product teams, and business stakeholders.
Required Skills:
- 5+ years of Data Engineering experience.
- Strong Python and Advanced SQL skills.
- Hands-on experience with Databricks, PySpark, Delta Lake, and Snowflake .
- Strong knowledge of Azure Data Factory, ADLS Gen2, and Azure Synapse .
- Experience with REST/GraphQL APIs, OAuth2/JWT, and API integrations.
- Knowledge of data architecture, CDC, Medallion Architecture, and dimensional modeling.
- Experience with Git and CI/CD .
Good to Have: Airflow, dbt, Kafka, Terraform, Azure OpenAI, and AI/ML data pipelines.
📌 Senior Data Engineer (India)
🏢 Duobridge
📍 India