10 Sep
|
Zorba consulting
|
Mumbai
10 Sep
Zorba consulting
Mumbai
- We are looking for a highly skilled Azure Databricks & Snowflake Data Engineer with strong experience in Spark, PySpark, Python/Scala, Azure Cloud, SQL, ETL/ELT pipelines, and Snowflake development.
- The ideal candidate should have hands-on experience in building and migrating enterprise data pipelines and should be comfortable working across the complete Software Development Life Cycle.
- Exposure to Snowflake Cortex/CoCo, Databricks Genie, and AI-assisted coding tools such as GitHub Copilot is highly desirable.
- Candidates with strong core engineering skills and a willingness to learn emerging AI capabilities will also be considered.
Key Responsibilities
- Design, develop, optimize, and maintain scalable data engineering solutions using Azure Databricks, Apache Spark, PySpark, Python, and/or Scala.
- Develop and optimize complex ETL/ELT data pipelines for large-scale data processing.
- Perform migration of existing ETL/ELT pipelines and data workloads to Azure Databricks and Snowflake.
- Develop robust data transformation, ingestion, validation, and processing frameworks.
- Work extensively with Snowflake for data loading, transformation, optimization, and integration.
- Collaborate with data architects, developers, analysts, and business teams to understand requirements and deliver scalable solutions.
- Implement CI/CD pipelines and automated deployment processes using Git, GitHub, GitHub Workflows, and Azure DevOps.
- Follow software engineering best practices including version control, code reviews, unit testing, debugging, and documentation.
- Work with workflow/orchestration technologies such as Azure Data Factory (ADF), Azure Synapse, and Airflow.
- Participate in architecture discussions and contribute to improving data platform performance, reliability, and scalability.
- Utilize Terraform and Infrastructure-as-Code concepts where applicable.
- Explore and leverage AI-assisted development tools such as GitHub Copilot and Databricks Genie to improve engineering productivity.
- Gain exposure to and contribute to solutions using Snowflake Cortex/CoCo and other AI capabilities.
- Troubleshoot production issues and provide root-cause analysis and permanent fixes.
- Participate in all phases of the Software Development Life Cycle (SDLC).
Mandatory Technical Skills
- 5+ years of hands-on experience with Azure Databricks.
- 5+ years of experience with Apache Spark / PySpark.
- 5+ years of programming experience using Python and/or Scala.
- 5+ years of experience working on the Azure Cloud Platform.
- 3+ years of hands-on Snowflake development experience.
- Strong knowledge of SQL and database concepts with 5+ years of experience.
- Strong experience in ETL/ELT pipeline development and migration.
- Hands-on experience with CI/CD, Git, and GitHub Workflows.
- Strong software development and coding skills.
- 5+ years of complete SDLC experience.
Good-to-Have Skills
- Knowledge or hands-on experience with Snowflake Cortex / CoCo.
- Exposure to Databricks Genie.
- Experience using GitHub Copilot or other AI-assisted coding tools.
- Knowledge of Azure DevOps.
- Knowledge of Terraform / Infrastructure as Code.
- Experience with Azure Data Factory (ADF).
- Exposure to Azure Synapse Analytics.
- Exposure to Apache Airflow.
- Experience with Delta Lake and modern data lakehouse architecture.
- Understanding of data engineering best practices, performance tuning, and cloud-native architectures.
- Core Competencies Strong programming and problem-solving skills.
- Excellent understanding of data structures, SQL, databases, and data processing concepts.
- Ability to develop clean, maintainable, reusable, and production-ready code.
- Strong analytical and debugging skills.
- Ability to work independently as well as collaboratively in a distributed team.
- Valuable communication and stakeholder-management skills.
- Strong willingness to learn emerging technologies, particularly Generative AI and AI-assisted development.
Location - Pan India,Mumbai, Delhi / NCR, Bengaluru , Kolkata, Chennai, Hyderabad, Ahmedabad, Pune.
📌 Azure Databricks & Snowflake Data Engineer (Mumbai)
🏢 Zorba consulting
📍 Mumbai