24 Aug
|
Willware Technologies
|
Bengaluru
24 Aug
Willware Technologies
Bengaluru
Company Name:WillwareTechnologies
Role: Azure Databricks & Snowflake Data Engineer
Experience:8+ Years
Location: Mumbai, Chennai, Bangalore, Hyderabad, Pune
WorkMode: Onsite
Job Summary
We are looking for an experienced Azure Databricks & Snowflake Data Engineer to design, develop, and optimize scalable cloud-based data solutions. The ideal candidate should have strong hands-on experience in Azure, Databricks, Snowflake, PySpark, SQL, ETL/ELT, and data warehousing .
The role involves building modern data platforms, developing high-performance data pipelines, implementing data engineering solutions, and supporting enterprise analytics and data initiatives.
Key Responsibilities
- Design and develop scalable data pipelines and ETL/ELT solutions using Azure Databricks and Snowflake.
- Develop and optimize data processing using PySpark and Spark SQL .
- Build data solutions using Delta Lake and implement reliable batch and streaming pipelines.
- Develop and manage Databricks Workflows for automated data processing.
- Implement data governance and security using Unity Catalog .
- Design and develop Snowflake data warehouse solutions and optimize query performance.
- Work with Snowpipe, Streams & Tasks, Dynamic Tables, Time Travel, and Secure Data Sharing .
- Develop and maintain data pipelines using Azure Data Factory (ADF) .
- Work with Azure Data Lake Storage Gen2 (ADLS Gen2) for large-scale data storage and processing.
- Develop data solutions using Azure Synapse Analytics and Azure Functions .
- Implement CI/CD and deployment processes using Azure DevOps .
- Perform data modeling and develop solutions using relational databases.
- Optimize data pipelines, Spark jobs, SQL queries, and Snowflake workloads for performance and cost.
- Collaborate with data architects, analysts, developers, and business teams to deliver enterprise data solutions.
- Ensure data quality, reliability, security, and scalability across data platforms.
Mandatory Skills
- 8-10 years of experience in Data Engineering / Cloud Data Engineering.
- Strong hands-on experience with Azure Databricks .
- Strong expertise in PySpark and Spark SQL .
- Hands-on experience with Delta Lake .
- Experience with Databricks Workflows and Unity Catalog .
- Strong hands-on experience with Snowflake .
- Good knowledge of Snowflake Architecture .
- Experience with Snowpipe, Streams & Tasks, Dynamic Tables, Time Travel, and Secure Data Sharing .
- Robust SQL and Python programming skills.
- Experience in ETL/ELT development and data warehousing .
- Hands-on experience with Azure Data Factory and ADLS Gen2 .
- Experience with data modeling and relational databases .
- Strong understanding of cloud-based data engineering and big data processing.
Good to Have
- Experience with Azure Synapse Analytics .
- Experience with Azure Functions .
- Experience with Azure DevOps and CI/CD .
- Knowledge of Shell Scripting .
- Experience working with SQL Server, Oracle, or PostgreSQL .
- Experience with real-time/streaming data processing.
- Knowledge of data governance, security, and enterprise data platforms.
📌 Azure Databricks & Snowflake Data Engineer (Bengaluru)
🏢 Willware Technologies
📍 Bengaluru