: Data Engineer (Azure Databricks & PySpark)

Position: Data Engineer

Experience: 3 to 12 years

Primary Skills: Azure Databricks, PySpark, SQL

Secondary Skills: ADF (Azure Data Factory)

Location: Bengaluru/Hyderabad

Mode of Work: Hybrid

Notice Period: 0 to 30 Days

Key Responsibilities:

ETL Development & Data Processing:
Design, develop, and maintain ETL pipelines using Azure Databricks, PySpark, and ADF.
Extract, transform, and load data from multiple sources into Azure Data Lake Storage.
Optimize and fine-tune ETL workflows for performance and scalability.

SQL & Query Optimization:
Write complex SQL queries, joins, subqueries, and functions for data processing.
Perform SQL tuning and query optimization for improved efficiency.
Work with large datasets and ensure optimal data handling.

Azure Cloud Integration:
Implement data solutions using Azure technologies (Databricks, ADF, ADLS).
Work with batch and streaming data processing for real-time analytics (preferred).
Ensure data security, compliance,



and reliability in cloud-based systems.

Collaboration & Problem-Solving:
Work closely with cross-functional teams to support data analytics and business intelligence initiatives.
Identify and resolve data-related issues proactively.
Provide insights and recommendations for improving ETL performance.

Required Skills & Qualifications:

3 to 5 years of experience in Azure Databricks, PySpark, and SQL.

Robust SQL skills complex joins, functions, procedures, query tuning.

Hands-on experience in ADF (Azure Data Factory) and Azure Data Lake Storage.

Ability to handle large datasets efficiently.

Experience in performance optimization of ETL workflows.

Valuable understanding of batch and streaming data processing (preferred).

Solid analytical and problem-solving skills.

Ability to work independently and take ownership of tasks.

📌 Azure Databricks Data Engineer 3 To 5 Years Gyansys Hybrid Bengaluru
🏢 PradeepIT Consulting Services
📍 Bengaluru