Job Description

Job Title: - Azure Databricks with Pyspark
Location: - AIA Kolkata

Experience: - 6 to 9 Years

An Azure Databricks Developer job description typically requires expertise in PySpark, Azure Data Factory (ADF), and Delta Lake architecture

Key Responsibilities
Pipeline Development: Build and optimize scalable data pipelines and ETL processes using Azure Databricks and PySpark.
Data Integration: Connect and manage data from Azure Data Lake Storage (ADLS Gen2) and various file formats (Parquet, Delta, JSON, CSV).
Performance Tuning: Tune Spark SQL queries, optimize cluster configurations, and manage data partitioning strategies.
CI/CD & Deployment: Implement version control via Git and deploy automated workflows using Azure Dev Ops or Git Hub Actions.

Required Qualifications
Experience: 4 to 8+ years of total IT experience, with at least 2–3 years of hands-on production experience in Azure Databricks.
Tech Stack: Advanced proficiency in Python, PySpark, and complex SQL.
Cloud Knowledge: Solid familiarity with core Azure services including ADF, ADLS Gen2, and Azure SQL.

📌 Azure Databricks With Pyspark Chennai (India)
🏢 Cognizant Consulting
📍 India