15 Oct
|
Vinovaai Private
|
Gurugram
15 Oct
Vinovaai Private
Gurugram
*Job Title:** Senior Data Engineer – PySpark, SQL
**Location:** Gurugram / Chennai
**Employment Type:** Full-time
**Experience:** 10+ Years
**Notice Period:** Immediate Joiner or maximum 7 days
**Mandatory Certification:** Active Azure Databricks Certification
### Job Overview
We are looking for an experienced Senior Data Engineer with strong hands-on expertise in PySpark, SQL, Python, and Azure Databricks. The candidate will be responsible for designing, developing, and optimizing scalable data pipelines and ETL/ELT processes to support analytics, reporting, and machine learning applications.
### Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines.
- Build and optimize large-scale data processing solutions using PySpark and Apache Spark.
- Write complex SQL queries, perform query tuning, and optimize database performance.
- Develop and maintain data lakes, data warehouses, and efficient data models.
- Work with Azure Databricks, Azure Data Lake, and Azure Synapse.
- Automate data workflows using Azure Data Factory, Apache Airflow, or similar tools.
- Implement data quality checks, validation, monitoring, and governance practices.
- Troubleshoot production pipeline issues and ensure reliable data delivery.
- Collaborate with data scientists, analysts, and business stakeholders.
- Conduct code reviews, follow CI/CD and Git practices, and mentor junior engineers.
### Mandatory Skills
- 10+ years of overall experience in data engineering.
- Solid hands-on experience with **PySpark and Advanced SQL**.
- **Active Azure Databricks certification is mandatory.**
- Strong Python programming and scripting skills.
- Experience designing and implementing enterprise-level ETL/ELT pipelines.
- Knowledge of distributed systems, big data architecture, data lakes, and data warehousing.
- Experience with SQL performance tuning, complex joins, and window functions.
- Immediate joiner or maximum 7 days notice period.
### Good-to-Have Skills
- Azure Data Factory, Azure Synapse, Azure Key Vault.
- Apache Airflow or other workflow orchestration tools.
- Kafka, Spark Structured Streaming, or Azure Event Hubs.
- Data governance, data cataloging, and data security.
- Exposure to Snowflake, Amazon Redshift, AWS, or Google BigQuery.
### Education
Bachelor's or Master's degree in Computer Science, Engineering, or a related field.ring, or a related field.
📌 Senior Data Engineer (PySpark, SQL) (Gurugram)
🏢 Vinovaai Private
📍 Gurugram