Senior Data Engineer

Key Responsibilities
Design, build, and maintain scalable data pipelines using Python & PySpark
Develop and optimize ETL/ELT workflows for structured and semi-structured data
Work with Azure data services to ingest, process, and store large datasets
Write effective SQL queries for data analysis, transformation, and reporting
Collaborate with data analysts, data scientists, and business teams to deliver reliable datasets
Implement data quality checks, monitoring, and performance tuning
Handle batch and (if applicable) streaming data workloads
Follow best practices for data security, governance, and documentation

Mandatory Skills
Python PySpark
Azure

Required Skills

Programming & Data Processing
Robust hands-on experience with Python
Solid expertise in PySpark / Apache Spark
Experience with data transformations, joins, aggregations, window functions

Databases & SQL
Advanced SQL (CTEs, indexing, query optimization)
Experience with relational and analytical databases

Cloud & Azure
Hands-on experience with Microsoft Azure, including:




Azure Data Factory (ADF)
Azure Data Lake Storage (ADLS)
Azure Synapse Analytics
Azure Databricks
Understanding of cloud-based data architecture

Data Engineering Concepts
ETL vs ELT
Data modeling (Star / Snowflake schemas)
Partitioning, caching, performance tuning
Handling large-scale datasets

Positive to Have
Experience with streaming tools (Kafka, Event Hubs)
Knowledge of CI/CD pipelines for data workflows
Exposure to Airflow / Azure Data Factory scheduling
Understanding of DevOps / Git / version control
Basic knowledge of data governance & compliance

Education
Bachelor s degree in Computer Science, Engineering, or related field(Equivalent practical experience is welcome)

Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Senior Data Engineer Gujarat (India)
🏢 Rysun Labs
📍 India