Job Description: Data Engineer (Databricks / PySpark / SQL)

Experience

26 years

Role Overview

We are looking for a skilled Data Engineer with strong expertise in Databricks, PySpark, and SQL to design, develop, and optimize scalable data pipelines. The ideal candidate will work closely with data analysts, data scientists, and business stakeholders to deliver high-quality data solutions and insights.

Key Responsibilities
- Design, develop, and maintain scalable data pipelines using PySpark on Databricks
- Build and optimize large-scale data processing workflows for structured and unstructured data
- Develop and maintain SQL queries for data transformation, extraction, and reporting
- Work with Delta Lake and Databricks ecosystem for effective data storage and processing
- Collaborate with cross-functional teams to gather and understand data requirements
- Ensure data quality, integrity, and performance optimization of pipelines
- Troubleshoot and debug data-related issues in production environments
- Implement best practices for data engineering, including version control, CI/CD,



and testing
- Optimize queries and Spark jobs for performance and cost efficiency
- Document technical designs, processes, and workflows

Required Skills & Qualifications
- Strong experience with Databricks platform (workflows, notebooks, clusters)
- Proficiency in PySpark for large-scale data processing
- Advanced knowledge of SQL (joins, window functions, query optimization)
- Experience working with big data technologies and distributed computing
- Hands-on experience with ETL/ELT pipeline development
- Understanding of data warehousing concepts and data modeling (Star/Snowflake schema)
- Experience with Delta Lake / Lakehouse architecture
- Familiarity with cloud platforms (Azure, AWS, or GCP preferred)
- Knowledge of data integration tools and scheduling (Airflow, ADF, etc.) is a plus

Preferred Skills
- Experience with Python programming beyond PySpark
- Exposure to streaming fram

📌 Databricks Engineer (Pune)
🏢 Tecnoprism
📍 Pune