Job Summary

Role: Lead Python Engineer Location: Remote, India

Experience: 7+ Years

Role Overview

We are seeking a highly skilled Data Engineer with a strong background in Databricks and a passion for data engineering. The ideal candidate will have extensive experience in designing and implementing data solutions using various technologies, including PySpark, Spark SQL and Power BI. This role requires a proactive individual who can work independently and collaboratively in a fast-paced environment.

Key Responsibilities

- Design, develop and optimize scalable data pipelines using Python, PySpark, Spark SQL and Databricks.
- Build and manage data solutions using Azure Databricks, Delta Lake, Unity Catalog and Databricks Workflows.
- Develop complex SQL queries, views, data models and semantic layers for data warehousing and visualization.
- Integrate and manage data using Azure Data Lake, Azure Data Factory and other Azure services.
- Develop and maintain Power BI solutions, including dashboards, DAX measures and Power Query transformations.
- Implement data governance, security, compliance and data management best practices.
- Participate in code reviews, CI/CD processes, Agile ceremonies and end-to-end SDLC activities.
- Collaborate with cross-functional teams to understand business requirements and translate them into effective technical solutions.

Required Skills and Qualifications

- 7+ years of hands-on experience in software development, with a strong emphasis on Python programming.
- Deep expertise in PySpark (Spark SQL, DataFrames and Datasets) for building and optimizing scalable data pipelines. (PySpark)
- Solid experience with Azure Databricks (Unity Catalog), including working with Delta tables, workflows and scripting in Python.
- Understanding of data governance, data management, security and compliance in cloud environments.




- Familiar with Azure cloud services, especially Azure Data Lake, Azure Data Factory,
- Experience with Power BI, including DAX and Power Query.
- Modeling data for Data Warehousing and Visualization projects through Semantic Layers.
- Skilled in writing complex SQL queries and building views involving joins across multiple tables.
- Proficient in tools like DevOps (project management) and version control and CI/CD pipelines.

Must Have Skills

- Python
- PySpark and Spark SQL
- Azure Databricks and Unity Catalog
- Delta Lake / Delta Tables
- Azure Data Lake and Azure Data Factory
- Advanced SQL
- Power BI, DAX and Power Query
- Data Modeling and Semantic Layers
- Azure DevOps, Git and CI/CD

Good to Have Skills

- Experience Azure AI, Azure Copilot Studio and Azure Fabric.
- Experience building workflows with Power Automate.
- EventHub or Kafka for streaming data ingestion.
- Proficient in designing and implementing real-time data streaming solutions.
- Experience across the Software Development Lifecycle (SDLC), with a strong grasp of Agile Scrum methodologies.

Desired Attributes

- Strong ownership and a proactive approach to problem-solving.
- Excellent collaboration and communication skills.
- Comfortable participating in mobbing sessions, code reviews and cross-team discussions.
- Ability to translate business requirements into practical technical solutions.
- Adaptable and willing to learn new technologies and tools.
- Robust attention to quality, scalability, security and performance.
- Ability to prioritize effectively and work independently in a fast-paced environment.

Interview Process

2 rounds of discussion. Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Lead Python Engineer (Noida)
🏢 Algoworks
📍 Noida