Responsibilities

- Design, develop, and maintain scalable data pipelines and architectures using Databricks.
- Collaborate with data scientists and analysts to understand data requirements and deliver high-quality data solutions.
- Implement data ingestion processes from various sources, ensuring data quality and integrity.
- Optimize data storage and retrieval processes for performance and cost efficiency.
- Monitor and troubleshoot data pipeline performance and reliability issues.
- Document data engineering processes, architectures, and best practices.
- Stay updated with the latest trends and technologies in data engineering and analytics.

Mandatory Skills

- Solid expertise in Databricks and its ecosystem.
- Proficient in programming languages such as Python, Scala, or Java.
- Experience with data modeling,



ETL processes, and data warehousing concepts.
- Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their data services.
- Solid understanding of SQL and experience with relational databases.
- Knowledge of big data technologies (e.g., Spark, Hadoop) is a plus.

Preferred Skills

- Experience with data visualization tools (e.g., Tableau, Power BI).
- Familiarity with machine learning concepts and frameworks.
- Knowledge of data governance and data security best practices.
- Experience with CI/CD pipelines and version control systems (e.g., Git).

Databricks

📌 LEAD ENGINEER - Databricks (Pune)
🏢 Happiest Minds Technologies
📍 Pune