Role & responsibilities

- Design and implement ETL pipelines using Databricks and Apache Spark.

- Collaborate with data scientists and analysts to understand data requirements and provide optimized solutions.

- Ensure the quality and integrity of data through data validation and testing processes.

- Monitor and troubleshoot performance issues within Databricks workflows and clusters.

- Develop and maintain documentation for data processing workflows and architectures

- Perform data modeling and analysis to enhance data accessibility and usability.

- Build dashboards using Microsoft Power BI

- Stay updated on new features and best practices within the

- Databricks environment to leverage the latest advancements.

Requirements





- Bachelor's degree in Computer Science, Information Technology, or related field.

- Proven experience working with Databricks and Apache Spark for data processing and analysis.

- Solid programming skills in languages such as Python, Scala, or SQL.

- Hands-on experience in data engineering workflows and ETL processes.

- Familiarity with cloud platforms such as AWS, Azure, or Google Cloud.

- Knowledge of data warehousing concepts and architectures.

- Excellent problem-solving skills and ability to work collaboratively in a team environment.

📌 Data Engineer (Chennai)
🏢 Neurealm
📍 Chennai