16 Aug
|
ElfOnze Technologies
|
Chennai
16 Aug
ElfOnze Technologies
Chennai
Roles & Responsibilities:
- Develop and maintain data pipelines and workflows using PySpark on Databricks
- Write Python scripts for data processing, transformation, and integration with Spark
- Utilize PySpark and related libraries (e.g., pyspark-ai) to incorporate AI and machine learning functionalities into Spark workflows
- Perform data analysis, transformations, and aggregations on large datasets
- Deploy and optimize machine learning models within the Databricks workplace
- Collaborate with data engineers, data scientists, and business teams to understand requirements and deliver solutions
- Use the Databricks platform to manage, monitor, and optimize Spark jobs and workflows
- Work with the English SDK for Apache Spark to facilitate natural language interactions with Spark DataFrames
- Ensure code quality, performance, and scalability of data pipelines and Spark jobs.
📌 Databricks (Chennai)
🏢 ElfOnze Technologies
📍 Chennai