Description : Data Engineer / Data Scientist

Experience: 5–8 years
Position Type: Immediate Requirement
Location: Remote

Role Summary

We are looking for an experienced Data Engineer / Data Scientist with strong hands-on expertise in Python, PySpark, Azure Databricks, and MLOps. The candidate should be capable of working independently as an individual contributor while collaborating effectively with cross-functional teams.

The ideal candidate will have experience developing scalable batch and streaming data pipelines, working on machine learning projects, debugging Spark applications, and supporting model deployment.

Key Responsibilities

- Develop scalable data processing solutions using Python, PySpark, and Azure Databricks.
- Build and maintain batch and streaming data pipelines.
- Develop and optimize Spark Data Frame-based transformations.
- Debug and troubleshoot Spark jobs in Azure Databricks.
- Implement Delta Lake solutions for data reliability, versioning, and effective querying.
- Develop APIs using Python or Scala for data and machine learning applications.
- Support machine learning projects, MLOps workflows, and model deployment activities.
- Work with Azure services for data ingestion, storage, security, integration, and processing.
- Configure and manage Databricks job clusters and notebook workflows.
- Validate processed data by building and executing Data Frame-based checks.
- Collaborate with technical and business teams while independently owning assigned deliverables.

Must-Have Skills

- Strong hands-on expertise in Python and PySpark.
- Good knowledge of Microsoft Azure and Azure Databricks.
- Hands-on experience with MLOps practices and tools.
- Basic understanding of machine learning model deployment.
- Practical experience working on machine learning projects.




- Experience developing and debugging Spark-based applications.
- Ability to work independently and interact effectively with project teams.

Additional Relevant Skills

- Hands-on experience in Databricks notebook development.
- Strong experience with Spark Data Frames using PySpark or Scala.
- Experience debugging and optimizing Spark jobs in Azure Databricks.
- Knowledge of API development using Python or Scala.
- Working knowledge of Azure services, including:
- Azure Event Hubs
- Azure Storage Accounts
- Azure Key Vault
- Azure Service Bus
- Azure Functions
- Azure Data Lake Storage

- Understanding of Databricks job clusters and compute configurations.
- Hands-on experience implementing Azure cloud-based data solutions.
- Knowledge of real-time streaming technologies such as Kafka.
- Experience developing batch and streaming pipelines using Event Hubs, Kafka, or IoT data sources.
- Experience implementing Delta Lake solutions for data reliability, versioning, and query performance.
- Working knowledge of Git Hub or similar version-control platforms.

Preferred Technical Exposure

- MLflow or similar tools for experiment tracking and model lifecycle management.
- CI/CD implementation for data and machine learning workloads.
- Performance tuning of Spark applications and Databricks workloads.
- Data quality validation, monitoring, and production support.
- Secure integration of Azure services using managed identities and secrets.

Qualifications

- Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, Information Technology, or a related discipline.
- 5–8 years of relevant experience in data engineering, data science, machine learning, or cloud analytics.

Location:

IND Bangalore - MTP C4, 3rd Flr

Language Requirements:

Time Type:

Full time

📌 Data Engineer / Data Scientist (India)
🏢 Concentrix
📍 India