Apply Job Description: Data Engineer / Data ScientistExperience: 5–8 yearsPosition Type: Immediate RequirementLocation: RemoteRole SummaryWe are looking for an experienced Data Engineer / Data Scientist with strong hands-on expertise in Python, PySpark, Azure Databricks, and MLOps. The candidate should be capable of working independently as an individual contributor while collaborating effectively with cross-functional teams.The ideal candidate will have experience developing scalable batch and streaming data pipelines, working on machine learning projects, debugging Spark applications, and supporting model deployment.Key ResponsibilitiesDevelop scalable data processing solutions using Python, PySpark, and Azure Databricks.Build and maintain batch and streaming data pipelines.Develop and optimize Spark DataFrame-based transformations.Debug and troubleshoot Spark jobs in Azure Databricks.Implement Delta Lake solutions for data reliability, versioning, and productive querying.Develop APIs using Python or Scala for data and machine learning applications.Support machine learning projects, MLOps workflows, and model deployment activities.Work with Azure services for data ingestion, storage, security, integration, and processing.Configure and manage Databricks job clusters and notebook workflows.Validate processed data by building and executing DataFrame-based checks.Collaborate with technical and business teams while independently owning assigned deliverables.Must-Have SkillsStrong hands-on expertise in Python and PySpark.Good knowledge of Microsoft Azure and Azure Databricks.Hands-on experience with MLOps practices and tools.Basic understanding of machine learning model deployment.Practical experience working on machine learning projects.Experience developing and debugging Spark-based applications.Ability to work independently and interact effectively with project teams.Additional Relevant SkillsHands-on experience in Databricks notebook development.Strong experience with Spark DataFrames using PySpark or Scala.Experience debugging and optimizing Spark jobs in Azure Databricks.Knowledge of API development using Python or Scala.Working knowledge of Azure services, including:Azure Event HubsAzure Storage AccountsAzure Key VaultAzure Service BusAzure FunctionsAzure Data Lake StorageUnderstanding of Databricks job clusters and compute configurations.Hands-on experience implementing Azure cloud-based data solutions.Knowledge of real-time streaming technologies such as Kafka.Experience developing batch and streaming pipelines using Event Hubs, Kafka, or IoT data sources.Experience implementing Delta Lake solutions for data reliability, versioning, and query performance.Working knowledge of GitHub or similar version-control platforms.Preferred Technical ExposureMLflow or similar tools for experiment tracking and model lifecycle management.CI/CD implementation for data and machine learning workloads.Performance tuning of Spark applications and Databricks workloads.Data quality validation, monitoring, and production support.Secure integration of Azure services using managed identities and secrets.QualificationsBachelor’s or Master’s degree in Computer Science, Data Science, Engineering, Information Technology, or a related discipline.5–8 years of relevant experience in data engineering, data science, machine learning, or cloud analytics. Apply

📌 Aep - Data Architect (Bengaluru)
🏢 3M
📍 Bengaluru