Data Engineer (Bengaluru)

Data Engineer (Bengaluru)

19 Nov
|
Concentrix
|
Bengaluru

19 Nov

Concentrix

Bengaluru

Job Overview

We are seeking an experienced and results-driven Data Engineer with 4–6 years of relevant experience in designing, developing, and optimizing scalable data solutions using Azure Databricks and related technologies.

The ideal candidate should have strong hands-on expertise in Azure Databricks, PySpark, Apache Spark, SQL, Python, ETL/ELT development, Delta Lake, Azure cloud services, CI/CD, and performance optimization. The role requires practical experience in building reliable, secure, and production-grade data pipelines.

Key Responsibilities

- Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Databricks, PySpark, Python, and SQL.
- Build and optimize batch and large-scale data-processing solutions using Apache Spark.
- Develop data ingestion and transformation workflows for structured and semi-structured data.
- Implement reliable data-processing frameworks using Delta Lake capabilities.
- Perform data cleansing, validation, reconciliation, and quality checks to ensure accuracy and consistency.
- Optimize Spark jobs, Databricks notebooks, SQL queries, clusters, and data pipelines for performance and cost efficiency.
- Troubleshoot pipeline failures, data-quality issues, production incidents, and performance bottlenecks.
- Implement incremental data processing, partitioning, caching, join optimization, and file-management techniques.
- Develop reusable and configurable data-engineering components.




- Implement version control and automated deployment processes using CI/CD and DevOps practices.
- Collaborate with data architects, analysts, application teams, and business stakeholders to translate requirements into technical solutions.
- Maintain technical documentation covering pipeline architecture, transformation logic, dependencies, and operational procedures.
- Ensure solutions comply with organizational standards for security, governance, monitoring, and data protection.

Required Technical Skills

- 4–6 years of relevant experience in data engineering and cloud-based data platforms.
- Strong hands-on experience with Azure Databricks, PySpark, Apache Spark, Python, and advanced SQL.
- Proven expertise in developing scalable ETL/ELT pipelines for batch and large-scale data processing.
- Solid understanding of Delta Lake, including Delta tables, ACID transactions, schema evolution, merge/upsert operations, and table optimization.
- Experience with Azure data services such as Azure Data Lake Storage and Azure Data Factory.
- Solid knowledge of Spark and Databricks performance optimization, including partitioning, caching, join strategies, shuffle optimization, file sizing, and cluster configuration.
- Experience with Git-based version control, CI/CD pipelines, and DevOps practices for automated deployment across environments.
- Good understanding of data quality, security, monitoring, troubleshooting, and production support.

📌 Data Engineer (Bengaluru)
🏢 Concentrix
📍 Bengaluru

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: data engineer (bengaluru) / bengaluru

Subscribe to this job alert:

Get the latest job offers by email for: data engineer (bengaluru) / bengaluru