Senior Technical Specialist (Bengaluru)

Senior Technical Specialist (Bengaluru)

21 Nov
|
HCLTech
|
Bengaluru

21 Nov

HCLTech

Bengaluru

Job Summary

The Databricks Developer will design, develop, test, and optimize scalable data-processing solutions using Apache Spark, PySpark, Delta Lake, and the Databricks platform. The role requires practical experience in reliable batch and change-data-capture pipelines, cloud integration, workload troubleshooting, and storage and processing optimization.

Key Responsibilities

- Design, build, and maintain scalable Spark and PySpark data pipelines.
- Develop Delta Lake solutions with effective schema handling, data quality controls, and read/write patterns.
- Implement batch and CDC pipelines for incremental processing.
- Optimize Spark jobs, storage layouts, and pipeline execution for performance, reliability, and cost.
- Develop Databricks notebooks, workflows, jobs, and reusable components.
- Integrate Databricks workloads with AWS services such as S3, EC2, ECS, and DynamoDB.
- Troubleshoot pipeline failures, data-quality issues, performance bottlenecks, and production incidents.
- Apply modular development, version control, testing, code review, deployment automation, and documentation practices.
- Use REST APIs and platform automation where required.

Skill Requirements





- Apache Spark knowledge from core concepts through distributed processing patterns.
- Production-grade Python and PySpark development.
- Practical Delta Lake and Delta table experience.
- Storage, file-layout, Spark execution, and pipeline optimization.
- Databricks notebooks, jobs, workflows, compute, and development practices.
- Strong in Python data structures and algorithms.
- AWS data-engineering services, particularly S3, with familiarity with EC2, ECS, and DynamoDB.
- CDC and incremental data-processing solutions.
- REST APIs for integration and automation.
- Ability to develop solutions using Databricks Apps when required.

Must Have Skill

Databricks,PySpark,Python

Good to Have Skill

RESTAPI

Other Requirements

Positive-to-Have Qualifications

- Spark Structured Streaming fundamentals.
- Exposure to Mosaic AI capabilities.
- Relevant Databricks certification or preparation toward certification.

Expected Developer Profile

The preferred candidate is a hands-on developer who can translate requirements into maintainable Databricks solutions. The strongest emphasis is on Spark, PySpark, Delta Lake, pipeline and storage optimization, AWS integration, and CDC.

📌 Senior Technical Specialist (Bengaluru)
🏢 HCLTech
📍 Bengaluru

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: senior technical specialist (bengaluru) / bengaluru

Subscribe to this job alert:

Get the latest job offers by email for: senior technical specialist (bengaluru) / bengaluru