Role & responsibilities

Hiring: Senior Data Engineer Databricks / AWS / PySpark

Location: Pan India

Experience: 6+ Years

Relevant Experience: 4+ Years

We are looking for an experienced Data Engineer with strong expertise in Databricks, PySpark, SQL, AWS, and ETL/ELT pipeline migration to join our team.

Key Responsibilities

- Analyze existing ETL pipelines, data sources, and transformation logic across platforms
- Drive end-to-end migration of data pipelines to Databricks
- Design, develop, and maintain scalable batch and near-real-time data pipelines
- Re-engineer legacy ETL/ELT workflows using Spark, PySpark, SQL, and Databricks
- Integrate data from multiple heterogeneous sources
- Implement robust data transformation and processing logic
- Perform data validation, reconciliation, and consistency checks after migration
- Optimize pipelines for performance, scalability, and cost efficiency
- Collaborate with business stakeholders, analysts, and data teams to understand requirements
- Support reporting, analytics, and downstream applications with reliable datasets
- Monitor production pipelines, troubleshoot failures,



and resolve data issues
- Implement logging, alerting, and observability using CloudWatch / Databricks monitoring
- Follow data governance, security, and compliance standards

• Implement CI/CD for data workflows and maintain technical documentation

Mandatory Skill Strong proficiency in SQL and relational databases such as Oracle, PostgreSQL, etc.

Required / Desired Skills

Robust SQL and database expertise

ETL/ELT development and migration experience

Strong programming skills in Python and/or Scala

Hands-on experience with Apache Spark / PySpark

Strong experience with Databricks – Delta Lake, Notebooks, Jobs, Workflows

AWS data engineering services

- AWS Glue
- AWS Lambda
- Amazon S3
- EventBridge
- SQS
- Amazon Redshift
- Kinesis Firehose
- CloudWatch

Knowledge of data modeling, data warehousing, and data lake architectures

Experience handling large-scale datasets and distributed processing

Knowledge of pipeline orchestration and workflow management

📌 Senior Data Engineer (Bengaluru)
🏢 Tekskills
📍 Bengaluru