Data Engineer | Python, PySpark, Databricks

Location: Chennai / Bengaluru

Work Mode: Hybrid

Experience: 4+ Years

Joining: Immediate to 30 Days

Work Location:

- Chennai: Taramani
- Bengaluru: Cessna Business Park

Key Requirements

- 4+ years of experience in Data Engineering or a similar role.
- Strong hands-on experience with Python, PySpark, SQL, and Databricks.
- Experience working with AI coding assistants such as Claude Code, GitHub Copilot, Codex, or Cursor.
- Familiarity with Apache Airflow for workflow orchestration and scheduling.
- Experience with Linux administration and shell scripting.
- Strong understanding of data modelling, ETL processes, and data warehousing concepts.
- Good problem-solving, analytical, communication, and collaboration skills.
- Bachelor's degree in Computer Science, Engineering, or a related field.

Responsibilities

Data Pipeline Development

- Design, develop, and maintain scalable data pipelines using Python, PySpark, and SQL.
- Ingest, process, and transform data from multiple sources.
- Implement data quality checks and monitoring to ensure pipeline reliability and data integrity.

Databricks & Data Engineering
- Use Databricks for data ingestion, ETL processing, transformation, and data analysis.
- Develop and optimize Databricks jobs and clusters for performance and cost efficiency.
- Work with large volumes of structured and unstructured data.

Data Warehouse & Modelling
- Build and optimize data warehouse solutions for efficient data storage and retrieval.
- Design and implement data models for analytical and reporting requirements.




- Apply best practices across ETL and data warehousing processes.

Airflow & Workflow Orchestration
- Develop, configure, and manage Airflow DAGs for data pipeline orchestration and scheduling.
- Implement error handling, monitoring, and alerting to ensure reliable workflow execution.

Linux & Shell Scripting
- Work with Linux-based environments supporting data engineering workloads.
- Troubleshoot infrastructure and server-related issues.
- Use shell scripting for automation and data engineering tasks.

AI Coding Tools
- Leverage AI coding assistants such as Claude Code, GitHub Copilot, Codex, or Cursor to improve development productivity, code quality, and efficiency.
- Demonstrate practical experience using AI-assisted development tools in day-to-day engineering work.

Collaboration & Documentation
- Collaborate with Data Scientists, Data Analysts, and other stakeholders to understand requirements and deliver effective data solutions.
- Document data engineering processes, workflows, technical solutions, and best practices.
- Contribute to continuous improvement of data engineering practices and standards.

Ideal Candidate

We are looking for a hands-on Data Engineer with 4+ years of experience who has strong expertise in Python, PySpark, SQL, and Databricks, along with exposure to Airflow, Linux, shell scripting, data modelling, and ETL/data warehousing. Experience with modern AI coding assistants is an significant part of this requirement.

Candidates who can join immediately or within 30 days will be preferred.

📌 Databricks Engineer (Bengaluru)
🏢 Nice Software Solutions
📍 Bengaluru