Key Responsibilities

- Design, develop, and maintain scalable data processing solutions using Databricks on AWS.

- Build and maintain ETL/ELT pipelines for batch and near real-time data ingestion.

- Develop data transformation and processing workflows using Apache Spark, PySpark, Spark SQL, Python, and Scala.

- Implement Delta Lake solutions, including ACID transactions, schema evolution, and performance optimization.

- Design and implement analytical and dimensional data models, including Star and Snowflake schemas.

- Work with AWS cloud storage services to manage data ingestion, storage, and processing.

- Optimize data pipelines for performance, scalability, reliability, and maintainability.

- Follow coding standards, documentation practices, and data engineering best practices.

- Collaborate with solution architects, BI teams, and data science teams to deliver end-to-end data solutions.

- Support data quality, troubleshooting, and pipeline performance improvements.

Required Skills and Qualifications

- 8+ years of relevant experience in data engineering and development.

- Hands-on development experience with AWS Databricks.

- Solid experience developing and maintaining ETL/ELT pipelines.

- Proficiency in Apache Spark, PySpark, Spark SQL, and Python.

- Working knowledge of Scala and Power BI.





- Hands-on experience with Delta Lake, including ACID transactions, schema evolution, and performance tuning.

- Experience building batch and near real-time data ingestion pipelines.

- Knowledge of analytical data modelling and dimensional modelling, including Star and Snowflake schemas.

- Experience with AWS cloud-based storage systems.

- Strong understanding of data engineering best practices, coding standards, and technical documentation.

- Ability to collaborate effectively with cross-functional technical teams.

Secondary Skills and Preferred Experience

- Exposure to AI/ML workloads on Databricks and feature engineering.

- Familiarity with MLflow or similar model tracking tools.

- Experience with Databricks CI/CD practices and Git-based version control.

- Understanding of data governance, security, and compliance.

- Familiarity with Agile/Scrum delivery methodologies.

- Experience in the insurance domain.

Pay: ₹1,400,000.00 - ₹1,800,000.00 per year

Application Question(s)

- By applying, you agree that your resume and personal details may be stored securely for up to 2 years for recruitment purposes.

Work Location: Hybrid remote in Bangalore City, Bengaluru, Karnataka

📌 AWS Databricks Developer (Bengaluru)
🏢 Ananta Services
📍 Bengaluru