Forward Deployed Engineer II - Data & AI Engineer (Databricks & AWS)

Location: Bengaluru / Delhi / Mumbai / Hyderabad / Hybrid

Employment Type: Full-Time

Experience: 6+ Years

About the Role

Avashya is looking for a Data & AI Engineer – Databricks & AWS with deep hands-on expertise in building, optimizing and modernizing enterprise data and AI platforms.

This is a hands-on engineering and customer-facing role for someone who understands Databricks deeply, has strong AWS expertise, and can make pragmatic architectural decisions across Data Engineering, ML and Generative AI.

You will work with customer engineering and architecture teams to assess existing platforms, identify performance and cost optimization opportunities, design modern data and AI architectures, and implement solutions that deliver measurable business outcomes.

Key Responsibilities

- Design, build and optimize Databricks-based data and AI platforms across data engineering, analytics, ML and GenAI workloads.
- Deep-dive into Databricks performance and cost, identify bottlenecks and implement measurable optimization improvements.
- Design scalable data pipelines, lakehouse architectures, data models and real-time/batch processing solutions using Python, SQL, PySpark, Spark and Delta Lake.
- Apply deep understanding of Spark internals, query execution, partitioning, shuffles, caching, joins and cluster configuration to optimize workloads.
- Design and implement ML and MLOps solutions using MLflow, model development, deployment and monitoring.
- Build GenAI solutions including LLM applications, RAG, vector search, embeddings and AI agents.
- Architect and implement AWS-based data and AI solutions using services such as S3, Glue, Athena, Redshift, EMR, Kinesis, Lambda, Step Functions, DMS, SageMaker, Bedrock and IAM.
- Evaluate and map Databricks capabilities to AWS-native services, recommending the appropriate platform based on workload, architecture, performance, cost and operational requirements.
- Lead technical discovery sessions,



architecture workshops and solution assessments with customer engineering teams, architects and leadership.
- Build POCs, accelerators and reference implementations to validate technical approaches.
- Develop architecture documents, technical recommendations, migration strategies and optimization reports.
- Support migration and modernization initiatives involving Hadoop, EMR, Teradata, Oracle and legacy data warehouses.
- Contribute to Avashya's assessment frameworks, accelerators, reusable architectures and best-practice playbooks.
- Identify opportunities for follow-on engineering, modernization and managed services engagements.

Required Technical Skills

- 6+ years of experience in Data Engineering, Data Science, ML, AI Engineering or related roles, with 3+ years of hands-on Databricks and AWS experience.
- Deep hands-on expertise in Databricks, including Delta Lake, Spark, Workflows, Unity Catalog and performance/cost optimization.
- Strong understanding of Spark internals, including query execution, Catalyst, Tungsten, partitioning, shuffles, joins, caching and cluster optimization.
- Strong hands-on Python, SQL and PySpark skills, with advanced proficiency in PySpark.
- Strong experience designing data pipelines, lakehouse architectures, ETL/ELT workflows and data models.
- Working knowledge of ML fundamentals, model development, MLflow and MLOps.
- Hands-on experience with LLMs, RAG, vector search, embeddings and AI agents.
- Strong AWS data and AI experience across S3, Glue, Athena, Redshift, EMR, Kinesis, Lambda, Step Functions, DMS, SageMaker, Bedrock and IAM.
- Ability to make architectural trade-offs between Databricks and AWS-native services based on workload and business requirements.




- Strong understanding of data platform security, governance, networking, scalability, reliability and cost optimization.
- Strong customer-facing communication skills with the ability to conduct discovery sessions and present technical recommendations to architects, engineering leaders and executives.
- Strong technical documentation and architecture communication skills.

Preferred

- Databricks certifications: Data Engineer Professional, Machine Learning Professional, Generative AI Engineer Associate.
- AWS certifications: Solutions Architect Associate/Professional, Data Engineer Associate, Machine Learning Specialty or ML Engineer Associate.

What Success Looks Like

- Customers achieve measurable Databricks performance improvements and cost savings.
- Customers trust your architecture recommendations and understand the trade-offs between Databricks and AWS-native services.
- Technical assessments translate into successful implementations and follow-on engagements.
- You help build Avashya's Data & AI assessment frameworks, accelerators and engineering playbooks.
- You become a trusted technical advisor to customer data, AI and cloud engineering teams.

Why Join Avashya?

- Work alongside former AWS and Microsoft leaders.
- Work on cutting-edge AI, Data, Databricks and AWS transformation projects.
- Solve complex enterprise data and AI engineering problems across industries.
- Build reusable accelerators, platforms and IP rather than only delivering project work.
- High-ownership environment with direct impact on Avashya's technology and business growth.
- Continuous exposure to emerging AWS, Databricks and AI technologies.

Diversity & Inclusion

Avashya is an equal prospect employer committed to creating an inclusive workplace. We celebrate diversity and encourage applications from candidates of all backgrounds.

- If you are passionate about Data, AI, Databricks and AWS and enjoy solving complex engineering problems with customers, we'd love to hear from you.

📌 Forward Deployed Engineer II - Data and AI (India)
🏢 Avashya
📍 India