Job Summary:

- Client is seeking a Senior Data Engineer / AI Engineer with deep expertise in data engineering, AI-driven development, and agentic workflow automation using Claude and Claude Code.
- This role will focus on transforming ingestion and intake processes from multi-week timelines to less than 24 hours through the design and implementation of sophisticated Data Engineering and AI-powered workflows.
- The ideal candidate is not simply an AI engineer, but a senior-level data engineering professional who understands how to build, orchestrate, and deploy enterprise-scale data pipelines while leveraging Claude-based agentic systems to automate complex business processes.
- Experience building harnesses, skills, specifications, and agent orchestration frameworks is highly desired.

Responsibilities:

- Build, deploy, monitor, and optimize enterprise data pipelines.
- Design, develop, and deploy advanced agentic workflows using Claude and Claude Code.
- Build and optimize solutions that reduce ingestion and intake timelines from 5-10 weeks to under 24 hours.
- Develop AI-powered automation frameworks for large-scale, complex workflow orchestration.
- Create and maintain agent harnesses, specifications, and reusable AI skills.
- Design scalable ingestion frameworks supporting high-volume data processing.
- Collaborate with data engineering, platform, and business teams to identify automation opportunities.
- Integrate Claude-based workflows into existing enterprise platforms and services.




- Ensure reliability, governance, and scalability of AI-driven systems.

Experience:

- 6+ years of professional experience in Data Engineering, or related fields.
- Databricks and Unity Catalog expertise
- Claude/Claude Code experience
- Azure Cloud/Azure technologies
- Strong experience with Spark
- Proven experience building and orchestrating enterprise-scale data pipelines.
- Hands-on experience with agentic workflow design and execution.

AI & Agentic Development

- Demonstrated experience using Claude and Claude Code in production environments.
- Ability to clearly explain real-world projects built using Claude Code.

Experience developing:

- Agent frameworks
- Harnesses
- Skills
- Specifications (Spec Development)
- Workflow orchestration systems
- Experience with complex multi-step agent execution patterns.

Data Engineering

Solid expertise in:

- Databricks
- Unity Catalog
- Apache Spark
- PySpark
- Data pipeline development
- Data ingestion and transformation frameworks
- Strong understanding of data architecture, governance, and distributed processing.

Nice to Have:

- Good English communication skills.
- Experience designing AI systems beyond traditional RAG architectures.
- Knowledge of AI workflow orchestration platforms and autonomous agents.
- Background in enterprise-scale automation and operational efficiency initiatives.
- Experience working in Agile product and engineering environments.

📌 Senior Data Engineer / AI Engineer (Bengaluru)
🏢 System Soft Technologies
📍 Bengaluru