Role Overview :

Experience : 4-8 yrs.

Age Limit : Maximum 35 Years.

Location : On-Site / Hybrid.

Employment Type : Full-Time.

We are seeking a high-performing Lead Engineer (API & Data Engineering) to drive our integration platforms and Lakehouse ecosystem. This role sits at the intersection of modern cloud-native API development and large-scale data engineering.

You will be responsible for designing and deploying high-performance REST APIs (FastAPI) alongside robust, scalable ETL/ELT data pipelines utilizing Databricks, PySpark, Azure Data Factory (ADF), and AKS/ACA. If you excel at bridging technical architecture with stakeholder management across cross-functional business lines, this role offers direct impact and visibility.

Key Competencies :

- Hands-on Problem Solver : Proven ability to troubleshoot, debug, and optimize complex data pipelines and cloud infrastructure.
- Production Governance : Deep understanding of secure data exchange, authentication protocols, and compliance across multi-tenant systems.
- Agile Leadership : Ability to manage day-to-day delivery autonomously while consulting leadership on high-impact architectural decisions.

Key Responsibilities :

API & Microservices Engineering :

- Design, build, and maintain scalable, low-latency REST APIs using FastAPI and Python.
- Implement microservices architecture, secure API endpoints, and optimize runtime performance under high-concurrency workloads.

Data Engineering & Lakehouse Management :

- Build and optimize enterprise data pipelines using Databricks (PySpark, SQL) and Azure Data Factory (ADF).
- Perform advanced data modeling, schema design, and query optimization for high-volume analytics.

Cloud-Native Deployments & Containerization :

- Package microservices and pipelines using Docker.




- Deploy, manage, and monitor containerized workloads on Azure Kubernetes Service (AKS) and Azure Container Apps (ACA).

Stakeholder Management & Technical Governance :

- Independently drive maintenance, bug fixes, operational readiness, and stakeholder interactions.
- Collaborate with internal business verticals (Wheels, Collections, Emerging Business) and external partners (FSS Subsidiaries, Data Vendors) to translate requirements into technical solutions.
- Partner with Engineering Management on major architectural calls, infrastructure investments, and technology roadmaps.

Duties & Responsibilities :

- Design, develop, and maintain API-led integration solutions and data engineering pipelines to support business applications.
- Build and enhance scalable REST APIs using frameworks such as FastAPI, ensuring high performance and reliability.
- Develop and optimize data processing workflows using Databricks (PySpark, SQL) for effective data transformation and analytics.
- Collaborate with business and technical teams to understand requirements and translate them into robust API and data solutions.
- Implement containerized applications using Docker and deploy/manage services on cloud platforms such as Azure Container Apps (ACA) and AKS.
- Develop and manage data integration workflows using Azure Data Factory (ADF)



or equivalent to enable seamless data movement across systems.
- Ensure adherence to best practices in API design, microservices architecture, security, and performance optimization.
- Troubleshoot, debug, and resolve issues related to APIs, data pipelines, and cloud deployments.
- Stay updated with emerging technologies and continuously improve technical skills in API engineering, cloud-native development, and data platforms.
- External Stakeholders : FSS Subsidiary members and Data vendors.

5. Key Challenges :

- Ensuring high performance, scalability, and reliability of REST APIs under varying workload conditions.
- Managing secure API integrations and data exchange across multiple systems with proper authentication and governance.
- Deploying, monitoring, and optimizing containerized applications on Docker, ACA, and AKS environments.
- Handling large-scale data processing and optimization in Databricks (PySpark, SQL) while maintaining efficiency.
- Troubleshooting and maintaining stability across APIs, data pipelines, and cloud-native infrastructure.

Skills :

- Develop scalable and high performance REST APIs using FastAPI.
- Strong hands-on experience with Databricks (PySpark, SQL) for data processing.
- Experience in building and managing data pipelines and integrations.
- Proficiency in SQL for data querying, transformation, and optimization.
- Hands-on experience with Docker and deployment on ACA/AKS environments.
- Ability to develop and manage workflows using Azure Data Factory (ADF) or equivalent.
- Good understanding of database design and data architecture.
- Strong debugging, troubleshooting, and performance tuning skills.
- Python programming for API development and data engineering use cases.

📌 Data Engineer (India)
🏢 Magnet HR Consulting Services
📍 India