ABOUT THE ROLE

Role Description:

Lets do this. Lets change the world. We are looking for a highly motivated expert Data Engineer to design and developscalable, secure, and reliable data pipelines and ingestion solutions that power knowledge layers and assistant experiencesviagenerative AI solutions for our Manufacturing Applications Product Team. The ideal candidate will be responsible for designing, developing, and optimizing data pipelines, data integration frameworks, and metadata-driven architectures that enable seamless data access and analytics for Manufacturing and Operations use cases. This role requires deep expertise in big data processing, distributed computing, data modeling,LLMs, vector stores,productized assistant workflows. and governance frameworks to support self-service analytics, AI-driven insights, and enterprise-wide data management.

Roles & Responsibilities

- Design, develop, and maintain complex ETL/ELT data pipelines in Databricks using PySpark, Scala, and SQL to process large-scale datasets.

- Champion ML/LLM feature engineering including data ingestion, embeddings/vector DBs, RAG/LLM serving, latency optimization,and knowledge graph/metadata.

- Build highly efficient data pipelines to migrate and deploy complex data across systems, with an understanding of biotech/pharma/manufacturing or related domains.

- Design and implement solutions to enable secure access, logging, privacy controls.unified data access, governance, and interoperability across hybrid cloud environments.

- Ingest and transform structured and unstructured data from databases (PostgreSQL, MySQL, SQL Server, MongoDB, etc.), APIs, logs, event streams, images, PDFs, and third-party platforms.

- Ensure data integrity,



accuracy, and consistency through rigorous quality checks and monitoring.

- Innovate, explore, and implement new tools and technologies to enhance efficient data processing.

- Proactively identify and implement opportunities to automate tasks and develop reusable frameworks.

- Work in an Agile and Scaled Agile (SAFe) environment, collaborating with cross-functional teams, product owners, and Scrum Masters to deliver incremental value.

- Use JIRA, Confluence, and Agile DevOps tools to manage sprints, backlogs, and user stories.

- Support continuous improvement, test automation, and DevOps practices in the data engineering lifecycle.

- Collaborate and communicate effectively with product teams and cross-functional teams to understand business requirements and translate them into technical solutions.

Must-Have Skills

- Hands-on experience in data engineering technologies such as Databricks, PySpark, SparkSQL, Apache Spark, AWS, Python, SQL, and Scaled Agile methodologies.

- Proficiency in workflow orchestration and performance tuning on big data processing.

- Strong understanding of AWS services.

- Ability to quickly learn, adapt, and apply new technologies.

- Strong problem-solving and analytical skills.

- Excellent communication and teamwork skills.

- Experience with Scaled Agile Framework (SAFe),



Agile delivery practices, and DevOps practices.

- Experience with streaming technologies such as Apache Kafka, Debezium, or similar platforms for real-time data processing and integration.

Valuable-to-Have Skills

- Experience with AI assisted code development using tools like GitHub Copilot, Cursor, Claude Code.

- Collaboration with ML engineers, prompt engineers, Product Managers and Owners.

- Data engineering experience in biotechnology or pharma industry.

- Experience in writing APIs to make data available to consumers.

- Experience with SQL/NoSQL databases, vector databases for large language models.

- Experience with data modeling and performance tuning for both OLAP and OLTP databases.

- Experience with software engineering best practices, including version control (Git, Subversion, etc.), CI/CD (Jenkins, Maven, etc.), automated unit testing, and DevOps.

- Experience with manufacturing related data sources like SCADA, Data Historian is a plus

Education and Professional Certifications

- Doctorate Degree OR

- Masters degree with 6-8 years of experience in Computer Science, IT or related field OR

- Bachelors degree with 8-10 years of experience in Computer Science, IT or related field

Soft Skills

- Excellent analytical and troubleshooting skills.

- Strong verbal and written communication skills.

- Ability to work effectively with global, virtual teams.

- High degree of initiative and self-motivation.

- Ability to manage multiple priorities successfully.

- Team-oriented, with a focus on achieving team goals.

- Ability to learn quickly, be organized, and detail-oriented.

📌 Sr Machine Learning Engineer (Hyderabad)
🏢 Amgen
📍 Hyderabad