Technical Skills Requirement - Experienced Gen-AI Engineer
- Strong proficiency in Python for data processing and automation. Ability to write efficient and well-structured
code.
- Experienced with generative AI models and their integration into data workflows.
- Experienced with prompt engineering and LLM models (Opensource and licensed. Ex Llama, OpenAI)
- Experienced with Application development framework like LangChain or similar frameworks.
- Experienced working with REST frameworks like Quick API, Flask and Django.
- Good understanding of machine learning workflows and deployment.
- Knowledge of ETL processes, data modeling, and data warehousing principles.
- Familiarity with containerization and orchestration tools (Docker, Kubernetes).
- Familiarity with Snowflake for data warehousing and analytics.
- Experienced with cloud platforms (AWS, GCP, Azure) and related services is a plus.
- Strong communication and collaboration skills.
- Excellent problem-solving skills and attention to detail.
- Data Pipeline Development:
- Design and implement scalable data pipelines using Python to ingest, process, and transform log data from various sources.
- Generative AI Integration:
- Collaborate with data scientists to integrate generative AI models into the log analysis workflow.
- Develop APIs and services to deploy AI models for real-time log analysis and insights generation.
- Data Monitoring and Maintenance:
- Set up monitoring and alerting systems to ensure the reliability and performance of data pipelines.
- Troubleshoot and resolve issues related to data ingestion, processing, and storage.
- Collaboration and Documentation:
- Work closely with cross-functional teams to understand requirements and deliver solutions that meet business needs.
- Document data pipeline architecture, processes, and best practices for future reference and knowledge sharing.