We are seeking an experienced AWS Data Engineer to design, build, and maintain scalable cloud-based data solutions. The ideal candidate will have strong expertise in Python, FastAPI, AWS data services, and up-to-date data engineering practices, along with exposure to AI/LLM technologies for building intelligent data-driven applications.
Key Responsibilities:
- Design, develop, and maintain scalable ETL/ELT pipelines using Python and AWS services.
- Build and optimize data ingestion, transformation, and processing workflows for structured and semi-structured data.
- Develop RESTful APIs using FastAPI to support data services and platform integrations.
- Implement data lakes and data warehouses using AWS services such as S3, Glue, Athena, Redshift, and Lambda.
- Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and cost efficiency.
- Collaborate with data analysts, software engineers, and business stakeholders to deliver data solutions.
- Integrate AI/LLM capabilities into data workflows, including document processing, intelligent search,
data summarization, and chatbot-enabled analytics.
- Work with vector databases, embeddings, and Retrieval-Augmented Generation (RAG) concepts where applicable.
- Implement security, governance, and best practices for cloud-based data platforms.
- Participate in code reviews, CI/CD implementation, and infrastructure automation.
- Required Skills:
- Strong proficiency in Python.
- Experience with FastAPI for API development.
- Hands-on experience with AWS services such as S3, Glue, Lambda, Athena, Redshift, IAM, CloudWatch, and EC2.
- Experience building ETL/ELT pipelines and data integration solutions.
- Knowledge of SQL and data modeling concepts.
- Familiarity with CI/CD pipelines and Git-based version control.
- Understanding of AI/LLM concepts, prompt engineering, embedding models, and LLM API integration.
- Exposure to vector databases and RAG-based applications is an added advantage.
📌 Aws Data Engineer (Telangana)
🏢 CGI
📍 Telangana