Developing a Web-based Q&A system using a RAG-based approach to LLMs - June 25, 2024

Course Information

• Title: Developing a Web-based Q&A system using a RAG-based approach to LLMs
• Instructor: Sujee Maniyam, Node 51
• Time: Tuesday, June 25, 8.00 am PDT/11am pm EDT till 2.00pm PDT / 5pm EDT
• Cost: $300 for NET+ AWS and GCP Subscribers, $400 for Internet2 members(see the list of member institutions here), $500 for non-members
• Registration: https://tinyurl.com/3tnfey9n

Introduction

We have all seen how powerful and impactful Large Language Models (LLMs) like ChatGPT are. These very powerful AI models are enabling a new generation of conversational applications. This 6-hour hands-on workshop introduces the audience to LLMs, data augmentation using the Retrieval Generation Augmentation (RAG) approach, and how to how to build web using open-source tools.

Skills Level and Target Audience

• Introductory to Intermediate
• Research IT, Enterprise IT

What You Will Learn

• Embeddings and embedding models; searching text for meanings / context using embeddings
• Vector databases and performing text search with vector databases
• LLMs and using them (via API and locally)
• Developing applications using LLM
• Developing RAG applications
• Deploying containerized applications

Prerequisites

• Comfortable with python programming and notebooks
• Access to Google Colab
• A free subscription to a vector database like MongoDB Atlas
• Sign up for Mistral AI
• GPU access to TBD

Details

1 - Embeddings

• Understanding embeddings
• Various embedding models
• Semantic text search using embeddings
• evaluating a various embedding models

2 - Vector Databases

• Introduction to vector databases
• getting started with MongoDB Atlas
• Loading data into database and populate embeddings
• Vector search with database and embeddings

3 - LLMs

• Introduction to LLMs, the eco system
• Access LLMs via API
• Run LLMs locally using llama-cpp, oolama, lm-studio, jen
• Experiment with different LLMs

4 - Running LLMs

• We will use a framework like llama-cpp, llama-index and Langchain to run local LLMs
• Run a local LLM. Experiment with various open LLMs like Mistral, Llama

5 - Develop a custom application using LLM

• Use frameworks like streamlit, flask to build sample applications

6 - Building RAG Applications

• Here we will query PDF documents using LLMs
• Index documents with embeddings. Use various embedding models (OpenAI, Mistral, open source models)
• query the documents using various LLMs (OpenAI, Mistral, LLama)

7 - Deploying RAG Applications

• Containerizing applications
• Model serving
• Scalability

8 - Workshop / Project

• Attendees will a sample application using data of interest, and LLM of their choice