Can LLMs Learn? Let’s Customize an LLM to Chat With Your Own Data

Rate this content
Bookmark
Github

Feeling the limitations of LLMs? They can be creative, but sometimes lack accuracy or rely on outdated information. In this workshop, we’ll break down the process of building and easily deploying a Retrieval-Augmented Generation system. This approach enables you to leverage the power of LLMs with the added benefit of factual accuracy and up-to-date information.

This workshop has been presented at C3 Dev Festival 2024, check out the latest edition of this Tech Conference.

FAQ

Retrieval augmented generation (RAG) is an approach where proprietary data is stored in a specialized storage, and based on a user's question, relevant data is retrieved and provided to the LLM to generate an accurate response.

Fine-tuning an LLM with proprietary data is expensive, requires machine learning expertise, and a large amount of data. It may not be effective if the proprietary data is not substantial enough to make a difference in the model.

Andrea suggests using a self-hosted model like Lama if you are concerned about privacy, ensuring that your data never leaves your control. Alternatively, you can use public models for non-sensitive proprietary data.

The workshop demonstrates creating a speaker recommendation application that retrieves information about speakers and talks from the C3 Festival website based on user interests, using Node.js, React, and OpenAI.

The technology stack for the speaker recommendation application includes Node.js, React, and TypeScript. The backend service communicates with an OpenAI model, and the application is deployed using Genesio, a serverless platform.

The brute force approach involves putting all proprietary data into the prompt itself before asking a question. However, this can be inefficient, expensive, and may exceed the maximum prompt length of the model.

You can set up an LLM to chat with your own data by using a method called retrieval augmented generation (RAG), which involves storing your proprietary data in a specialized storage and retrieving relevant context to provide to the model along with your question.

Andrea believes that LLMs save a lot of time with tasks like debugging and writing code, allowing software engineers to focus more on learning, designing architectures, and thinking about products.

Andrea is a software engineer working at Genesio who is passionate about LLMs and generative AI.

The workshop focuses on how to integrate LLMs, OpenAI, and AI models into your own applications, with a specific use case of creating a speaker recommendation application.

Andreia Ocanoaia
Andreia Ocanoaia
48 min
12 Jun, 2024

Comments

Sign in or register to post your comment.
Video Summary and Transcription
Today's workshop introduced the concept of LLMs and their potential to free up time for software engineers. It discussed setting up LLMs to chat with proprietary data, utilizing retrieval augmented generation for small chats, and building a speaker recommendation application using this approach. The workshop also addressed concerns about running data on OpenAI and explored the process of splitting and storing data in a vector database. It concluded with the deployment of an end-to-end application using Genesio and invited attendees to provide feedback and stay in touch.

1. Introduction to LLMs and Generative AI

Short description:

Today we are going to talk about LLMs and how they can free up time for software engineers to focus on product development and robust architectures.

Well, hello, everyone. So I'm Andrea. I'm working at Genesio and I'm so happy to be here and I'm hyped to talk about LLMs and generative AI because it's a pretty interesting subject right now.

Today we are going to talk about LLMs and I am pretty sure that you are also hyped about the topic because you are here at the workshop where you can actually learn how to integrate LLMs, OpenAI, and AI models in general into your own applications.

So I am actually pretty passionate about this subject because OpenAI is actually giving us, and LLMs in general, are actually giving us the gift of time. So I'm going to be honest, I have a few dozen conversations with ChatGPT right now and it fairly saved me a lot of time with debugging and writing code. And now I can actually focus more on learning, on designing architectures, on stuff that ChatGPT cannot do but I can do and I have more time to do it. So actually I'm pretty hyped about this topic because I want to see LLMs and AI models more embedded into our work as software engineers so we can actually free our time to think about products and to think about robust architectures and so on.

2. Setting up LLMs to Chat with Proprietary Data

Short description:

Today I will show you how to set up an LLM, such as OpenAI, to chat with your own data. One challenge is that if you have proprietary data, the model won't know how to respond. To overcome this, we can provide the proprietary data and context to the model. There are different approaches, including fine-tuning the model with proprietary data, but this is expensive and requires expertise. Another approach is to include all the proprietary data in the prompt itself.

So with that being said and with that in mind, what I want to show you today is how can you actually set up an LLM, such as OpenAI, to chat with your own data.

So, okay. So usually up until now, how we are communicating with LLM models such as OpenAI or LLAMA or any other model, we are just putting some questions, we have a user, he has a question and the model is going to respond to us. But the caveat here is that if we have some proprietary data, the model, unfortunately, is not trained on that data and it won't know how to respond to your question.

So I saw these days a very clear example about this. So, for example, if you want to ask about some policies from your company, so, for example, the vacation days that you have, you cannot ask a model. You can actually have to go to the internal guidelines and policies in your company or to the HR and you have to ask the person and you have to spend time on all of this back and forth. So what we actually can do is we have a way to give the proprietary data to and give a context to the model in order to help us to ask these kind of questions.

So now there are a few more, there are a few approaches that we can actually take. First of all, I want to emphasize when I'm saying proprietary data, some of you probably go directly with the thought to privacy concerns. So there are two things that you can do here. If you are concerned about the privacy of your data, you can actually use a model that you are hosting yourself. So, for example, you can get Lama three models which are open source. You can host them on any cloud provider and you are then sure that your data never leaves this whole environment in this whole architecture. So you have totally control and total privacy for this workshop. I use the open AI just for the convenience because it's already public. It's already there and I don't have to spend time to set it up. But keep it in mind, if you want total privacy, you can host your own model. But not all proprietary data is also sensitive data. So we can actually have a public documentation for an open source project and things like that, that we just can feed to a third party model. So you don't have to worry about that all the time.

So getting back to the presentation and to the approach itself, the first thing that comes to mind is that we can fine tune the model with the proprietary data. But unfortunately, although this is the best thing that we can do, because then the model will natively know the things about the data, this is very expensive and requires machine learning expertise. So fine tuning is actually an art and you have to know how to do it in order to do it right. And you also have to have a lot of proprietary data because otherwise, if you don't have a lot of data about the subject, you won't really make a difference in the model itself because the model is huge. It knows a lot of data. So if I'm just adding a few sentences about a certain topic, it will just get lost in all of the data that is already there. So for this kind of application, fine tuning, it might not really be a solution. So we can actually go and do the naive brute force approach and we can put all the proprietary data into the prompt itself. So before actually asking a question, we can say to the model, here is all the data from my company, all the guidelines or all the policies.

QnA

Watch more workshops on topic

AI on Demand: Serverless AI
DevOps.js Conf 2024DevOps.js Conf 2024
163 min
AI on Demand: Serverless AI
Top Content
Featured WorkshopFree
Nathan Disidore
Nathan Disidore
In this workshop, we discuss the merits of serverless architecture and how it can be applied to the AI space. We'll explore options around building serverless RAG applications for a more lambda-esque approach to AI. Next, we'll get hands on and build a sample CRUD app that allows you to store information and query it using an LLM with Workers AI, Vectorize, D1, and Cloudflare Workers.
Leveraging LLMs to Build Intuitive AI Experiences With JavaScript
JSNation 2024JSNation 2024
108 min
Leveraging LLMs to Build Intuitive AI Experiences With JavaScript
Featured Workshop
Roy Derks
Shivay Lamba
2 authors
Today every developer is using LLMs in different forms and shapes, from ChatGPT to code assistants like GitHub CoPilot. Following this, lots of products have introduced embedded AI capabilities, and in this workshop we will make LLMs understandable for web developers. And we'll get into coding your own AI-driven application. No prior experience in working with LLMs or machine learning is needed. Instead, we'll use web technologies such as JavaScript, React which you already know and love while also learning about some new libraries like OpenAI, Transformers.js
Llms Workshop: What They Are and How to Leverage Them
React Summit 2024React Summit 2024
66 min
Llms Workshop: What They Are and How to Leverage Them
Featured Workshop
Nathan Marrs
Haris Rozajac
2 authors
Join Nathan in this hands-on session where you will first learn at a high level what large language models (LLMs) are and how they work. Then dive into an interactive coding exercise where you will implement LLM functionality into a basic example application. During this exercise you will get a feel for key skills for working with LLMs in your own applications such as prompt engineering and exposure to OpenAI's API.
After this session you will have insights around what LLMs are and how they can practically be used to improve your own applications.
Table of contents: - Interactive demo implementing basic LLM powered features in a demo app- Discuss how to decide where to leverage LLMs in a product- Lessons learned around integrating with OpenAI / overview of OpenAI API- Best practices for prompt engineering- Common challenges specific to React (state management :D / good UX practices)
Working With OpenAI and Prompt Engineering for React Developers
React Advanced 2023React Advanced 2023
98 min
Working With OpenAI and Prompt Engineering for React Developers
Top Content
Workshop
Richard Moss
Richard Moss
In this workshop we'll take a tour of applied AI from the perspective of front end developers, zooming in on the emerging best practices when it comes to working with LLMs to build great products. This workshop is based on learnings from working with the OpenAI API from its debut last November to build out a working MVP which became PowerModeAI (A customer facing ideation and slide creation tool).
In the workshop they'll be a mix of presentation and hands on exercises to cover topics including:
- GPT fundamentals- Pitfalls of LLMs- Prompt engineering best practices and techniques- Using the playground effectively- Installing and configuring the OpenAI SDK- Approaches to working with the API and prompt management- Implementing the API to build an AI powered customer facing application- Fine tuning and embeddings- Emerging best practice on LLMOps
Building AI Applications for the Web
React Day Berlin 2023React Day Berlin 2023
98 min
Building AI Applications for the Web
Workshop
Roy Derks
Roy Derks
Today every developer is using LLMs in different forms and shapes. Lots of products have introduced embedded AI capabilities, and in this workshop you’ll learn how to build your own AI application. No experience in building LLMs or machine learning is needed. Instead, we’ll use web technologies such as JavaScript, React and GraphQL which you already know and love.
Building Your Generative AI Application
React Summit 2024React Summit 2024
82 min
Building Your Generative AI Application
WorkshopFree
Dieter Flick
Dieter Flick
Generative AI is exciting tech enthusiasts and businesses with its vast potential. In this session, we will introduce Retrieval Augmented Generation (RAG), a framework that provides context to Large Language Models (LLMs) without retraining them. We will guide you step-by-step in building your own RAG app, culminating in a fully functional chatbot.
Key Concepts: Generative AI, Retrieval Augmented Generation
Technologies: OpenAI, LangChain, AstraDB Vector Store, Streamlit, Langflow

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Building a Voice-Enabled AI Assistant With Javascript
JSNation 2023JSNation 2023
21 min
Building a Voice-Enabled AI Assistant With Javascript
Top Content
This Talk discusses building a voice-activated AI assistant using web APIs and JavaScript. It covers using the Web Speech API for speech recognition and the speech synthesis API for text to speech. The speaker demonstrates how to communicate with the Open AI API and handle the response. The Talk also explores enabling speech recognition and addressing the user. The speaker concludes by mentioning the possibility of creating a product out of the project and using Tauri for native desktop-like experiences.
AI and Web Development: Hype or Reality
JSNation 2023JSNation 2023
24 min
AI and Web Development: Hype or Reality
Top Content
This talk explores the use of AI in web development, including tools like GitHub Copilot and Fig for CLI commands. AI can generate boilerplate code, provide context-aware solutions, and generate dummy data. It can also assist with CSS selectors and regexes, and be integrated into applications. AI is used to enhance the podcast experience by transcribing episodes and providing JSON data. The talk also discusses formatting AI output, crafting requests, and analyzing embeddings for similarity.
The Rise of the AI Engineer
React Summit US 2023React Summit US 2023
30 min
The Rise of the AI Engineer
Watch video: The Rise of the AI Engineer
The rise of AI engineers is driven by the demand for AI and the emergence of ML research and engineering organizations. Start-ups are leveraging AI through APIs, resulting in a time-to-market advantage. The future of AI engineering holds promising results, with a focus on AI UX and the role of AI agents. Equity in AI and the central problems of AI engineering require collective efforts to address. The day-to-day life of an AI engineer involves working on products or infrastructure and dealing with specialties and tools specific to the field.
TensorFlow.js 101: ML in the Browser and Beyond
ML conf EU 2020ML conf EU 2020
41 min
TensorFlow.js 101: ML in the Browser and Beyond
TensorFlow.js enables machine learning in the browser and beyond, with features like face mesh, body segmentation, and pose estimation. It offers JavaScript prototyping and transfer learning capabilities, as well as the ability to recognize custom objects using the Image Project feature. TensorFlow.js can be used with Cloud AutoML for training custom vision models and provides performance benefits in both JavaScript and Python development. It offers interactivity, reach, scale, and performance, and encourages community engagement and collaboration between the JavaScript and machine learning communities.
Web Apps of the Future With Web AI
JSNation 2024JSNation 2024
32 min
Web Apps of the Future With Web AI
Web AI in JavaScript allows for running machine learning models client-side in a web browser, offering advantages such as privacy, offline capabilities, low latency, and cost savings. Various AI models can be used for tasks like background blur, text toxicity detection, 3D data extraction, face mesh recognition, hand tracking, pose detection, and body segmentation. JavaScript libraries like MediaPipe LLM inference API and Visual Blocks facilitate the use of AI models. Web AI is in its early stages but has the potential to revolutionize web experiences and improve accessibility.
Building the AI for Athena Crisis
JS GameDev Summit 2023JS GameDev Summit 2023
37 min
Building the AI for Athena Crisis
Join Christoph from Nakazawa Tech in building the AI for Athena Crisis, a game where the AI performs actions just like a player. Learn about the importance of abstractions, primitives, and search algorithms in building an AI for a video game. Explore the architecture of Athena Crisis, which uses immutable persistent data structures and optimistic updates. Discover how to implement AI behaviors and create a class for the AI. Find out how to analyze units, assign weights, and prioritize actions based on the game state. Consider the next steps in building the AI and explore the possibility of building an AI for a real-time strategy game.