Can LLMs Learn? Let’s Customize an LLM to Chat With Your Own Data

Andrea suggests using a self-hosted model like Lama if you are concerned about privacy, ensuring that your data never leaves your control. Alternatively, you can use public models for non-sensitive proprietary data.

The workshop demonstrates creating a speaker recommendation application that retrieves information about speakers and talks from the C3 Festival website based on user interests, using Node.js, React, and OpenAI.

The brute force approach involves putting all proprietary data into the prompt itself before asking a question. However, this can be inefficient, expensive, and may exceed the maximum prompt length of the model.

You can set up an LLM to chat with your own data by using a method called retrieval augmented generation (RAG), which involves storing your proprietary data in a specialized storage and retrieving relevant context to provide to the model along with your question.

The technology stack for the speaker recommendation application includes Node.js, React, and TypeScript. The backend service communicates with an OpenAI model, and the application is deployed using Genesio, a serverless platform.

Andrea believes that LLMs save a lot of time with tasks like debugging and writing code, allowing software engineers to focus more on learning, designing architectures, and thinking about products.

Retrieval augmented generation (RAG) is an approach where proprietary data is stored in a specialized storage, and based on a user's question, relevant data is retrieved and provided to the LLM to generate an accurate response.

Fine-tuning an LLM with proprietary data is expensive, requires machine learning expertise, and a large amount of data. It may not be effective if the proprietary data is not substantial enough to make a difference in the model.

Andrea is a software engineer working at Genesio who is passionate about LLMs and generative AI.

The workshop focuses on how to integrate LLMs, OpenAI, and AI models into your own applications, with a specific use case of creating a speaker recommendation application.