Llms Workshop: What They Are and How to Leverage Them

A large language model (LLM) is a type of artificial intelligence model designed to understand and generate human language. It consists of parameters and a run file, and it is trained on vast amounts of text data to predict the next word in a sequence. Examples include Meta's LLAMA 2 and LLAMA 3 models.

The presenters are Nathan Mars, the tech lead of the DataViz squad at Grafana Labs, and Horace Rzajac, a software engineer on the Explore team at Grafana Labs.

The LLAMA 2 model, released by Meta.ai, is a large language model with variants ranging from 7 billion to 70 billion parameters. It is an open weights model, meaning that its architecture and parameters are publicly released, allowing anyone to work on it.

Unlike models like ChatGPT, whose architecture and parameters are not publicly available, LLAMA 2's architecture and parameters are released by Meta.ai. This allows individuals to work on the LLAMA 2 model independently.

Model inference is the process of running a trained model to generate outputs, such as text, on a local machine without needing internet connectivity. Model training, on the other hand, involves training the model on large datasets using specialized GPU clusters. Training is computationally intensive and expensive.

Large language models like LLAMA 2 are trained by taking a large chunk of internet text (approximately 10 terabytes) and running it through a GPU cluster with about 6,000 specialized GPUs over 12 days, costing around $2 million. This process compresses the text into a parameter file used by the model.

Pre-training involves training the model on a large amount of internet text to learn general knowledge and language patterns. Fine-tuning involves training the model on a smaller, high-quality dataset with specific instructions to generate desired responses, such as answering questions accurately.

The purpose of fine-tuning a model is to adapt it from a general document generator to a more specialized assistant that can provide accurate and helpful responses to specific queries. This involves training the model with high-quality Q&A documents created by human labelers.

Grafana Labs uses large language models in various ways, such as generating panel titles and descriptions, summarizing incidents, and analyzing flame graph profiling data. These applications help reduce user toil and make complex data more accessible.

Example applications of LLMs within Grafana include the Dashboard Assistant, which helps generate titles and descriptions for panels, and Flame Graph AI, which analyzes performance data and highlights bottlenecks.