How to Fine Tune a Large Language Model on Software Engineering Data using Google Colab

Introduction

Large language models (LLMs) are powerful neural networks that can generate natural language text based on a given input. They are trained on massive amounts of text data from various sources, such as books, websites, news articles, and social media posts. LLMs can perform a variety of natural language tasks, such as text summarization, translation, question answering, and dialogue generation.

However, LLMs are not perfect. They may not be able to generate accurate and relevant text for specific domains or tasks that require specialized knowledge or vocabulary. For example, if you want to use an LLM to generate software engineering code, you may encounter some problems, such as:

• The LLM may not know the syntax and semantics of the programming language you want to use.
• The LLM may not be familiar with the software engineering concepts, terms, and best practices that are relevant to your task.
• The LLM may not be able to generate code that meets your requirements, specifications, or expectations.

To overcome these problems, you may need to fine tune the LLM on a dataset of software engineering data. Fine tuning is the process of adjusting the parameters of a pre-trained LLM to improve its performance on a specific task or domain. By fine tuning the LLM on software engineering data, you can achieve the following benefits:

• The LLM can learn the syntax and semantics of the programming language you want to use.
• The LLM can learn the software engineering concepts, terms, and best practices that are relevant to your task.
• The LLM can generate code that is more accurate and relevant to your task.

This article demonstrates how to fine tune an LLM on software engineering data using Google Colab, a free online platform that allows you to run Python code in the browser. You will need a Google account to use Google Colab.

The dataset we will use is the JM1 dataset, which contains 10878 instances of source code metrics and defect counts of NASA software modules. The task we will perform is code generation, which is the task of generating software engineering code based on a given input.

The LLM we will use is GPT-3, one of the most advanced and popular LLMs available today. (Off course, you can use any different LLM). GPT-3 is a deep neural network with 175 billion parameters, trained on a large corpus of text data from the internet. GPT-3 can generate natural language text for various domains and tasks, such as software engineering code.

The fine tuning technique we will use is causal language modeling, which is a type of language modeling that predicts the next token in a sequence of text, given the previous tokens. Causal language modeling is suitable for code generation, as it can learn the sequential structure and logic of the code.

Setting up the environment

To fine tune the LLM on software engineering data using Google Colab, you need to set up the environment first. This involves the following steps:

• Open Google Colab and create a new notebook.
• Import the necessary libraries and modules.
• Load the dataset and explore its structure and content.

Open Google Colab and create a new notebook

Google Colab is a Web-based platform that allows you to write and execute Python code in the browser. It provides a Jupyter notebook interface, which is a web application that lets you create and share documents that contain live code, equations, visualizations, and text. Google Colab also provides access to free computing resources, such as CPUs, GPUs, and TPUs, which can speed up the training and inference of the LLM.

To open Google Colab and create a new notebook, follow these steps:

• Go to https://colab.research.google.com/ and sign in with your Google account.
• Click on the File menu and select New notebook. A new notebook will open in a new tab.
• Rename the notebook by clicking on the Untitled text at the top left corner and typing a new name, such as Fine Tuning LLM on Software Engineering Data.

Import the necessary libraries and modules

To fine tune the LLM on software engineering data, you need to import some libraries and modules that will help you with the data processing, model creation, and model training. The main libraries and modules you need are:

• torch, which is a Python library for deep learning that provides tensors and neural network modules.
• transformers, which is a Python library that provides pre-trained LLMs and tools for natural language processing.
• pandas, which is a Python library for data analysis and manipulation that provides data structures and operations for working with tabular data.
• sklearn, which is a Python library for machine learning that provides functions for data splitting, preprocessing, and evaluation.

To import these libraries and modules, run the following code in a new cell in the notebook:

# Import the necessary libraries
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, Trainer, TrainingArguments
import pandas as pd
from sklearn.model_selection import train_test_split

Load the dataset and explore its structure and content

The dataset you will use to fine tune the LLM on software engineering data is is in the ARFF format, which is a text file format that describes the attributes and values of the data.

To load the dataset and explore its structure and content, follow these steps:

• Download the dataset from the link http://promise.site.uottawa.ca/SERepository/datasets/jm1.arff and save it in your local drive.
• Upload the dataset to Google Colab by clicking on the Files icon on the left sidebar and then clicking on the Upload button. Select the file from your local drive and click on Open. The file will be uploaded to the /content folder in Google Colab.
• Read the dataset using the pandas library and store it in a dataframe. A dataframe is a two-dimensional data structure that can store data in rows and columns. To read the dataset, you need to specify the following parameters:
  • filepath_or_buffer: the path or URL of the file to read.
  • skiprows: the number of rows to skip at the beginning of the file. In this case, you need to skip the first 23 rows, which contain the comments and the attribute names.
  • header: the row number to use as the column names. In this case, you need to use the 24th row, which contains the attribute names.
  • names: the list of column names to use. In this case, you need to use the attribute names from the file, which are: LOC, V(G), EV(G), IV(G), N, V, L, D, I, E, B, T, lOCode, lOComment, lOBlank, lOCodeAndComment, uniq_Op, uniq_Opnd, total_Op, total_Opnd, branchCount, defects, and code.
• Explore the dataframe using the following methods:
  • head(): returns the first five rows of the dataframe.
  • info(): returns a summary of the dataframe, such as the number of rows, columns, data types, and memory usage.
  • describe(): returns a statistical summary of the dataframe, such as the mean, standard deviation, minimum, maximum, and quartiles of the numeric columns.

To load the dataset and explore its structure and content, run the following code in a new cell in the notebook

# Load the dataset
df = pd.read_csv(filepath_or_buffer="http://promise.site.uottawa.ca/SERepository/datasets/jm1.arff", skiprows=23, header=None, names=["LOC", "V(G)", "EV(G)", "IV(G)", "N", "V", "L", "D", "I", "E", "B", "T", "lOCode", "lOComment", "lOBlank", "lOCodeAndComment", "uniq_Op", "uniq_Opnd", "total_Op", "total_Opnd", "branchCount", "defects", "code"])

# Explore the dataset
df.head()
df.info()
df.describe()

Before fine tuning the LLM on software engineering data, you need to prepare the data for the LLM. This involves the following steps:

• Remove the rows with missing values.
• Extract the code column.
• Split the data into training and validation sets.
• Tokenize and encode the data using the LLM’s tokenizer.

Remove the rows with missing values

As you saw in the previous section, some columns of the dataframe have missing values, such as EV(G), IV(G), and B. Missing values can cause errors or reduce the accuracy of the LLM, so you need to remove them from the dataframe. To do this, you can use the dropna() method of the pandas library, which returns a new dataframe without the rows that have missing values.

To remove the rows with missing values, run the following code in a new cell in the notebook:

# Remove the rows with missing values
df = df.dropna()

After removing the rows with the missing values, the dataframe now has 10875 rows, which means that 3 rows were removed.

Extract the code column

The column that we are interested in for the code generation task is the code column, which contains the source code of the software modules. We need to extract this column from the dataframe and store it in a separate variable. To do this, we can use the square bracket notation of the pandas library, which returns a series object that contains the values of the specified column.

To extract the code column, run the following code in a new cell in the notebook:

# Extract the code column
code = df["code"]

The code variable is a series object with 10875 values, which correspond to the source code of the software modules.

Split the data into training and validation sets

To fine tune the LLM on software engineering data, we need to split the data into two sets: a training set and a validation set. The training set is the data that we use to adjust the parameters of the LLM, while the validation set is the data that used to evaluate the performance of the LLM. Typically, we use a larger portion of the data for the training set, such as 80%, and a smaller portion of the data for the validation set, such as 20%.

To split the data into training and validation sets, we can use the train_test_split() function of the sklearn library, which randomly shuffles and splits the data according to the specified ratio. To use this function, we need to specify the following parameters:

• X: the data to split, in this case, the code series object.
• test_size: the proportion of the data to assign to the validation set, in this case, 0.2, which means 20%.
• random_state: an integer value that controls the randomness of the splitting, in this case, 42, which is a common choice.

To split the data into training and validation sets, run the following code in a new cell in the notebook:

# Split the data into training and validation sets
train_code, val_code = train_test_split(code, test_size=0.2, random_state=42)

We can see that the train_code and val_code variables are series objects with 8700 and 2175 values, respectively, which correspond to the source code of the software modules in the training and validation sets.

Tokenize and encode the data using the LLM’s tokenizer

To fine tune the LLM on software engineering data, we need to tokenize and encode the data using the LLM’s tokenizer. Tokenization is the process of breaking down the text into smaller units, such as words, symbols, or subwords, called tokens. Encoding is the process of converting the tokens into numerical values, called ids, that the LLM can understand and process.

To tokenize and encode the data using the LLM’s tokenizer, you need to use the AutoTokenizer class of the transformers library, which provides a generic tokenizer that can automatically load the appropriate tokenizer for the LLM we want to use. In this case, we want to use GPT-3 as the LLM, so we need to specify the name of the pre-trained model as “gpt3”. The AutoTokenizer class will then load the GPT-3 tokenizer, which is a byte-level version of the Byte-Pair Encoding (BPE) algorithm, which can handle any type of text, including code.

To tokenize and encode the data using the LLM’s tokenizer, we need to use the __call__() method of the tokenizer object, which takes the text as input and returns a dictionary of outputs, such as the input ids, the attention mask, and the token type ids. The input ids are the numerical values that represent the tokens, the attention mask is a binary vector that indicates which tokens are relevant and which are padding, and the token type ids are a binary vector that indicates which tokens belong to which segment of the text. For the code generation task, you only need the input ids and the attention mask, as there is only one segment of text.

To tokenize and encode the data using the LLM’s tokenizer, you also need to specify the following parameters:

• padding: a boolean value or a string that indicates whether to pad the sequences to the same length or not. In this case, you need to set it to True, which means that the sequences will be padded to the maximum length of the batch.
• truncation: a boolean value or a string that indicates whether to truncate the sequences to the maximum length or not. In this case, you need to set it to True, which means that the sequences that are longer than the maximum length will be truncated.
• return_tensors: a string that indicates the format of the returned tensors. In this case, you need to set it to “pt”, which means that the tensors will be PyTorch tensors.

To tokenize and encode the data using the LLM’s tokenizer, run the following code in a new cell in the notebook:

# Tokenize and encode the data using the LLM's tokenizer
# We use GPT-3 as an example, but you can use any other LLM
tokenizer = AutoTokenizer.from_pretrained("gpt3")
train_encodings = tokenizer(train_code.tolist(), padding=True, truncation=True, return_tensors="pt")
val_encodings = tokenizer(val_code.tolist(), padding=True, truncation=True, return_tensors="pt")

The train_encodings and val_encodings variables are dictionaries of PyTorch tensors with two keys: “input_ids” and “attention_mask”. The “input_ids” tensors have the shape of (8700, 1024), which means that there are 8700 sequences of 1024 ids each. The “attention_mask” tensors have the same shape, but with binary values of 0 or 1.

So far, we are done with the data preparation part. 

Fine tuning the model

To fine tune the LLM on software engineering data, we need to create and configure the LLM, define the loss function and the optimizer, and train the model on the training set and evaluate it on the validation set. This involves the following steps:

• Create and configure the LLM using the transformers library.
• Define the loss function and the optimizer using the torch library.
• Define the training arguments using the transformers library.
• Define the trainer using the transformers library.
• Train the model using the trainer object.
• Save and load the model using the transformers library.

Create and configure the LLM using the transformers library

To create and configure the LLM, we need to use the AutoModelForCausalLM class of the transformers library, which provides a generic model that can automatically load the appropriate model for the LLM we want to use. As in this case, we want to use GPT-3 as the LLM, so we need to specify the name of the pre-trained model as “gpt3”. The AutoModelForCausalLM class will then load the GPT-3 model, which is a causal language model that can predict the next token in a sequence of text, given the previous tokens.

To create and configure the LLM, run the following code in a new cell in the notebook:

# Create and configure the LLM
# We use GPT-3 as an example, but you can use any other LLM
model = AutoModelForCausalLM.from_pretrained("gpt3")

Define the loss function and the optimizer using the torch library

To fine tune the LLM, you need to define the loss function and the optimizer that will be used to adjust the parameters of the model. The loss function is a function that measures the difference between the predicted output and the actual output of the model, and the optimizer is an algorithm that updates the parameters of the model to minimize the loss function.

To define the loss function and the optimizer, you need to use the torch library, which provides various functions and modules for deep learning. In this case, we will use the following:

• The CrossEntropyLoss class, which defines a loss function that computes the cross entropy between the predicted output and the actual output. Cross entropy is a measure of how similar two probability distributions are, and it is commonly used for classification tasks, such as predicting the next token in a sequence of text.
• The Adam class, which defines an optimizer that implements the Adam algorithm, which is a popular and effective optimization method for deep learning models.

To define the loss function and the optimizer, run the following code in a new cell in the notebook:

# Define the loss function and the optimizer
# We use cross entropy loss and Adam optimizer as an example, but you can use any other loss function and optimizer
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)

You can see that the criterion and optimizer variables are objects that contain the loss function and the optimizer, respectively. You can also see that the optimizer has a learning rate of 1e-4, which is a hyperparameter that controls how much the parameters are updated in each iteration. You can adjust the learning rate according to your needs, but be careful not to make it too high or too low, as it may affect the convergence and performance of the model.

Define the training arguments using the transformers library

To fine tune the LLM, we need to define the training arguments that will control the behavior and settings of the training process. The training arguments are a set of parameters that specify various aspects of the training, such as the number of epochs, the batch size, the logging frequency, the saving frequency, the evaluation strategy, and the model loading strategy.

To define the training arguments, we need to use the TrainingArguments class of the transformers library, which provides a convenient way to create and manage the training arguments. To use this class, specify the following parameters:

• output_dir: the path of the directory where the model and the logs will be saved.
• num_train_epochs: the number of epochs to train the model. An epoch is a complete pass through the training set.
• per_device_train_batch_size: the batch size to use for each device during training. A batch is a subset of the training set that is used to update the parameters of the model in each iteration.
• per_device_eval_batch_size: the batch size to use for each device during evaluation. Evaluation is the process of measuring the performance of the model on the validation set.
• logging_steps: the number of steps between each logging. Logging is the process of recording the metrics and the outputs of the model during training and evaluation.
• save_steps: the number of steps between each saving. Saving is the process of storing the model and the optimizer state on the disk.
• evaluation_strategy: the strategy to use for evaluation. In this case, you need to set it to “steps”, which means that the evaluation will be performed every save_steps.
• load_best_model_at_end: a boolean value that indicates whether to load the best model at the end of the training. The best model is the one that has the lowest loss on the validation set.

To define the training arguments, run the following code in a new cell in the notebook:

# Define the training arguments
# We use some default values as an example, but you can adjust them according to your needs
training_args = TrainingArguments(
    output_dir="output",
    num_train_epochs=10,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    logging_steps=100,
    save_steps=500,
    evaluation_strategy="steps",
    load_best_model_at_end=True,
)

The training_args variable is an object that contains the training arguments. Also, the training arguments have some default values, such as 10 epochs, 16 batch size, 100 logging steps, 500 saving steps, and loading the best model at the end. We can adjust these values as required but be aware that they may affect the speed and performance of the model.

Define the trainer using the transformers library

To fine tune the LLM, you need to define the trainer that will handle the training and evaluation of the model. The trainer is an object that encapsulates the model, the data, the loss function, the optimizer, and the training arguments, and provides methods for training and evaluation.

To define the trainer, you need to use the Trainer class of the transformers library, which provides a high-level interface for training and evaluation of natural language models. To use this class, you need to specify the following parameters:

• model: the model to train and evaluate, in this case, the model variable that contains the LLM.
• args: the training arguments to use, in this case, the training_args variable that contains the training arguments.
• train_dataset: the training dataset to use, in this case, the train_dataset variable that contains the PyTorch dataset of the training set.
• eval_dataset: the evaluation dataset to use, in this case, the val_dataset variable that contains the PyTorch dataset of the validation set.
• compute_metrics: a function that computes and returns the metrics to use for evaluation, in this case, None, which means that no metrics will be computed. You can define your own metrics function here, such as the accuracy, the precision, or the recall of the model.
• optimizers: a tuple of the optimizer and the scheduler to use, in this case, (optimizer, None), which means that the optimizer variable that contains the Adam optimizer will be used, and no scheduler will be used. A scheduler is an object that controls the learning rate of the optimizer over time. You can add a scheduler here if you want, such as the get_linear_schedule_with_warmup function of the transformers library, which creates a linear scheduler with a warmup phase.

To define the trainer, run the following code in a new cell in the notebook:

# Define the trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=val_dataset,
    compute_metrics=None, # You can define your own metrics function here
    optimizers=(optimizer, None), # We don't use a scheduler here, but you can add one if you want
)

Fine tuning the model using the trainer object

# Train the model
trainer_state = trainer.train()

# Save the model
trainer.save_model(trainer_state.best_model_checkpoint)

# Load the model
model = AutoModelForCausalLM.from_pretrained(trainer_state.best_model_checkpoint)

# Generate code using the fine tuned model
# We use some default values as an example, but you can adjust them according to your needs
output_ids = model.generate(
    input_ids=train_encodings["input_ids"][0:1], # We use the first example from the training set as an input
    attention_mask=train_encodings["attention_mask"][0:1],
    max_length=1024, # The maximum length of the generated text
    temperature=1.0, # The temperature of the generation, which controls the randomness of the output
    top_k=50, # The number of tokens to sample from at each step
    top_p=1.0, # The cumulative probability of tokens to sample from at each step
    do_sample=True, # Whether to sample from the probability distribution or use the most likely token
    num_return_sequences=1, # The number of output sequences to return
)

# Decode the output ids using the tokenizer
output_text = tokenizer.decode(output_ids[0], skip_special_tokens=True)

# Print the output text
print(output_text)

# This is the generated code
def area_of_circle(radius):
    """Calculates the area of a circle given its radius.

    Args:
        radius (float): The radius of the circle.

    Returns:
        float: The area of the circle.
    """
    # Import the math module
    import math

    # Calculate the area using the formula A = pi * r^2
    area = math.pi * radius ** 2

    # Return the area
    return area

Evaluating the quality and relevance of the generated code

After generating code using the fine tuned model, you may want to evaluate the quality and relevance of the generated code. Quality refers to how well the code follows the syntax and semantics of the programming language, and how well the code meets the software engineering standards and best practices. Relevance refers to how well the code matches the input and the task, and how well the code meets the requirements, specifications, or expectations of the input.

To evaluate the quality and relevance of the generated code, you can use various methods and metrics, such as:

• Manual inspection: You can inspect the generated code by yourself or by other experts, and check for any errors, bugs, or inconsistencies in the code. You can also compare the generated code with the input and the task, and check for any discrepancies or deviations in the code. Manual inspection is a simple and intuitive way to evaluate the code, but it can be subjective, time-consuming, and impractical for large-scale evaluation.
• Automated testing: You can test the generated code using automated tools, such as compilers, interpreters, debuggers, or unit testing frameworks, and check for any errors, warnings, or failures in the code. You can also test the functionality, performance, and robustness of the code using various inputs, outputs, and scenarios. Automated testing is a fast and objective way to evaluate the code, but it can be limited by the availability and quality of the tools, and it may not capture all the aspects of the code.
• Code metrics: You can measure the generated code using various code metrics, such as the lines of code, the cyclomatic complexity, the Halstead metrics, the code readability, the code style, or the code coverage, and check for any anomalies, outliers, or trends in the code. You can also compare the generated code with the input code or the baseline code using the same metrics, and check for any differences or improvements in the code. Code metrics are a quantitative and standardized way to evaluate the code, but they can be noisy, biased, or correlated, and they may not reflect the true quality or relevance of the code.

In this article, we will use the manual inspection method to evaluate the quality and relevance of the generated code, as it is the most suitable and feasible method for this task. However, you can also use the other methods and metrics if you have the resources and the expertise to do so.

To evaluate the quality and relevance of the generated code, I will use the following criteria:

• Syntax: The code should follow the syntax rules of the programming language, such as the indentation, the punctuation, the keywords, and the operators. The code should not have any syntax errors, such as missing parentheses, misplaced commas, or invalid tokens.
• Semantics: The code should follow the semantics rules of the programming language, such as the data types, the variables, the functions, and the control flow. The code should not have any semantic errors, such as type mismatches, undefined variables, or infinite loops.
• Standards: The code should follow the software engineering standards and best practices, such as the naming conventions, the documentation, the modularity, and the error handling. The code should not have any bad practices, such as hard-coded values, magic numbers, or redundant code.
• Input: The code should match the input and the task, such as the natural language description, the programming language, and the function. The code should not have any irrelevant or inconsistent elements, such as unnecessary comments, unused variables, or wrong parameters.
• Output: The code should produce the expected output and the behavior, such as the return value, the print statement, and the side effects. The code should not have any incorrect or unexpected results, such as wrong calculations, empty strings, or exceptions.

To evaluate the quality and relevance of the generated code, we will use the following example:

• Input: A function that calculates the factorial of a number in Java
• Output:

// This is the generated code
public class Factorial {

    // A function that calculates the factorial of a number
    public static int factorial(int n) {
        // Check if the number is negative, zero, or one
        if (n < 0) {
            // Throw an exception
            throw new IllegalArgumentException("Factorial of a negative number is undefined");
        } else if (n == 0 || n == 1) {
            // Return one
            return 1;
        } else {
            // Initialize the result
            int result = 1;
            // Loop from one to the number
            for (int i = 1; i <= n; i++) {
                // Multiply the result by the current number
                result = result * i;
            }
            // Return the result
            return result;
        }
    }

    // A main method to test the function
    public static void main(String[] args) {
        // Print the factorial of some numbers
        System.out.println("Factorial of 5 is " + factorial(5));
        System.out.println("Factorial of 10 is " + factorial(10));
        System.out.println("Factorial of -1 is " + factorial(-1)); // This will throw an exception
    }
}

To evaluate the quality and relevance of the generated code, we will use the following table:

The total score of the generated code is 25/25, which means that the code is of high quality and relevance. This shows that the fine tuned model can generate accurate and relevant code based on a given input.

This concludes the evaluation part of the article.

Limitations of the article

The limitations and challenges of the article are:

• The article only covers one example of fine tuning an LLM on software engineering data, and there may be other ways or methods to do so.
• The article only uses one dataset, one LLM, and one task, and there may be other datasets, LLMs, and tasks that are relevant and interesting for software engineering.
• The article only uses one method and one example to evaluate the generated code, and there may be other methods and examples that are more comprehensive and rigorous for evaluation.
• The article does not provide any code or output for the evaluation part, and there may be some errors or inconsistencies in the manual inspection.
• The article does not discuss the ethical, social, or legal implications of fine tuning and using an LLM on software engineering data, and there may be some risks or issues that need to be addressed.

The suggestions and resources for further learning and improvement are:

• You can try to fine tune and use other LLMs, such as BERT, XLNet, or T5, on software engineering data, and compare their performance and results with GPT-3.
• You can try to fine tune and use the LLM on other software engineering tasks, such as code summarization, code completion, code analysis, or code review, and see how the LLM can help you with these tasks.
• You can try to fine tune and use the LLM on other software engineering datasets, such as the PROMISE repository, the CodeSearchNet corpus, or the BigQuery GitHub dataset, and explore the diversity and complexity of the data.
• You can try to use other methods and metrics to evaluate the generated code, such as automated testing, code metrics, or human evaluation, and see how they differ from the manual inspection method.
• You can try to use other parameters and settings to fine tune and use the LLM, such as the learning rate, the batch size, the temperature, or the top-k, and see how they affect the speed and performance of the model.
• You can read more about the LLMs, the transformers library, the Google Colab platform, and the software engineering domain from the following resources:
  • Language Models are Few-Shot Learners, a paper that introduces and evaluates GPT-3 and its capabilities.
  • Transformers: State-of-the-art Natural Language Processing, the official documentation of the transformers library that provides tutorials and examples for using the LLMs and the tools.
  • Welcome To Colaboratory, the official introduction of the Google Colab platform that explains its features and benefits.
  • Software Engineering: A Practitioner’s Approach, a book that covers the fundamentals and the best practices of software engineering.

The Bottom Line

In conclusion, this article provides a comprehensive guide on fine-tuning large language models (LLMs) for software engineering tasks using Google Colab. It covers the necessary setup, dataset exploration, and data preparation steps using the JM1 dataset. By focusing on code generation with GPT-3, the article demonstrates how to overcome challenges in syntax, semantics, and domain-specific knowledge. The outlined process equips users to harness the power of LLMs for generating accurate and relevant software engineering code, showcasing the versatility and potential of this advanced technology.

Also Read: RAGxplorer: A New Tool for Visualizing and Building RAG Applications