Building a memory for a chatbot agent

In this homework assignment you are going to be working with langgraph, agents, and tools.

The problem we are going to solve is giving a chatbot a memory.

Earlier in the course I mentioned that the LLMs we are working with can not remember things. If you tell them something in one query they will not be able to remember that in subsequent queries.

In the chapter 12 examples the author demonstrated one technique that starts to address this problem. In that chapter the author built some agents with langgraph. The state information he used for that langgraph application featured a list of messages. That list of messages acts like a conversation transcript. With each query we send to the LLM we send along that transcript of past messages so the LLM has access to that conversation history. This will allow the LLM to carry out something closer to a real conversation with a user.

The only problem with this mechanism is that it does not persist between conversations. If you speak to the travel chatbot, shut down the application, and then start it up again the agent will have no recollection of that earlier conversation.

Our goal for this assignment is to build a chatbot agent that is equipped with a long term memory that can remember key facts across multiple conversations.

What we are going to do

In this assignment you are going to build a chatbot that can carry on conversations with a user and also remember details of earlier conversations.

To start with you should replicate the mechanism the author used in chapter 12 to maintain conversation state within the context of a single conversation.

To enable your chatbot to remember things between conversations you are going to equip your chatbot with a set of tools. These tools will allow the chatbot to store information in a SQLite database for use in later conversations.

The database we are going set up for this assignment will have only one table. That table will have three fields:

  1. A fact field that stores facts from conversations that the chatbot may want to remember for later conversations.
  2. A topic field that associates each fact with a particular conversation topics.
  3. A timestamp field that records when a particular fact entered the database. This will help give the chatbot a sense of time.

To implement a long term memory system for your chatbot you are going to equip the chatbot with the following tools:

  1. A tool that returns a list of all of the topics that have been entered in the database.
  2. A tool that records a new fact given a fact and a topic.
  3. A tool that returns all of the stored facts given a topic.

Your chatbot should use at least one agent that will have access to all three of these tools. In the system prompt for this agent you should tell it that it is a chatbot charged with carrying out a conversation with a user. The chatbot should first try to determine a topic for the conversation by consulting the list of topics the chatbot has discussed with the user in the past. If the conversation aligns with one of those established topics, the chatbot should use that topic for this round of the conversation. If the conversation does not align with any prior topic the chatbot should assign a new topic to this part of the conversation. Once the chatbot has determined what topic to use it should use a tool to fetch facts for that topic and then use those facts to help construct a response for the user. If the user tells the chatbot anything that the chatbot thinks is worth remembering for later, the chatbot should use the appropriate tool to put that fact in its memory.

How to implement the tools

In the chapter 11 and 12 examples the author constructs some tools designed to work with a SQLite database. Those tools work by asking the LLM to generate SQL code that it can send to the database to do any interactions it needs to do with the database.

For this assignment we are going to use a different approach. For each of your three tools you are going to construct a function that takes ordinary parameters, does an interaction with the SQLite database, and then returns a response in JSON form.

For example, the tool that records a new fact will do the following:

  1. The tool function will have two parameters: a string for the fact to store and a string for the topic.
  2. The tool function will construct a SQL Insert statement to insert this information into the database and will execute that statement.
  3. After the successful information the tool will return a confirmation message.

Version upgrade

For this assignment I would like for you to use the latest version of langgraph.

The example code I posted for chapter 12 has been rewritten to use that most recent version. Please note that some of the code in the examples I posted has been modified from the book's code to work with the new version.

To upgrade to the new version, simply run this command in a terminal window:

pip install --upgrade langchain

Which LLM to use

For this assignment you will be using the cmsc106.net LLM. That LLM is best suited for carrying out complex conversations.

Presentation

On the last day of classes I will be asking each of you to give a short five minute presentation demonstrating your chatbot in operation. In your presentation you should demonstrate that if you tell the chatbot something in one conversation and then start up a new conversation the chatbot can successfully retrieve that earlier fact and use it in the new conversation.