LangChain

LangChain

Example Code

Examples from this module can be found on GitHub

Before using LangChain, install the langchain package. Since we will be working with OpenAI models, install the langchain-openai package. It will also install the langchain package as a dependency.

pip install langchain-openai

Models

The ChatOpenAI class is a wrapper around the OpenAI API and is used to interact with the OpenAI models. Import it from the langchain_openai package.

from langchain_openai import ChatOpenAI

The ChatOpenAI class expects a couple of keyword arguments, the first of which is the api_key. This is the GitHub developer token you stored in the secrets. Next is the model, and we will continue to use the openai/gpt-4-mini model. Again, notice the model name is prefixed with openai/ since we are using the GitHub-hosted OpenAI models. And also for GitHub Models, include the base_url and use https://models.github.ai/inference as the URL for the endpoint. Optionally you could set the temperature to control the creativity of the response.

llm = ChatOpenAI(
    api_key=os.environ.get("GH_MODELS_PAT"),
    model="openai/gpt-4-mini",
    base_url="https://models.github.ai/inference",
    temperature=0.7
)

To send a prompt to the LLM, call the invoke method and pass in the prompt as a string.

response = llm.invoke("When was Microsoft founded?")

The invoke method returns an AIMessage object. The actual text of the response is in the content attribute.

print(response.content)

The full example code for this section can be found in basic_langchain.py in the 06_langchain/finished folder of the GitHub repository.

Prompts

In addition to wrappers for LLMs, LangChain also includes classes for the other components of a typical LLM application. The ChatPromptTemplate class manages and generates prompts for an LLM. It lives in the langchain_core.prompts module.

from langchain_core.prompts import ChatPromptTemplate

To create a prompt template, call the from_messages class method. The messages passed in the method are a list of tuples (not dictionaries as you saw for the OpenAI API). Each tuple has two elements, the role, and the content. The roles are the same as the OpenAI API, such as system or user.

prompt_template = ChatPromptTemplate.from_messages([
    ("system", "You are a tech analyst that answers questions about big tech companies."),
    ("user", "When was Microsoft founded?")
])

To generate the prompt, call the invoke method on the prompt_template and pass an empty dictionary. (Humor me, you’ll see why in a moment.) The return value can the be used as the prompt for the LLM.

prompt = prompt_template.invoke({})
response = llm.invoke(prompt)
print(response.content)

That might seem like the scenic route if all you need is to send a prompt to the LLM, and you’re right. We haven’t used the capabilities of the ChatPromptTemplate that well. Right now, we can only find out when Microsoft was founded. But what if we wanted to find out when other tech companies were founded? In the user message, we can include a variable enclosed in curly braces. Then when we call the invoke method on the prompt_template, we can pass in a dictionary with key-values pairs for each variable.

prompt_template = ChatPromptTemplate.from_messages([
    ("system", "You are a tech analyst that answers questions about big tech companies."),
    ("user", "When was {company} founded?")
])

prompt = prompt_template.invoke({"company": "Microsoft"})
response = llm.invoke(prompt)
print(response.content)

prompt = prompt_template.invoke({"company": "Apple"})
response = llm.invoke(prompt)
print(response.content)

The full example code for this section can be found in prompt_templates.py in the 06_langchain/finished folder of the GitHub repository.

Messages

In addition to abstracting the models and prompts, LangChain also abstracts the messages. We’ve seen three roles for messages, system, user and assistant. LangChain has classes for each of these roles named SystemMessage, HumanMessage, and AIMessage. These live in the langchain_core.messages module.

from langchain_core.messages import SystemMessage, HumanMessage, AIMessage

Now you can rewrite the prompt template using messages instead of tuples.

prompt_template = ChatPromptTemplate.from_messages([
    (SystemMessage(content="You are a tech analyst that answers questions about big tech companies.")),
    (HumanMessage(content="When was Microsoft founded?"))
])

Wait? Where did the {company} variable go? All of the messages classes are static, they don’t support variables. For messages with variables you have to use a message template. Import the HumanMessagePromptTemplate from the langchain_core.prompts module. Then call the from_template class method to create a message template.

from langchain_core.prompts import HumanMessagePromptTemplate

prompt_template = ChatPromptTemplate.from_messages([
    (SystemMessage(content="You are a tech analyst that answers questions about big tech companies.")),
    (HumanMessagePromptTemplate.from_template("When was {company} founded?"))
])

Now use the prompt_template as before, passing in a dictionary with the company key and the name of the company as the value.

prompt = prompt_template.invoke({"company": "Microsoft"})
response = llm.invoke(prompt)
print(response.content)

The full example code for this section can be found in messages.py in the 06_langchain/finished folder of the GitHub repository.

Output Parsers

Notice that we have to explicitly access the content attribute of the AIMessage to get the text. LangChain has a collection of output parsers that we use to perform post-processing on the output of the LLM. In the langchain_core.output_parsers module, there is a StrOutputParser class that extracts the content from an AIMessage. Import it and create an instance.

from langchain_core.output_parsers import StrOutputParser

output_parser = StrOutputParser()

Pass the response from the LLM to the parse method of the output_parser to get the text.

parsed_output = output_parser.parse(response)
print(parsed_output)

You’ll notice two things. First, the output is the same as before. Second, this is the long way around to get the text. Again humor me.

The full example code for this section can be found in output_parsers.py in the 06_langchain/finished folder of the GitHub repository.

LangChain Expression Language (LCEL)

You’ve now got a complete picture of all of the step in the pipeline. The prompt template, with the help of messages, sends a prompt to a model. The response from the model is then passed to an output parser to pull out the text.

So far in this module, we’ve been taking responsibility for getting the output of one step in the pipeline and passing it to the next step. It’s tedious and error-prone. It also takes a while for someone who didn’t write the code to get up to speed.

LangChain has feature called the LangChain Expression Language, or LCEL, that defines the steps of the pipeline in a single, concise, expression. Essentially, LCEL overloads the | (pipe) operator to direct the output of one step to the next step. The result is a chain. (Hence the name LangChain.)

chain = prompt_template | llm | output_parser

Now we call invoke on the chain and pass in a dictionary with a key for each varaible in the prompt template. The return value is the parsed output from the LLM.

response = chain.invoke({"company": "Microsoft"})
print(response)

response = chain.invoke({"company": "Apple"})
print(response)

And while the output still has not changed (depending on the temperature of the model), the code is much more concise and easier to read.

The full example code for this section can be found in lcel.py in the 06_langchain/finished folder of the GitHub repository.

Refactoring Chat History

In the previous module, you saw how to implement chat history by tracking the previous messages in a list and passing them to the client.chat.completions.create method. To wrap up this module, we’ll refactor that code to use LangChain.

First, remove the import for OpenAI. Next add import for:

• SystemMessage, HumanMessage, and AIMessage from the langchain_core.messages module
• StrOutputParser from the langchain_core.output_parsers module
• ChatPromptTemplate and HumanMessagePromptTemplate from the langchain_core.prompts module
• ChatOpenAI from the langchain_openai package

from langchain_core.messages import SystemMessage, HumanMessage, AIMessage
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate, HumanMessagePromptTemplate
from langchain_openai import ChatOpenAI

Remove the initializer for the OpenAI client and replace it with an initializer for the ChatOpenAI class.

llm = ChatOpenAI(
    api_key=os.environ.get("GH_MODELS_PAT", ""),
    model_name=MODEL,
    base_url="https://models.github.ai/inference",
)

The chat_history is just a plain list. It will contain objects of type SystemMessage, HumanMessage, and AIMessage. The first message in the list is a SystemMessage. After that is the chat history. LangChain includes a class specifically for a list of messages called MessagesPlaceholder. Import it from the langchain_core.prompts module and use it to replace the chat history in the prompt template. The MessagesPlaceholder class takes a single argument, the name of the variable that will hold the chat history. In this case, we will use chat_history. And the current user message is a HumanMessagePromptTemplate with a variable for the user input.

from langchain_core.prompts import MessagesPlaceholder

# ...

prompt_template = ChatPromptTemplate.from_messages([
    (SystemMessage(content=SYSTEM_PROMPT)),
    (MessagesPlaceholder(variable_name="chat_history")),
    (HumanMessagePromptTemplate.from_template("{prompt}"))
])

Now set up the chain with the prompt template, the LLM, and the output parser.

chain = prompt_template | llm | output_parser

Finally, in the chat function, replace the code that calls the client.chat.completions.create method with a call to the invoke method on the chain. Pass in a dictionary with two key-value pairs, one for the user input and one for the chat history.

content = chain.invoke({"prompt": user_input, "chat_history": chat_history})

Now append the user input to the chat_history as a HumanMessage and the response from the LLM as an AIMessage.

chat_history.append(HumanMessage(content=request.prompt))
chat_history.append(AIMessage(content=content))

Run the API with the command:

uvicorn starter:app --reload

Got to https://localhost:8080/docs in your browser and test the API. You should see the same behavior as before, but now the code is much cleaner and easier to read.

The full example code for this section can be found in api.py in the 06_langchain/finished/fastapigpt folder of the GitHub repository.