Chat GPT is currently the talk of the town, and it turns out that GPT4, in particular, is very powerful. But the model also has its weaknesses. One of them is that it was trained on a huge amount of data, but it only lasts until September 2021. This means that current information is not taken into account in the model. There is a browser plug-in for the Pro users, but my tests with it have shown that the search is not very productive, especially when GPT has to research the data from numerous different sources in advance, which also takes a relatively long time. Secondly, this plugin is not included in the API.

So if you want to build a web app such as a chatbot with GPT, you need other ways to get around these disadvantages. In the last few weeks, I have been working intensively on solutions for this, whereby the use of langchain in connection with the vector database Pinecone, in particular, seems to be gaining ground. Langchain is a framework for large language models (LLMs) that can be used for generative question answering, text summarisation etc. Components such as prompt templates, agents, memory or LLMs can be “chained”.

GPT only allows a certain number of tokens as input and also only generates a certain number of tokens as output (depending on the version). The approach presented below has the following idea. If one wants to use external data, this data must be converted into vectors (document embedding). This data is then stored in Pinecone, a vector database. If a user then asks a question in the app, this is also converted into vectors. This vector is then compared with the vectors in the database, and the most similar vectors (text elements) are used to generate the prompt. In this way, the problem of the limited number of tokens can be elegantly circumvented.

I have built a prototype based on this idea that can be tested. I have crawled some descriptions of Amsterdam from Lonely Planet (as a prototype, I did not pay attention to completeness). You can now ask the Tourism Bot questions about Amsterdam (in any language).

Probiere den Prototypen