Train OpenAI GPT on App Data: 2 Proven Methods [2024]

Two Ways to Provide Training Data for AI in No-Code Apps

Here are two ways that you can provide your own training data or background knowledge for an AI in a no-code app which you could be building with Bubble or any no-code platform out there. And I'm going to just talk about the pros and cons of each one. We're going to be talking about OpenAI's API in particular with chat completion but also the Assistant's API pros and cons coming up. But before I launch into it, click the link down in the description if you want to learn more about building an app with Bubble and how to launch your startup and accelerate that process by joining our community and accessing all of the videos we've ever made on the topic of Bubble and no-code.

The Chat Completion Endpoint

Let's start here. The chat completion endpoint is the basics. This is the protocol, the manner of which we've been able to generate AI content with text generation for a couple of years now. And it works really well with Bubble because each time you have to submit the whole history of the conversation. So we can see here in the example, here is the system prompt and then here is the user message and then next message would be the Assistant's reply and then if the user has a conversation then it would be the user assistant user assistant. And to get the historical awareness of the conversation to work, for example if you said plan a trip to London and then you asked a follow-up message saying I'd like to go to the second recommendation, it would know what its second recommendation was only because you have supplied the whole history of messages each time you send the call.

Increasing Context Window

And most of our OpenAI videos, if you do a search for them, they work this way. And as AIs advance, you know, it is there's like a new announcement every day, a serious new announcement every week it seems. The context window, that means how much data, how many tokens, and a token being kind of roughly a short word, three, four characters, how many tokens can you send in that request to the API? And I remember the days when it was 8k. Well, GPT 4.0 is 128k tokens, so you know that is a book, a small book.

Quick MVP Solution

So if you want a really quick way, if you're going down that MVP-purist route, then I would suggest you find a way of taking any data that you want the AI to be aware of and you simply include it in messages at the start of the conversation. So if I was building a CRM and I wanted to add in AI features for my users, so perhaps they could query, you know, what was that email that I received two weeks ago? Well, if you just list all of the emails, now be very aware of the privacy issues there could be with doing that, but if you were to list all of the emails in, say, the system prompt or even a user message that comes earlier in the conversation saying, "Here is the background data you need to know in order to answer the user's questions," you can just include all of that text, paragraphs, paragraphs, pages and pages of text into the prompt and once it's in that message history, the AI should be able to use that for context.

The Assistants API

Now, the other way of going about this is the Assistants API and let's point out right on the top that this is in beta, which means that they could change it, how it works, how it interacts with your application. There's just a general rule that is so easy to get burnt if you integrate beta software into a production application, so there's your warning. But the Assistants API has two advantages and actually quite a negative, I'm going to say that at the end.

Advantages of Assistants API

The two advantages are that you no longer have to track the full conversation. Imagine you've had 50 messages between your user and the Assistant. Well, you don't have to necessarily store them all in your Bubble app because the Assistants API makes use of threads, which is how you group together messages. So now OpenAI is taking on the burden of keeping track of the conversation and all you need to do is add a message to a thread and then run it for an Assistant message to be generated and added on to the thread. The other advantage with Assistants, let me just see if I can quickly spot this, is that Assistants can be trained because you can supply files to them.

Training Assistants with Files

I'm missing where that vector store files, is this where they want them? It's not really vectors. Okay, take my word for it. There is the ability, is it on threads? Threads? No, Assistants? Oh, it's been a while since I looked at this. Files. Okay, yeah. When you upload a file, you can say that it is for an Assistant, which means that you can provide a library of PDFs that then the Assistant can draw data on. Now, do be aware that there are limitations here in terms of file size and how much you can host per user, but you can approach it differently rather than having to supply all of the content in the opening message each time, which may well be quite expensive in the long run. You can use files to train the Assistant.

Drawbacks of Assistants API in Bubble

Now, here's the negative of using Assistants, at least if you're working in Bubble, which is that the process goes, you create an Assistant, you create a thread, and then you start adding messages to the thread. When you want the AI to respond, you use Run, but this does not inform your application when the AI has written a new message. Now, if the reply is really short, you've only got a few seconds you could add in a pause, and then you could recheck the thread for new messages. But if you're asking the AI to write something lengthy, it could take 10-20 seconds, and you might not know, or you couldn't know, how long that's going to take.

Challenges with Bubble and Assistants API

The fact is OpenAI does not inform your Bubble app when a new message is created, so you have to keep checking. And this is something that Bubble is not optimized to do, because you can have repeating workflows, you could say every one second on your page, check the thread and import new messages if need be. But it just, to me, that is far from optimal, because you've kind of just got an unlimited, yeah, it's slow, but it's unlimited kind of production conveyor belt of workload units being used up to check if the Assistant has finished, if the Run command has finished, and whether a new message has been written. And it just seems really sub-optimal to do that.

Considerations for Using Assistants API

Now, I would say that if you desperately need to use some of the other features of Assistant, then yes, you can make it work with Bubble, and we've got videos, and other people have got videos how to do that. But it's far from optimal, and I'd be really concerned about the fact that OpenAI does not inform your Bubble app when a new message is written. That is in contrast, of course, to the chat completion endpoint, which when you send an API request, your Bubble app waits until it receives a response. It's kind of all done in that moment, it's drawn out if it's a longer reply, but you don't get that with Assistant.

Bonus: Vector Databases and Pinecone

So there you go, that is two ways. In fact, I'll give you a bonus subway if you've made it this point in the video, which is something that I'll put my hands up and say I've not delved into, which is vectors and using pinecone. This was before the Assistant's API, the original, the OG way, of ensuring that your Bubble app had a large library of data to train an Assistant with. So have a Google for pinecone, Bubble, and OpenAI, and you may well find a third way to achieve this knowledge base, this background knowledge training of a custom GPT.