Should you use OpenAI Assistant API with Bubble?

Understanding OpenAI Assistant API and Chat Completion API

I've received quite a lot of questions about the OpenAI Assistance API. So I thought I'd record a quick video just to talk about what exactly it is and why it's different from chat completion and whether you should use it in your Bubble app or not. So here I am on the OpenAI documentation and yeah I'm just going to compare and contrast the two different ways of getting AI into your Bubble app. We've done plenty of videos on the technicalities of it but this is more about choosing the right method to get AI into your Bubble app.

The Risks of Using Beta Software

With the Assistance API first thing to notice is that it is in beta and if you've been watching the AI landscape as I'm sure you have you'll realize that things change really rapidly. So it is always a risk if you build something that is production ready and you're using beta software applications in it because OpenAI could shut it down they could change it. We just don't know what's going to happen.

How Chat Completion API Works

Second is just to realize how exactly they work and so actually I'm going to go in and start us with the text generation chat completion API because this is where the bulk of the AI and Bubble videos that I've recorded come from. It is using the chat completion API and you send through an API request and you record all you record you submit through to OpenAI all of the messages that have been part of that conversation and that's how the AI can be aware and contextually aware of the history of the conversation.

Limitations and Advantages of Chat Completion API

So there are some downsides here particularly in the early days when you were limited to how much data how many tokens you could submit in your input. That's no longer such a big issue because we're into the tens of thousands of tokens you know you can submit half a book a whole book now in your prompt so that's no longer an issue but it does mean that it is your role as the app creator in Bubble to keep track of the conversation because your conversation won't be able to have the historical awareness if you aren't able to submit the previous steps.

Implementing Chat Completion in Bubble

Now this is by far the easiest way to add in chat completion AI text generated content into your Bubble app because when you say you submit the system prompt and you submit user that's when your Bubble app waits for a response from the AI and the AI will respond with this message here and you can save that into a thread or a series of chat messages. Again we've got loads of videos covering how to actually implement this but it is that your Bubble app waits and then it gets a response like it's it gets notified when the AI has completed writing the assistant part of the conversation.

How Assistant API Works

Now this is different to how the assistance API works because the assistance API you have to create an assistant. Now the advantage here is that you can have your users create different assistants so you can almost it's like putting your system prompt elsewhere and kind of wrapping it in the persona of an assistant and the benefits of that is are that you can draw on that assistant any way that you like in the app once you've created it and apply that persona to a conversation.

Advantages of Assistant API

The other advantage with the assistance API is that you can upload files and that can basically inform the knowledge base upon which the API draws upon but we create an assistant then we create a thread so this is the main difference it is no longer your role as the app creator or the Bubble app maker to keep track of the conversation. Threads are the way they kept track the message history is held as maintained on the OpenAI server so you basically add a message to the thread so notice we're not sending a list of messages we're just sending the most recent message and yeah OpenAI is keeping track of what's going on and then that doesn't do anything you can literally have your user ask a question and then nothing happens because then you need to run so you need to use the run endpoint to basically bring it all together so you run the thread with any assistant and then that that's then when the AI adds the assistant message as the latest message to the thread.

Challenges with Assistant API in Bubble

Now the main difference and the main reason that I'm not using the assistance API to build AI apps with Bubble is because nothing happens at that point you run then nothing happens on your Bubble app on the OpenAI servers they write the assistance prompt and then nothing happens there's no notification back to your Bubble app and this is perhaps a weakness of Bubble which is that you need to be continuously checking the thread to see if a new message has appeared and this is quite tricky in Bubble because there isn't a great way especially if you're trying to really conserve those workload units there isn't a great way of checking in you know you want to you know you want to be checking like every second because that matters when your user is waiting for a response from the AI and if you are checking every second then you're running a workflow every second to check that a new message has been created and I've seen a number of YouTube Bubble educators do a few different approaches on this and simply put I don't think that there is an optimal way to do it they all have downsides so you can say check every minute no every second you could check every second that a new message has been written into the thread but then you can add in only when statements so you can say only check if the you know you put the the chat into a state that the user has submitted the message and is waiting for a response but then you're actually adding to the workload units that it's going to cost to run the operation because you're making the check the thread for new messages workflow more complicated and therefore it's going to cost more workload units.

Conclusion and Recommendations

So yeah that's it that's basically my assessment of the pros and cons like I say it's not going to be an API that I'm using mainly because it's in beta hopefully they'll add in webhooks that would give it the ability that when you submit a new message to a thread you can say notify my Bubble app of a response here and that would be perfect but it's in beta they say they're developing new features the AI landscape is constantly changing so right now if you wanted to add AI into your Bubble app I would recommend using chat completion now the two main differences are that you can't upload files for chat completion to inform it of noise or at least it's not quite as easy to do as with the assistant but if you wanted to to give a dynamic persona like the assistant in chat completion you could just make your system prompt dynamic so instead of you know you could have you are a helpful assistant for one user and you could have you are a pirate for another user and that's how you can inform the persona of the chat anyway I'd love to hear your thoughts and your opinions your experience and if I've missed anything if there is a really creative way of using the assistant's API in Bubble that doesn't cost a ton of unnecessary workload units please leave a comment down below and let me know.