The Ultimate Guide to Monitoring AI in Your No Code SaaS

Analytics and Monitoring with Cloudflare AI Gateway

If you've built an AI powered app and you're not using the Cloudflare AI gateway, here are four reasons why, followed by a demonstration of how we can add this in to a Bubble no code app. First of all is analytics. Cloudflare sits in the middle between your app and the AI provider, whether that's OpenAI or anthropic, or even one that's hosted by Cloudflare themselves. Like you can host your own AI powered workers with llama, etcetera. You get analytics, you get to see tokens, you get to see what your users have submitted.

Real-time Logs and Data Visibility

You literally see the data that goes from your app to the AI and back, and you can go and dig through it, see exactly what your users are doing on your app. Yeah, so we've got analytics, you've got real time logs, we've then got caching. So you've got the ability to say, well, what if people in my app keep asking the same question? Does it need to take that extra bit of time to wait for the LLM to send a reply back through the API? Or can I serve up the same answer that was queried previously?

Caching and Cost Savings

Well, you can do that. You can make your app quicker and you can save costs on tokens by using caching. You've then got rate limiting, so it offer so many. And you should definitely have this set up in the AI provider dashboard. Again, whether that is OpenAI or that is anthropic.

Rate Limiting and Security

Make sure you've got hard and soft limit set. You don't want your API key accidentally getting out there. You don't want a rogue person on your app spending all of your tokens racking up a huge bill. So you want to make sure that your card, especially if you've got all auto billing or auto charge, auto top up. You want to protect a massive bill from racking up on your account.

Additional Rate Limiting and Monitoring

So you can also set up additional rate limiting, again, making sure that when data goes from your app through Cloudflare, the Cloudflare is doing that monitoring for you and rate limiting per how you set up. We've then got a request for try and fall back. So if AI is mission critical to your app, the worst thing that can happen is that the OpenAI API goes down. We've already seen that in the news. It's similar to when AWS goes down.

Fallback Options for Reliability

It just seems like half the Internet stops working. Well, you can make sure that your app isn't in that half of the Internet because you can easily add in fallback. So you can say in your request and we've got a video demonstrating this, but the documentation is just on Cloudflare. You can say try OpenAI first, but if that fails, just send the same request to Anthropic. If that fails, add in another, add in another.

Multiple AI Provider Options

You can add in multiple fallbacks. And then we also get a reminder here that you can use workers AI. That's a model hosted on Cloudflare, but you can also connect directly to OpenAI as your OpenAI hugging face. Anthropic. They're basically perplexity. They're basically all in there. So let's dive into a demo of how easy it is to set this up with Bubble.

Setting Up Cloudflare AI Gateway in Bubble

Basically, if you've built a Bubble app and you've connected to an AI API, this is no different. But if you are building a Bubble app, then please click the link down in the description to check out all hundreds and hundreds of our Bubble tutorial videos. We've also got courses and we've got a no code community you can check out on our website.

Creating an AI Gateway in Cloudflare

But let's dive into a demo. I'm in my Cloudflare account and this is all free. This is available on Cloudflare free, but you will be using your API key from your own OpenAI account, anthropic etcetera. So that's where you get built. First thing we need to do is create an AI gateway.

Configuring the AI Gateway

And this is going to form part of your endpoint that sits in the middle between your app, Cloudflare, OpenAI, and it's one way of just grouping together analytics and settings. So we'll just call this one demo and you can already get a taste. Oh, okay. Actually I've. That's interesting. I deleted demo and then I renewed it. There we go. That's something learned. If you create a gateway with the same name, it loads in old data. Maybe there is some period to which it doesn't delete previous data.

Exploring Analytics and Logs

Anyway, I already had one called demo because I wanted to check this work before hitting record on this video. But this allows us to immediately see the source of analytics you can get. You can see the requests, you can see the tokens cost, although I think that's limited to just OpenAI for now, where the cache that's been hit and errors, and then we can dive into logs. So we can see, here are the requests that have been sent. We can see the question was asked, what is Cloudflare?

Analyzing User Interactions

And here is the response that comes back, you're just going to get so much data on your users and you can then just write your users activity with AI in your app. Yeah, the main thing that I've tried here is by sending custom metadata. So if you go into the Cloudflare API documentation, you can see how you can send additional data. And this can be a good way of grouping and easily filtering down what actions particular users take in your app. So I've just called the user 12345 and so I can now do a search and I can now just view the requests, the AR requests that the user 12345 has made.

Integrating Cloudflare AI Gateway with Bubble

But without further ado, let's actually add this into our Bubble app. So it's really easy if you worked with a Bubble API connector or you worked just with the OpenAI API. It's basically as simple as that. First of all, you can start with the universal one. This is what you would use if you needed to add in fullback because you're saying first try this AI and then try that AI if it doesn't work.

Setting Up the API Connector in Bubble

But if we want to just directly connect OpenAI, then we can click on here we have this endpoint. I will be deleting this afterwards. And then we've got this SQL co expression here. So let's add this into our Bubble apps. I'm going to copy this. I'm going to copy that. By the way, I'm using Setapp and I've got the paste app. It's amazing for keeping history on your clipboard. So let's go into our Bubble app and add in a new AI or API rather. So I've got, I'm in the Bubble API connector and I'm going to say Cloudflare AI gateway.

Configuring API Settings

I'm going to say private key and header. And then just because there's an annoying copy and paste from Cloudflare that I found when I was testing this, I'm going to choose post because go back here, it's got to be a post command. What's annoying is you can't select an individual bit. So I'm just selecting the whole thing, pasting into body and then taking out the relevant part. So here's our endpoint. It's going up there and authorization. Now when we connect directly through Cloudflare to a specific AI API, we just use that AI API's API key in authorization up here. I am just going to paste in my OpenAI API key. And of course I will be deleting that after I've run this demo.

Testing the Integration

Okay. By default with Bubble content type application, JSON is sent. Cool. We've done all that and then I just need to put in the data section here. And now this looks very similar, of course, to any other time that you've used an AI API. Of course if you wanted to use this in Bubble as part of workflow, we would change this to action. But I think that is now everything we need to run it. Let's put in a alternative question. We'll say what is Bubble.io and what is your take on the future of no code app development? Let's see what it says.

Analyzing the Response

Let's Derek. So we'll initialize the call. So this is going from Bubble to Cloudflare to OpenAI, back through Cloudflare and back to Bubble. So we get this response here. Bubble is a no code platform that allows users to build web apps and it just looks exactly like the data we get straight back from OpenAI.

Customizing Metadata for Tracking

Now, of course, the important bit is that we get the real time logs. And so we can see here that our user said what is Bubble.io and what is the future of no code? Now I'm going to add in one extra bit here, which is on the platform configuration custom metadata. So how do I track a particular user and make it searchable within the Cloudflare AI gateway? Well, it says I can add metadata to the header, so I'm going to copy this, add it into the header.

Adding Custom Metadata

Now you do just need to understand some basic JSON syntax to make this work. But you know, I could say team PNC user mat test equals true. Let's run it again.

Reviewing the Results

Gonna click save, gonna go back refresh. In fact, has it done that automatically? I think it has. So there we go. We get mat. And so that then means that if I wanted to see, so, you know, if Matt asked me, but if someone called Matt had reached out and said that part of the app wasn't working or they'd submitted something and got an error, I can dive right in. And frankly, this is much easier than the Bubble logs to see what's gone wrong, especially if you're using backend workflows to run your AI workflow actions.

Benefits of Using Cloudflare AI Gateway

This is just going to take that headache away because you can get right into the data of what your users are doing and remember, you can rate limit it, you can cache it. So there's tons do check out, of course, privacy implications of sending data through Cloudflare. That's something to consider. But as an analytic tool and something that adds in a series of really helpful kind of safety precautions, if you're building a no code sas app by AI, then I can't see why you wouldn't be using Cloudflare's AI gateway.