How to run a Browse AI web scraping robot from Bubble.io

Triggering an AI Web Scraping Robot from Bubble.io

Here's how you can trigger an AI web scraping robot that you've built in Browse AI and trigger it from your Bubble app. We need to be using the Browse AI API to do this. I'm on my extract post from Reddit search, but it's going to be the same whichever one you use. You just have to swap in the robot id and so it gives us the data here to send across and I'm going to open up the API documentation and go into robots and we want to Tasks, yeah, tasks, run a robot.

Setting Up the API Request

So this is what we need to send across to Bubble. I'm going to copy, here we go, put it into the shell version. So we have to send a post request to this URL. I'm going to copy that and go into my Bubble app and we use the API connector to connect to services like this. So go down here, add one in, call it Browse AI, and then we're making a post request to this URL.

Making the Request Dynamic

But we have to swap in the robot id. So let's make that dynamic robot id. So it says there how I can add in params into it, making it dynamic. I'm unticking private just so that if I was in the workflow, private is whether this data is accessible to the user. It doesn't really matter in this case if my users get access to my robot id, but that would allow me to dynamically swap it in in a workflow if I wanted to.

Adding Authorization and Parameters

So let's paste in the robot id that we're working with and go back to the documentation. I have to supply the robot id. I've done that. I have to authorize a bearer API key in the header. So in Bubble, the way I do that is I say private key and header authorization. And then I would write in bearer and I'm going to paste in my API key, which I've got, I think it's this one here. Of course I'll be deleting this before publishing this video. We'll call this one scrape content from Reddit needs to be an action so that I can run it in the workflow and we have to include some parameters.

Configuring Input Parameters

So let's go back to the documentation and just see how we do that. So the two property names, does it give it an example here? Okay, right. So data input property, I'm going to copy all of that. Just so I have an idea of the layout of probably add into parameters here. So the parameters we need to add in are here, we need search text and we need max posts.

So let's try that. We'll say search text. Ah no, this isn't going to work because it's got to have, it's nested inside this key here, input parameters so I can't put in there. So I've got, what was it, search text and this is where I said Bubble.io and then there is another one I guess. Yes, that would be a comma and that would be max posts.

Testing the API Call

Now that's a number, but I think it probably wants me to send it as a string. Let's just try that. So I'm going to see if I get an error and I do that by running initialize call. No, I get a success code and so I'm going to click save.

Verifying the Task Execution

And just so I'm really quick to show you what's going on here, if I go back and then into run tasks, we should see a new task is running. Yes, created by the API and it's running now. So if I have my webhook set up, that data will come through to my webhook as soon as it's finished running. Because remember it can take a web scrape per 5 seconds or if it's got to change some settings spoof being a user, it can take a few minutes to get get a result back.

Using Webhooks for Asynchronous Data Retrieval

That's why we use webhooks. It's so that our Bubble app isn't waiting and when Browse AI has got the data, it's going to send it through to our Bubble app, through to that backend workflow using a webhook.