How to add live web search data to the OpenAI API
Learn how to harness the power of Bubble.io and the no-code movement to combine multiple APIs, including OpenAI, web scraping, and web search, to create a Chat GPT clone that can fetch live data from the web and provide accurate and up-to-date information on things to do in Paris in February 2024.
Combine 3 APIs for live data: OpenAI, web search, and scraping power up your ChatGPT clone!
Fetch real-time info: Learn how to integrate Brave search API and Page2API with OpenAI for up-to-date responses.
Build a smarter AI assistant: Discover how to enhance your Bubble.io ChatGPT clone with live web data in just minutes!
Combining Multiple APIs for Enhanced Functionality
I've got something really cool to show you because I've combined three different APIs. I've got the OpenAI API for text generation with AI. I've got a search, a web search API, and I've got a web scraping API all working together, which means that my ChatGPT clone here, and that's a little bit rough, but it works like ChatGPT, can fetch live data from the web and feed that into the conversation. And here's a really good example. I'm asking the question, things to do in Paris in February 2024.
Limitations of Traditional AI Models
Now, when you ask OpenAI that question through their API, you get back sort of a generalized set of responses. For example, general things that happen in February in Paris, not specifically what happened in Paris in February 2024. And that's because the AI's training data is not necessarily going to have the most up-to-date information with things to do about events.
Combining APIs for Real-Time Data
So what I do with this when I ask the question is I feed my first message into the Brave Search API. I then get a list of the first three websites. I pass those over to the page to API web scraper API. I extract the main body content of those pages, and then I ask OpenAI to summarize that content. So therefore, I'm getting the top three search results with things to do in Paris in February 2024. And I'm passing that data into OpenAI.
Introduction to PlanetNoko.com
And I'm going to give you a tour of how to do that. But before I do that, if you're learning Bubble, there's no better place than PlanetNoko.com where we've got hundreds of Bubble.io videos, just like this one, to really kickstart and accelerate your app development journey.
Workflow Overview
Right, that's enough talk staring at screen. Let me show you how I've got this working. So, quick reminder, what happens when I click send? So, I trigger the search API when I've got no messages in the conversation. I want to run the search only with the first message that's sent. Things to do in Paris in February 2024, for example.
Creating Messages and Handling Data
I then create a new message. And this message is saying, "Write a summary of these web pages based on this statement." And then I pass in the user's query, again, things to do in Paris in February 2024. And then I just list out the content of those pages. I'm going to show you how to do that in a moment. I also mark that as visible to user no, because I'm effectively combining the two messages.
Displaying Results to the User
I've got the message that I send to OpenAI, and I want OpenAI to be aware of, because it's got all of this historical data, all of the important search result data in it. And then I've got create a new message. And the next one is effectively the one that I'm displaying back to the user, visible to user yes, because that's the one that they see.
Example: Querying Specific Information
Let's just give an example of how this works really well, because the Wall of Love, I'm going to just assume that that is something that is event-based, it's going to be there for a certain amount of time. And so I know that my web scraping content that is part of this conversation has got a lot more data about the Wall of Love. So I can say, tell me more about the Wall of Love.
Processing the Response
And so now I'm waiting on OpenAI to respond. Okay, and here we go. I get a response back, and I'm just going to assume that this is all correct. I'm not being too paris. But this is based on the web scrape data that is in the conversation, but it's hidden to the user.
JSON Formatting and Message Handling
I then do the usual thing of sending through all the messages and the correct JSON formatting. If you're jumping into how to use Bubble and OpenAI, this video, do go and check out our other ones, because we do cover all of this. Effectively, what I'm going to be showing in this video is the combination of the three APIs. I then save the response, make it visible to user and assistant, and I reset the input fields.
Using the Brave Search API
So let me just unpack what is going on. The search API is running this custom event. And so I go to Brave to the Brave search API, and you can find all the details on Brave. You can sign up for a free account and start using that API right away. And in my API connector, that looks like this.
API Configuration and Data Retrieval
So I've got my API key in the header. I've got the right key name. I send over my query. This is the one I used to initialize the call, but it's dynamic. And as an example, this is what I get back. And I get back, let's go back to all data. I get back all of this data about the websites.
Extracting Content from Web Pages
In the previous video, where I've been working with the Brave search API, I just used the description field. But now I'm digging into the actual content of each page, because I'm taking the URL that's returned from the top three results, and I'm passing that into the page to API.
Page to API Configuration
One important thing to note with the page to API API is that the API key doesn't go in the header. It goes in the body. So make sure you take it as private, because you don't want your users to access your API key. But what I'm doing is using the API, and we go back to documentation. Yeah, it's all explained here. And we've got previous examples of how to use the page to API.
Extracting Paragraphs from Web Pages
If you've got any questions about that, where am I going? Here we go. So I'm passing in my API key, my URL, I'm saying user real browser, that tends to get better results. And then I pass and I add in a set of instructions here. And this is not really a secret source. It's not perfect. But it does a good job when you combine it with how well the OpenAI API responds to rough bits of data. Because what I'm saying is, create a response that contains something that I've labeled paragraphs. And I'm saying go to the page, find all of the paragraphs, and return them as text.
Cleaning Up HTML and Formatting
So that's going to clear clean up any HTML around the paragraphs. I'm targeting paragraphs because paragraphs is one, it's not perfect, but it's one rough way of, of aiming at the important bits on the page, you know, some like listicle article, 10 things to do in Paris, it's going to the actual content is going to be all in paragraphs. So that's what I'm doing. And I'm getting a response back.
Example of API Response
And I'll give you an example because here's one of the search results. So I can initialize the call. Just take a moment because it's having to actually visit the page. And so I get a list of paragraphs in this JSON array, a list of all of the paragraphs on the page.
Workflow for Multiple API Calls
So going back to my workflow, I could have done this some clever like iteration back in workflow perhaps, but I'm just running three API calls to the page to API. And so I'm saying scrape the brave search results web item one's URL, item two, item three, and then using the new customer events return value. So I've just got site content one, site content two, site content three, and I'm passing the results into each one.
Formatting Data for OpenAI
So this is just going to be a lot of text. There's no HTML mess in it. It's cleaned up, but it's just paragraph, comma, paragraph, comma, paragraph, comma, and I'm returning that in. So then maybe this makes a bit more sense now, which is to say write a summary of these web pages based on the statement, things to do in Paris in February 2024, and then I list all of the paragraphs from all three web pages. And that goes into OpenAI.
Using GPT 3.5 16K Model
I will point out that I had to switch into the GPT 3.5 16K model because the content of all the web pages combined with my prompt, it was about 8,000 tokens. But yeah, then it goes into our pretty standard OpenAI API web API call. And yeah, I think we get a pretty cool result because we get that summary data, and then we can ask specific questions about it.
Testing the System with Real-Time Data
Let's try another one. Here we go. This one might not have come up if we just use the large language model data that is used to train the OpenAI API, because it might only be very recent data that is informing us that there is a sports event on February the 6th, the half marathon judo tournament. So let's say, tell us more about the judo tournament.
Demonstrating Contextual Understanding
Okay, so this is showing the power of all of these APIs combined because it's so vague. But because it's inserted into this conversation with all of the previous messages, it's going to know I'm talking about the judo tournament in Paris. And really, I'm hoping it's going to know that I'm talking in particular about this one. So let's send.
Analyzing the Response
Okay, so we are getting back some data. We can't tell whether this is informed by just OpenAI alone, whether it's informed by the content. I'm going to dare ask a question. Maybe this might undermine a slight bit everything that I've shown you so far in terms of how how awesome it is. But let's just say, what date is it?
Limitations and Future Improvements
Okay, gets again something that I wouldn't expect the OpenAI API to know on its own. Okay, it hasn't got that data. Okay, I'm going to still say this is pretty cool. And there's obviously some improvements that we made along the way. But I've just wanted to show in a very rough sense how we can combine three really powerful APIs.
Conclusion and Summary
We've got the OpenAI text generation with AI API, we've got web scraping with page page two API. And we've got search with the brave search API all pulling in data into our ChatGPT clone conversation.
Can't find what you're looking for?
Search our 300+ Bubble tutorial videos. Start learning no code today!
Flexible Pricing Plans to Fit Your No-Code Journey
Choose the plan that aligns with your goals and start building your startup today.
Have questions?
We have answers!
Find answers to common questions about our membership plans, programs, and more.
Both plans offer full access to our learning resources, community, and support. The Annual plan provides a significant discount (over 15%) compared to paying monthly, and it allows you to lock in your rate for a full year.
Absolutely! You can easily upgrade or downgrade your membership plan at any time by logging into your account and selecting the desired plan. Any unused portion of your current plan will be prorated and applied to your new plan.
As a Planet No Code member, you'll receive a discount on our Bubble coaching sessions. Monthly members receive a 10% discount, while Annual members receive a 17.5% discount. To redeem your discount, simply log into your account and book a coaching session through our platform.
Our 8-week intensive mentorship program is designed to provide personalized guidance and support to help you accelerate your startup journey. You'll be matched with a startup expert who will work with you one-on-one to set goals, overcome challenges, and make rapid progress.
To apply for the Mastery Program, simply click the "Request Invitation" button on our pricing page and fill out the application form. Our team will review your application and schedule a call with you to discuss your goals and determine if the program is a good fit for your needs.
We accept all major credit cards, including Visa, Mastercard, American Express, and Discover.
While we don't offer a free trial, we do provide a 14-day money-back guarantee. If you're not completely satisfied with your membership within the first 14 days, simply contact our support team, and we'll issue a full refund.
If you decide that Planet No Code isn't the right fit for you, you can cancel your membership at any time by logging into your account and navigating to the subscription management page. Click the "Cancel Membership" button, and your membership will be terminated at the end of your current billing cycle.