Web scraping the New York Times with Browserbear
In this Bubble tutorial we demonstrate how screenshot the homepage of the New York Times using the Browserbear API.
Unlock web scraping power: Learn how to capture screenshots of any website with BrowserBear and Bubble!
No-code magic revealed: Discover how to integrate API connectors seamlessly into your Bubble app for advanced functionality.
Boost your Bubble skills: Master the art of using webhooks and handling API responses in this must-watch tutorial!
Introduction to BrowserBear
BrowserBear is a fantastic web scraping tool with a really easy but versatile API. In this Bubble tutorial video, I'm going to show you how you can add web scraping into your Bubble app. In particular, we're going to focus on how to take a screenshot. We'll be taking a screenshot of the New York Times homepage and showing how we can get that data back into Bubble.
Accessing Planet No Code Tutorials
Before I launch into that, if you're learning Bubble (which you probably are because you're watching this video), we've got hundreds of Bubble tutorial videos just like this one. Many of these can only be accessed by our members on our website planetnocode.com. Let's dive into BrowserBear.
Setting Up BrowserBear Task
I've signed up in BrowserBear and modified one of their example tasks. I'm saying let's take a screenshot of the New York Times. If I examine the steps here, I've got "go to the homepage of the New York Times" and then I simply say "take a screenshot". I do that by adding in a step.
BrowserBear's Versatility
Just to give you an idea, this is really how versatile BrowserBear is because there's all of these steps that you can add into a task. All you need to do, and I'll be demonstrating that in just a few seconds, is run this task from your Bubble app and it will run the task and return the data to your Bubble app.
Previous AI Features Tutorial
In a previous video, I've demonstrated their AI features, which is a really cool way to set up quick web scraping without having to deconstruct the page. Anyway, we're focusing on screenshots in this tutorial.
Running BrowserBear Task in Bubble
So, how do I run this task in Bubble? I'm in my Bubble API connector and I've set up BrowserBear. I've added in all of the essential fields up here including authorization, bearer, my API key, and my content type. If you're unsure where I've got this data, I've taken it out of the BrowserBear API documentation.
Setting Up the API Call
I'm on "runs, create run" and then I've taken out the essentials here. All I really need to send through to run the task is to send through a webhook because that's where I want to receive the data when BrowserBear has completed its operation.
Creating a Backend Workflow
If I go into a separate Tambam Bubble, I've set up an API workflow as a back-end workflow called BrowserBear. If I click detect data, I get this URL. Going back into my API connector, I'm adding the parameter of webhook URL and I'm pasting my endpoint for my Bubble API back-end workflow into there because that's where I'm going to receive the data when it's taken the screenshot.
Demonstrating the API Connector
I'll be demoing all of this within the API connector. Do check out our other videos such as our OpenAI tutorials where you'll see about how we can save data from an API tool, but this is just the technicality of using the BrowserBear API.
Updating Task ID
The other thing I need to update is my task ID because this is a different task from a previous video. To find my task ID, I can just click up here and click copy, then go back to Bubble, paste my task ID, and I can re-initialize the call.
Handling API Response
I get this instant response back basically because the process of taking a screenshot can take a little bit of time. That's why I have to use a webhook. I don't get the data right away back in the return values for my API call.
Checking BrowserBear Logs
I can go into BrowserBear and go into logs. Here it is. So it's still running. I'm going to go back here because I'm waiting for the data to come in. By putting it into the request Data mode or initialize, Bubble is ready to learn the structure of the content as it returns.
Troubleshooting Connection Error
We've actually got an error now. I should have tested this beforehand. BrowserBear is not able to connect to the New York Times. It's recommending I change a setting. Let's try that one and run the API again.
Reviewing API Call Results
Going into my logs, this is my most recent call. Let's see what happens here. It's now saying finished, so I'm going to go back here and this is the data that's been sent in. I'm looking for the ID of the screenshot, so I'm going to just approve it.
Viewing the Screenshot
I'm going to take this, copy it, open in a new tab, and here is The New York Times. What it's done, obviously, is scroll to get the full page. I've got this Terms or, you know, other websites might have a cookie notice.
Next Steps
I'm going to stop this video here and I'm going to have a go at using some of the additional tools in BrowserBear to try and get rid of that cookie notice or terms notice. I'm going to record a follow-up video to this right away.
Can't find what you're looking for?
Search our 300+ Bubble tutorial videos. Start learning no code today!
Flexible Pricing Plans to Fit Your No-Code Journey
Choose the plan that aligns with your goals and start building your startup today.
Have questions?
We have answers!
Find answers to common questions about our membership plans, programs, and more.
Both plans offer full access to our learning resources, community, and support. The Annual plan provides a significant discount (over 15%) compared to paying monthly, and it allows you to lock in your rate for a full year.
Absolutely! You can easily upgrade or downgrade your membership plan at any time by logging into your account and selecting the desired plan. Any unused portion of your current plan will be prorated and applied to your new plan.
As a Planet No Code member, you'll receive a discount on our Bubble coaching sessions. Monthly members receive a 10% discount, while Annual members receive a 17.5% discount. To redeem your discount, simply log into your account and book a coaching session through our platform.
Our 8-week intensive mentorship program is designed to provide personalized guidance and support to help you accelerate your startup journey. You'll be matched with a startup expert who will work with you one-on-one to set goals, overcome challenges, and make rapid progress.
To apply for the Mastery Program, simply click the "Request Invitation" button on our pricing page and fill out the application form. Our team will review your application and schedule a call with you to discuss your goals and determine if the program is a good fit for your needs.
We accept all major credit cards, including Visa, Mastercard, American Express, and Discover.
While we don't offer a free trial, we do provide a 14-day money-back guarantee. If you're not completely satisfied with your membership within the first 14 days, simply contact our support team, and we'll issue a full refund.
If you decide that Planet No Code isn't the right fit for you, you can cancel your membership at any time by logging into your account and navigating to the subscription management page. Click the "Cancel Membership" button, and your membership will be terminated at the end of your current billing cycle.