Web scraping with Page2API Premium Proxy

Introduction to Web Scraping with Page2API Premium Proxy

Welcome to part two of a mini series showing how to get a user's country code in Bubble, save it to the database that's in the previous video and now we're going to use the country code to take advantage of the premium proxy with Page2API. This will allow us to scrape data from a website, take for example Amazon no matter which Amazon domain like if it's Amazon.com or Amazon.co.uk Amazon detects the location of the user as they visit the page and will adjust the information on it according to the location that they're judging the users from based on their IP address. But what we want to do is ensure that say a user is in Germany and they present us with a Amazon URL that they get back the data from the product page that Amazon presents to German users. So let's get into it.

Setting Up the API Connector

We've got a simple page here that saves the country code to the current user and we need to make adjustments to our API connector with Page2API. Now if this all looks a bit alien to you check out an earlier video where I go through every step that's needed in creating this API call for Page2API because right now I'm going to be just referring to the documentation so in order to make a API call with the premium proxy faking you know that the visitor the web scraper is from another country I need to add this line into my JSON here making sure that all the JSON syntax is correct like a comma afterwards and then I change this into a dynamic value and in here is where I can if I am tick private it lets me put it into a workflow and so I could put like DE in here and I don't know what's going to happen BBC.co.uk if you visit and beat and the BBC website thinks that you're in Germany let's have a look oh okay there we go pre-inbox is only available for paid accounts.

Implementing the Workflow

Now I'm creating this video because I literally just got off a call with one of our Plant No Code members and they were facing this exact conundrum and they were trying to scrape data from Amazon but what I'll just show you is how to set this up because I know that all of this is correct is I'll show you how to set this up in the workflow so if I go back onto my page here and then let's put in an input okay this is going to be my URL and then instead of this saving the country code which I demonstrated in the previous video I'm going to use it to scrape the data. So go into plugin and scrape and some reason it's not showing up the country code Oh private there we go I just had to re-add it behind the scenes simply because I'm not going to upgrade to a paid account right now anyway what you're seeing on screen now it is going to work there we go country code and so now I go current user country code.

Running the Workflow

Okay and when I run that workflow by the click of the button it's going to dynamically fill in the URL and the country code and it will then Page2API is going to go to the URL I provide under the disguise of coming from the country code that I provide and then it's going to return the values that I've requested that it scrapes.

Important Considerations

There's one thing to highlight where you're likely to get this wrong get frustrated that two things is you need a paid account in order to do the premium proxy I haven't checked that out ahead of time and then also make sure that your country codes are coming back in this format which is lowercase I tried uppercase with the client because sorry with our Plant No Code member because that's the way that the plugin returns to country code but it needs to be lowercase because that is what Page2API is expecting.