Unfortunately, a little bit of preparation is required beforehand. Let’s look at an example to illustrate the usage of Selenium. However, Selenium is equally powerful when it comes to automating repetitive web-based tasks. The initial idea behind Selenium, as far as I know, is automated testing. In layman’s term, selenium pretends to be a real user, it opens the browser, “moves” the cursor around and clicks buttons if you tell it to do so. Selenium can be used to automate web browser interaction with Python (also other languages). Most of the time, however, you will have to go through the steps of clicking buttons, scrolling pages, waiting for loads and all of that … or at least you have to make the webpage think you are doing all of that. In which case, it might be possible to directly access the API and fetch the resulting data without having to go through the unnecessary steps in-between. Sometimes the user action might trigger a call to an exposed backend API. Because this happens only after specific user interactions, there are few options when it comes to scraping the data (as those actions do have to take place). The concept of “single-page application” means that the web page itself will change without the user having to reload or getting redirected from page to page all the time. However, most modern web pages are quite interactive.
0 kommentar(er)
