selenium.ipynb

Kernel: Python 3

Selenium

Allow the remote contro of a browser, in this case firefox

See the Manual

! apt install firefox  xvfb > /dev/null

! pip3 install  pyvirtualdisplay selenium webdriver_manager  > /dev/null

Side by side mode

The default mode is that a new browser window is open from your, in this case, Python execution enviroment. The code window is the remote control of that specific browser window.

from webdriver_manager.firefox import GeckoDriverManager
from selenium import webdriver
    

browser = webdriver.Firefox(executable_path=
                GeckoDriverManager().install())

Simple example

browser.get('http://fisica.udea.edu.co')

browser.find_element_by_link_text('Instituto').click() 

Capture the last visited page

html=browser.page_source

browser.close()

Google search example

browser.get('http://google.com')

Find the id element by putting the mouse pointer on the search box.

browser.find_element_by_id('lst-ib').clear()
browser.find_element_by_id('lst-ib').send_keys('hello world')

With the mouse pointer on the search button, right click and choose Inspect and check the button name

browser.find_element_by_name("btnK").click()

html=browser.page_source

Hidden mode

A virtual display is generated where the broeser window is hidden

from webdriver_manager.firefox import GeckoDriverManager
from selenium import webdriver
from pyvirtualdisplay import Display

REMOTE=True
if REMOTE:
    display = Display(visible=0, size=(800, 600))
    display.start()
    
try:
    browser = webdriver.Firefox(executable_path=
                '{}/.wdm/geckodriver/v0.23.0/linux64/geckodriver'.format(home))
except WebDriverException:
    browser = webdriver.Firefox(executable_path=
                GeckoDriverManager().install())

if REMOTE:
    #https://stackoverflow.com/a/17536547/2268280
    browser.set_page_load_timeout(30)
    
browser.get('http://fisica.udea.edu.co')

browser.find_element_by_link_text('Instituto').click() 

browser.close()