[Solved] scraping with selenium web driver

Question

You should fix your XPath expressions. Use findElement for the first 3. findElements for the last.

To get the home odd :

//td[a[.="bet365"]]/following-sibling::td[span][1]/span

To get the draw odd :

//td[a[.="bet365"]]/following-sibling::td[span][2]/span

To get the away odd :

//td[a[.="bet365"]]/following-sibling::td[span][3]/span

To get them all :

//td[a[.="bet365"]]/following-sibling::td[span]/span

Getting them all is probably better since you call driver.find_elements_by_xpath 1 time. For each match, store each element of the result (a list) in your specific lists (append the first in the home odd list, the second in the draw odd list, the third in the away odd list).

EDIT : To get the opening odds, extract the value from the attribute data-opening-odd. XPath :

//td[a[.="bet365"]]/following-sibling::td[span][1]/@data-opening-odd
//td[a[.="bet365"]]/following-sibling::td[span][2]/@data-opening-odd
//td[a[.="bet365"]]/following-sibling::td[span][3]/@data-opening-odd
//td[a[.="bet365"]]/following-sibling::td[span]/@data-opening-odd

Selenium syntax :

awayodd = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, '//td[a[.="bet365"]]/following-sibling::td[span][3]'))).get_attribute("data-opening-odd")

Accepted Answer