beautifulsoup

[Solved] Scraping data from a dynamic web database with Python [closed]

by

You can solve it with requests (for maintaining a web-scraping session) + BeautifulSoup (for HTML parsing) + regex for extracting a value of a javascript variable containing the desired data inside a script tag and ast.literal_eval() for making a python list out of js list: from ast import literal_eval import re from bs4 import BeautifulSoup … Read more

[Solved] Unable to capture records name , price and rating and image in Requests Python [closed]

by

You have to scrape the adidas site and use regex: import requests import re endpoint = “https://www.adidas.com.au/continental-80-shoes/G27707.html” headers = { ‘User-Agent’: ‘Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36’ } response = requests.get(endpoint, headers = headers) data = response.text pricereg = r”(?<=\”price\”:)(.*)(?=,\”)” namereg = r”(?<=\”name\”:)(.*)(?=,\”co)” ratingreg= r”(?<=\”ratingValue\”:)(.*)(?=,\”reviewCou)” price = re.search(pricereg, data, re.MULTILINE).group() name … Read more

[Solved] Python: Get data BeautifulSoup

by

You should use the bs4.Tag.find_all method or something similar. soup.find_all(attrs={“face”:”arial”,”font-size”:”16px”,”color”:”navy”}) Example: >>>import bs4 >>>html=””‘<div id=”accounts” class=”elementoOculto”> <table align=”center” border=”0″ cellspacing=0 width=”90%”> <tr><th align=”left” colspan=2> permisos </th></tr><tr> <td colspan=2> <table width=100% align=center border=0 cellspacing=1> <tr> <th align=center width=”20%”>cuen</th> <th align=center>Mods</th> </tr> </table> </td> </tr> </table> <table align=”center” border=”0″ cellspacing=1 width=”90%”> <tr bgcolor=”whitesmoke” height=”08″> <td align=”left” width=”20%”> … Read more