[Solved] Interacting with a website and getting data using python

by


You can use BeautifulSoup, i.e.:

import requests, traceback
from bs4 import BeautifulSoup

domains = ["duckduckgo.com", "opensource.com"]
for dom in domains:
    try:
        req = requests.get(f"https://fortiguard.com/webfilter?q={dom}&version=8")
        if req.status_code == 200:
            soup = BeautifulSoup(req.text, 'html.parser')
            cat = soup.find("meta",  property="description")["content"].split(":")[1].strip()
            print(dom, cat)
    except:
        pass
        print(traceback.format_exc())

Output:

duckduckgo.com Search Engines and Portals
opensource.com Information Technology

Demo

2

solved Interacting with a website and getting data using python