[Solved] How do I convert a web-scraped table into a csv?

by


You Can use pd.read_html for this. 

import pandas as pd

Data = pd.read_html(r'https://www.boxofficemojo.com/chart/top_lifetime_gross/')
for data in Data:
    data.to_csv('Data.csv', ',')

2.Using Bs4

import pandas as pd
from bs4 import BeautifulSoup
import requests

URL = r'https://www.boxofficemojo.com/chart/top_lifetime_gross/'
print('\n>> Exctracting Data using Beautiful Soup for :'+ URL)

try:
    res = requests.get(URL)
except Exception as e:
    print(repr(e))

print('\n<> URL present status Code=",(res.status_code))
soup = BeautifulSoup(res.text,"lxml")
table = soup.find("table')

list_of_rows = []
for row in table.findAll('tr'):
    list_of_cells = []
    for cell in row.findAll(["td"]):
        text = cell.text
        list_of_cells.append(text)
    list_of_rows.append(list_of_cells)

for item in list_of_rows:
    ' '.join(item)

Data = pd.DataFrame(list_of_rows)
Data.dropna(axis = 0, how = 'all',inplace = True)
print(Data.head(10))

Data.to_csv('Table.csv')

2

solved How do I convert a web-scraped table into a csv?