You can use pandas to do that. Collect all the data into a dataframe, then just write the dataframe to file.
import pandas as pd
import requests
import bs4
root_url="https://www.estatesales.net"
url_list=['https://www.estatesales.net/companies/NJ/Northern-New-Jersey']
results = pd.DataFrame()
for url in url_list:
response = requests.get(url)
soup = bs4.BeautifulSoup(response.text, 'html.parser')
companies = soup.find_all('app-company-city-view-row')
for company in companies:
try:
link = root_url + company.find('a',{'itemprop':"name url"})['href']
print(link)
except:
continue
response = requests.get(link)
soup = bs4.BeautifulSoup(response.text, 'html.parser')
company_info = soup.find('div', {'id':'CompanyInfo'})
try:
name = company_info.find('h1',{'itemprop':"name"}).text
except:
name="N/A"
try:
city = company_info.find('span',{'itemprop':"addressLocality"}).text
except:
city = 'N/A'
try:
state = company_info.find('span',{'itemprop':"addressRegion"}).text
except:
state="N/A"
try:
phone = company_info.find('span',{'itemprop':"telephone"}).text
except:
phone="N/A"
temp_df = pd.DataFrame([[name, city, state, phone]], columns = ['Name','City','State','Telephone'])
results = results.append(temp_df).reset_index(drop=True)
results.to_csv('file.csv', index=False)
Output:
print (results)
Name ... Telephone
0 Hub Estate Liquidation ... (862) 259-5364
1 Pink Dog Estate And Moving Sales, LLC ... (201) 674-7464
2 Remember When Antiques And Estate Sales, LLC ... (917) 410-7100
3 Always Nostalgia ... (201) 388-2598
4 Before & After Corp ... (201) 747-5342
5 Discovery Estate Sales ... (908) 620-1776
6 Plum Cottage Estate Sales & Appraisals ... (732) 788-4101
7 Decorate On A Dime, LLC ... (908) 380-3340
8 Remmey Antiques & Fine Art Appraisers & Auctio... ... (973) 425-1608
9 Easy Picking Estate Sales ... (917) 691-6132
10 EstateSalesByOlga.com ... (908) 337-4240
11 Real McCoy ... (973) 418-1286
12 Then And Now Estate Sales ... (201) 259-8408
13 Insideout,llc ... (215) 630-4942
14 Beacon Hill Estate Sales & Appraisals LLC ... (908) 601-5381
15 Lori Palmer Estate Sales ... (732) 809-3382
16 Somerset Appraisal And Estate Services ... (908) 872-6236
17 Central Jersey Estate Sales & Liquidators ... (908) 625-1622
18 C. T. Peters Inc., Appraisers ... (732) 747-9450
19 Treasures Of Yesterday Estate Sales ... (201) 446-2790
20 Caring Transitions Of Central Jersey ... (732) 307-3881
21 Attic To The Basement Estate Sales ... (732) 778-7674
22 Griffin Estate Sales ... (908) 447-3044
23 Dodge Estate Sales LLC ... (973) 714-1401
24 Estate Sales By Kathy ... (732) 674-7330
25 Curated Estates ... (917) 470-9255
26 Brownstone Liquidators ... (845) 821-3254
27 Jersey Estate Sales ... (973) 428-1906
28 Arkay Resale, Inc. ... (201) 741-4165
29 Vinylpiggy's LLC ... (551) 804-7152
.. ... ... ...
116 Classic Estate Sales & Appraisals ... (201) 370-4021
117 Sullivintage ... (732) 890-3485
118 A Trotters Sale ... (973) 819-8685
119 Shore Estate Sales ... (732) 616-3371
120 Liberty Antiques ... (908) 581-6987
121 Johnson Estate Sales ... (201) 259-0442
122 Buy-Gone Trading Co. ... (201) 665-8208
123 Elite Auctions & Estate Sales ... (732) 751-1112
124 The Butler Did It ... (908) 892-8133
125 Drama Mama Home Estate Sales ... (862) 400-2081
126 New Jersey Gold Resources ... N/A
127 Lyrix Inc ... (973) 632-1600
128 Home Ready Services, LLC ... (908) 370-3062
129 Bygone Days Estate Sales, LLC ... (973) 857-9069
130 Fourty Fifty Sixty ... (973) 341-7891
131 True Salvage ... (973) 970-5400
132 Ina's Antiques & Estate Sales ... (908) 578-4118
133 Caring Transitions Jersey Shore ... (732) 681-0054
134 ENCORE ESTATE & TAG SALES ... (973) 220-4611
135 Ajtrains ... (732) 859-1606
136 M.T. House Estate & Moving Sales ... (973) 865-1173
137 WCL Antiques ... (201) 739-3173
138 Kens Antiques ... (732) 306-2717
139 Lee Dowdy Antiques ... (201) 650-7208
140 Garage Sale Goddesses ... (201) 612-8510
141 M & J Estate Sales ... (908) 956-4284
142 Granny's Attic ... (201) 632-0102
143 Red Barn Estate sales ... (201) 481-5428
144 D&r Estate & Tag Sales ... (201) 573-1009
145 Always Remember When LLC ... (347) 244-1591
[146 rows x 4 columns]
solved organizing data that I am pulling and saving to CSV