[Solved] How to read, edit, merge and save all csv files from one folder?

by


This should be fairly easy with something like the csv library, if you don’t want to get into learning dataframes.

import os
import csv

new_data = []
for filename in os.listdir('./csv_dir'):
    if filename.endswith('.csv'):
        with open('./csv_dir/' + filename, mode="r") as curr_file:
            reader = csv.reader(curr_file, delimiter=",")
            for row in reader:
                new_data.append(row[2]) # Or whichever column you need

with open('./out_dir/output.txt', mode="w") as out_file:
    for row in new_data:
        out_file.write('{}\n'.format(row))

Your new_data will contain the 2000 * 1123 columns.

This may not be the most efficient way to do this, but it’ll get the job done and grab each CSV. You’ll need to do the work of making sure the CSV files have the correct structure, or adding in checks in the code for validating the columns before appending to new_data.

6

solved How to read, edit, merge and save all csv files from one folder?