{"id":9592,"date":"2022-09-19T17:49:47","date_gmt":"2022-09-19T12:19:47","guid":{"rendered":"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/"},"modified":"2022-09-19T17:49:47","modified_gmt":"2022-09-19T12:19:47","slug":"solved-converting-a-massive-json-file-to-csv","status":"publish","type":"post","link":"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/","title":{"rendered":"[Solved] Converting a massive JSON file to CSV"},"content":{"rendered":"

[ad_1]
\n<\/p>\n

\n
\n
<\/div>\n
\n
\n

A 48mb json file is pretty small. You should be able to load the data into memory using something like this <\/p>\n

import json\n\nwith open('data.json') as data_file:    \n    data = json.load(data_file)\n<\/code><\/pre>\n
Dependending on how you wrote to the json file, data may be a list, which contains many dictionaries. Try running:<\/p>\n
type(data)\n<\/code><\/pre>\n
If the type is a list, then iterate over each element and inspect it. For instance:<\/p>\n
for row in data:\n    print(type(row))\n    # print(row.keys())\n<\/code><\/pre>\n
If row is a dict instance, then inspect the keys and within the loop, start building up what each row of the CSV should contain, then you can either use pandas, the csv module or just open a file and write line by line with commas yourself. <\/p>\n
So maybe something like:<\/p>\n
import json\n\nwith open('data.json') as data_file:    \n    data = json.load(data_file)\n\n\nwith open('some_file.txt', 'w') as f:\n\n    for row in data:\n        user = row['username']\n        text = row['tweet_text']\n        created = row['timestamp']\n        joined = \",\".join([user, text, created])\n        f.write(joined)\n<\/code><\/pre>\n
You may still run into issues with unicode characters, commas within your data, etc…but this is a general guide. <\/p>\n<\/p><\/div>\n
<\/div>\n<\/div>\n
            1<\/span> <\/p><\/div>\n<\/div>\n
[ad_2]<\/p>\n
solved Converting a massive JSON file to CSV <\/p>\n","protected":false},"excerpt":{"rendered":"
[ad_1] A 48mb json file is pretty small. You should be able to load the data into memory using something like this import json with open(‘data.json’) as data_file: data = json.load(data_file) Dependending on how you wrote to the json file, data may be a list, which contains many dictionaries. Try running: type(data) If the type … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[320],"tags":[483,356,341],"class_list":["post-9592","post","type-post","status-publish","format-standard","hentry","category-solved","tag-csv","tag-json","tag-sql"],"yoast_head":"\n[Solved] Converting a massive JSON file to CSV - JassWeb<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"[Solved] Converting a massive JSON file to CSV - JassWeb\" \/>\n<meta property=\"og:description\" content=\"[ad_1] A 48mb json file is pretty small. You should be able to load the data into memory using something like this import json with open('data.json') as data_file: data = json.load(data_file) Dependending on how you wrote to the json file, data may be a list, which contains many dictionaries. Try running: type(data) If the type ... Read more\" \/>\n<meta property=\"og:url\" content=\"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/\" \/>\n<meta property=\"og:site_name\" content=\"JassWeb\" \/>\n<meta property=\"article:published_time\" content=\"2022-09-19T12:19:47+00:00\" \/>\n<meta name=\"author\" content=\"Kirat\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Kirat\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/\"},\"author\":{\"name\":\"Kirat\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31\"},\"headline\":\"[Solved] Converting a massive JSON file to CSV\",\"datePublished\":\"2022-09-19T12:19:47+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/\"},\"wordCount\":146,\"publisher\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\"},\"keywords\":[\"csv\",\"json\",\"sql\"],\"articleSection\":[\"Solved\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/\",\"url\":\"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/\",\"name\":\"[Solved] Converting a massive JSON file to CSV - JassWeb\",\"isPartOf\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#website\"},\"datePublished\":\"2022-09-19T12:19:47+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/jassweb.com\/solved\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"[Solved] Converting a massive JSON file to CSV\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/jassweb.com\/solved\/#website\",\"url\":\"https:\/\/jassweb.com\/solved\/\",\"name\":\"JassWeb\",\"description\":\"Build High-quality Websites\",\"publisher\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/jassweb.com\/solved\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\",\"name\":\"Jass Web\",\"url\":\"https:\/\/jassweb.com\/solved\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png\",\"contentUrl\":\"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png\",\"width\":693,\"height\":132,\"caption\":\"Jass Web\"},\"image\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31\",\"name\":\"Kirat\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586\",\"contentUrl\":\"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586\",\"caption\":\"Kirat\"},\"sameAs\":[\"http:\/\/jassweb.com\"],\"url\":\"https:\/\/jassweb.com\/solved\/author\/jaspritsinghghumangmail-com\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"[Solved] Converting a massive JSON file to CSV - JassWeb","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/","og_locale":"en_US","og_type":"article","og_title":"[Solved] Converting a massive JSON file to CSV - JassWeb","og_description":"[ad_1] A 48mb json file is pretty small. You should be able to load the data into memory using something like this import json with open('data.json') as data_file: data = json.load(data_file) Dependending on how you wrote to the json file, data may be a list, which contains many dictionaries. Try running: type(data) If the type ... Read more","og_url":"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/","og_site_name":"JassWeb","article_published_time":"2022-09-19T12:19:47+00:00","author":"Kirat","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Kirat","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/#article","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/"},"author":{"name":"Kirat","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31"},"headline":"[Solved] Converting a massive JSON file to CSV","datePublished":"2022-09-19T12:19:47+00:00","mainEntityOfPage":{"@id":"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/"},"wordCount":146,"publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"keywords":["csv","json","sql"],"articleSection":["Solved"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/","url":"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/","name":"[Solved] Converting a massive JSON file to CSV - JassWeb","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/#website"},"datePublished":"2022-09-19T12:19:47+00:00","breadcrumb":{"@id":"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/jassweb.com\/solved\/solved-converting-a-massive-json-file-to-csv\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/jassweb.com\/solved\/"},{"@type":"ListItem","position":2,"name":"[Solved] Converting a massive JSON file to CSV"}]},{"@type":"WebSite","@id":"https:\/\/jassweb.com\/solved\/#website","url":"https:\/\/jassweb.com\/solved\/","name":"JassWeb","description":"Build High-quality Websites","publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/jassweb.com\/solved\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/jassweb.com\/solved\/#organization","name":"Jass Web","url":"https:\/\/jassweb.com\/solved\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/","url":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","contentUrl":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","width":693,"height":132,"caption":"Jass Web"},"image":{"@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31","name":"Kirat","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/image\/","url":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586","contentUrl":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586","caption":"Kirat"},"sameAs":["http:\/\/jassweb.com"],"url":"https:\/\/jassweb.com\/solved\/author\/jaspritsinghghumangmail-com\/"}]}},"_links":{"self":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/9592","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/comments?post=9592"}],"version-history":[{"count":0,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/9592\/revisions"}],"wp:attachment":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/media?parent=9592"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/categories?post=9592"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/tags?post=9592"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}