{"id":30380,"date":"2023-01-14T18:07:23","date_gmt":"2023-01-14T12:37:23","guid":{"rendered":"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/"},"modified":"2023-01-14T18:07:23","modified_gmt":"2023-01-14T12:37:23","slug":"solved-converting-txt-spark-output-to-csv","status":"publish","type":"post","link":"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/","title":{"rendered":"[Solved] Converting .txt Spark Output to .csv"},"content":{"rendered":"<p> [ad_1]<br \/>\n<\/p>\n<div id=\"answer-52985835\" class=\"answer js-answer accepted-answer js-accepted-answer\" data-answerid=\"52985835\" data-parentid=\"52982929\" data-score=\"0\" data-position-on-page=\"1\" data-highest-scored=\"1\" data-question-has-accepted-highest-score=\"1\" itemprop=\"acceptedAnswer\" itemscope itemtype=\"https:\/\/schema.org\/Answer\">\n<div class=\"post-layout\">\n<div class=\"votecell post-layout--left\"><\/div>\n<div class=\"answercell post-layout--right\">\n<div class=\"s-prose js-post-body\" itemprop=\"text\">\n<p>This is a simple approach that converts the txt output data into a data structure (that can easily be written into a csv file).<\/p>\n<p>The basic idea is using data structures along with the amount of headers \/ columns in order to parse entry sets from the one liner txt output.<\/p>\n<p>Have a look at the code comments, every <em>&#8220;<strong>TODO<\/strong> 4 U&#8221;<\/em> means work for you, mostly because I cannot really guess what you need to do at those positions in the code (like how to get the headers).<\/p>\n<blockquote>\n<p><em>This is just a main method that does its work straight forward. You may want to understand what it does and apply changes that make the code meet your requiremtens. Input and output are just <code>String<\/code>s that you have to create, receive or process yourself.<\/em><\/p>\n<\/blockquote>\n<pre><code>public static void main(String[] args) {\n\n    \/\/ TODO 4 U: get the values for the header somehow\n    String headerLine = \"NAME, UNIV, DEGREE, COURSE\";\n\n    \/\/ TODO 4 U: read the txt output\n    String txtOutput = \"John MIT Bachelor ComputerScience Mike UB Master ComputerScience\";\n\n    \/*\n     * then split the header line\n     * (or do anything similar, I don't know where your header comes from)\n     *\/\n    String[] headers = headerLine.split(\", \");\n\n    \/\/ store the amount of headers, which is the amount of columns\n    int amountOfColumns = headers.length;\n\n    \/\/ split txt output data by space\n    String[] data = txtOutput.split(\" \");\n\n    \/*\n     * declare a data structure that stores lists of Strings,\n     * each one is representing a line of the csv file\n     *\/\n    Map&lt;Integer, List&lt;String&gt;&gt; linesForCsv = new TreeMap&lt;Integer, List&lt;String&gt;&gt;();\n\n    \/\/ get the length of the txt output data\n    int a = data.length;\n\n    \/\/ create a list of Strings containing the headers and put it into the data structure\n    List&lt;String&gt; columnHeaders = Arrays.asList(headers);\n    linesForCsv.put(0, columnHeaders);\n\n    \/\/ declare a line counter for the csv file\n    int l = 0;\n    \/\/ go through the txt output data in order to get the lines for the csv file\n    for (int i = 0; i &lt; a; i++) {\n        \/\/ check if there is a new line to be created\n        if (i % amountOfColumns == 0) {\n            \/*\n             * every time the amount of headers is reached,\n             * create a new list for a new line in the csv file\n             *\/\n            l++; \/\/ increment the line counter (even at 0 because the header row is inserted at 0)\n            linesForCsv.put(l, new ArrayList&lt;String&gt;()); \/\/ create a new line-list\n            linesForCsv.get(l).add(data[i]); \/\/ add the data to the line-list\n        } else {\n            \/\/ if there is no new line to be created, store the data in the current one\n            linesForCsv.get(l).add(data[i]);\n        }\n    }\n\n    \/\/ print the lines stored in the map\n    \/\/ TODO 4 U: write this to a csv file instead of just printing it to the console\n    linesForCsv.forEach((lineNumber, line) -&gt; {\n        System.out.println(\"Line \" + lineNumber + \": \" + String.join(\",\", line));\n    });\n}\n<\/code><\/pre>\n<\/p><\/div>\n<div class=\"mt24\"><\/div>\n<\/div>\n<p>            <span class=\"d-none\" itemprop=\"commentCount\">0<\/span> <\/p><\/div>\n<\/div>\n<p>[ad_2]<\/p>\n<p>solved Converting .txt Spark Output to .csv <\/p>\n","protected":false},"excerpt":{"rendered":"<p>[ad_1] This is a simple approach that converts the txt output data into a data structure (that can easily be written into a csv file). The basic idea is using data structures along with the amount of headers \/ columns in order to parse entry sets from the one liner txt output. Have a look &#8230; <a title=\"[Solved] Converting .txt Spark Output to .csv\" class=\"read-more\" href=\"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/\" aria-label=\"More on [Solved] Converting .txt Spark Output to .csv\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[320],"tags":[803,5740,323,3298],"class_list":["post-30380","post","type-post","status-publish","format-standard","hentry","category-solved","tag-apache-spark","tag-apache-spark-dataset","tag-java","tag-rdd"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>[Solved] Converting .txt Spark Output to .csv - JassWeb<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"[Solved] Converting .txt Spark Output to .csv - JassWeb\" \/>\n<meta property=\"og:description\" content=\"[ad_1] This is a simple approach that converts the txt output data into a data structure (that can easily be written into a csv file). The basic idea is using data structures along with the amount of headers \/ columns in order to parse entry sets from the one liner txt output. Have a look ... Read more\" \/>\n<meta property=\"og:url\" content=\"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/\" \/>\n<meta property=\"og:site_name\" content=\"JassWeb\" \/>\n<meta property=\"article:published_time\" content=\"2023-01-14T12:37:23+00:00\" \/>\n<meta name=\"author\" content=\"Kirat\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Kirat\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/\"},\"author\":{\"name\":\"Kirat\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31\"},\"headline\":\"[Solved] Converting .txt Spark Output to .csv\",\"datePublished\":\"2023-01-14T12:37:23+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/\"},\"wordCount\":148,\"publisher\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\"},\"keywords\":[\"apache-spark\",\"apache-spark-dataset\",\"java\",\"rdd\"],\"articleSection\":[\"Solved\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/\",\"url\":\"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/\",\"name\":\"[Solved] Converting .txt Spark Output to .csv - JassWeb\",\"isPartOf\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#website\"},\"datePublished\":\"2023-01-14T12:37:23+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/jassweb.com\/solved\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"[Solved] Converting .txt Spark Output to .csv\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/jassweb.com\/solved\/#website\",\"url\":\"https:\/\/jassweb.com\/solved\/\",\"name\":\"JassWeb\",\"description\":\"Build High-quality Websites\",\"publisher\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/jassweb.com\/solved\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\",\"name\":\"Jass Web\",\"url\":\"https:\/\/jassweb.com\/solved\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png\",\"contentUrl\":\"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png\",\"width\":693,\"height\":132,\"caption\":\"Jass Web\"},\"image\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31\",\"name\":\"Kirat\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586\",\"contentUrl\":\"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586\",\"caption\":\"Kirat\"},\"sameAs\":[\"http:\/\/jassweb.com\"],\"url\":\"https:\/\/jassweb.com\/solved\/author\/jaspritsinghghumangmail-com\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"[Solved] Converting .txt Spark Output to .csv - JassWeb","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/","og_locale":"en_US","og_type":"article","og_title":"[Solved] Converting .txt Spark Output to .csv - JassWeb","og_description":"[ad_1] This is a simple approach that converts the txt output data into a data structure (that can easily be written into a csv file). The basic idea is using data structures along with the amount of headers \/ columns in order to parse entry sets from the one liner txt output. Have a look ... Read more","og_url":"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/","og_site_name":"JassWeb","article_published_time":"2023-01-14T12:37:23+00:00","author":"Kirat","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Kirat","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/#article","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/"},"author":{"name":"Kirat","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31"},"headline":"[Solved] Converting .txt Spark Output to .csv","datePublished":"2023-01-14T12:37:23+00:00","mainEntityOfPage":{"@id":"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/"},"wordCount":148,"publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"keywords":["apache-spark","apache-spark-dataset","java","rdd"],"articleSection":["Solved"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/","url":"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/","name":"[Solved] Converting .txt Spark Output to .csv - JassWeb","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/#website"},"datePublished":"2023-01-14T12:37:23+00:00","breadcrumb":{"@id":"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/jassweb.com\/solved\/solved-converting-txt-spark-output-to-csv\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/jassweb.com\/solved\/"},{"@type":"ListItem","position":2,"name":"[Solved] Converting .txt Spark Output to .csv"}]},{"@type":"WebSite","@id":"https:\/\/jassweb.com\/solved\/#website","url":"https:\/\/jassweb.com\/solved\/","name":"JassWeb","description":"Build High-quality Websites","publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/jassweb.com\/solved\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/jassweb.com\/solved\/#organization","name":"Jass Web","url":"https:\/\/jassweb.com\/solved\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/","url":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","contentUrl":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","width":693,"height":132,"caption":"Jass Web"},"image":{"@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31","name":"Kirat","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/image\/","url":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586","contentUrl":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586","caption":"Kirat"},"sameAs":["http:\/\/jassweb.com"],"url":"https:\/\/jassweb.com\/solved\/author\/jaspritsinghghumangmail-com\/"}]}},"_links":{"self":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/30380","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/comments?post=30380"}],"version-history":[{"count":0,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/30380\/revisions"}],"wp:attachment":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/media?parent=30380"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/categories?post=30380"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/tags?post=30380"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}