{"id":8903,"date":"2022-09-16T04:42:31","date_gmt":"2022-09-15T23:12:31","guid":{"rendered":"https:\/\/jassweb.com\/solved\/solved-shell-scripting-to-find-the-delimiter\/"},"modified":"2022-09-16T04:42:31","modified_gmt":"2022-09-15T23:12:31","slug":"solved-shell-scripting-to-find-the-delimiter","status":"publish","type":"post","link":"https:\/\/jassweb.com\/solved\/solved-shell-scripting-to-find-the-delimiter\/","title":{"rendered":"[Solved] Shell scripting to find the delimiter"},"content":{"rendered":"

[ad_1]
\n<\/p>\n

\n
\n
<\/div>\n
\n
\n

To count the number of columns with awk you can use the NF<\/code> variable:<\/p>\n

$ cat file\nABC|12345|EAR\nPQRST|123|TWOEYES\nssdf|fdas,sdfsf\n$ awk -F\\| 'NF!=3' file\nssdf|fdas,sdfsf\n<\/code><\/pre>\n
However, this does not seem to cover all the possible ways the data could be corrupted based on the various revisions of the question and the comments. <\/p>\n
A better approach would be to define the exact<\/em> format that the data must follow. For example, assuming that a line is “correct” if it is three columns, with the first and third letters only, and the second numeric, you could write the following script to match all non conforming lines:<\/p>\n
awk -F\\| '!(NF==3 && $1$3 ~ \/^[a-zA-Z]+$\/ && $2+0==$2)' file\n<\/code><\/pre>\n
Test (notice that only the second line (which is conforming) does not get printed):<\/p>\n
$ cat file\nA,BC|12345|EAR\nPQRST|123|TWOEYES\nssdf|fdas,sdfsf\nABC|3983|MAKE,\nsf dl lfsdklf |kldsamfklmadkfmask |mfkmadskfmdslafmka\nABC|abs|EWE\nsdf|123|123\n$ awk -F\\| '!(NF==3&&$1$3~\/^[a-zA-Z]+$\/&&$2+0==$2)' file\nA,BC|12345|EAR\nssdf|fdas,sdfsf\nABC|3983|MAKE,\nsf dl lfsdklf |kldsamfklmadkfmask |mfkmadskfmdslafmka\nABC|abs|EWE\nsdf|123|12\n<\/code><\/pre>\n
You can adapt the above command to your specific needs, based on what you think is a valid input. For example, if you wanted to also restrict the length of each line to 50 characters, you could do<\/p>\n
awk -F\\| '!(NF==3 && $1$3 ~ \/^[a-zA-Z]+$\/ && $2+0==$2 && length($0)<50)' file\n<\/code><\/pre>\n<\/p><\/div>\n
<\/div>\n<\/div>\n
            15<\/span> <\/p><\/div>\n<\/div>\n
[ad_2]<\/p>\n
solved Shell scripting to find the delimiter <\/p>\n","protected":false},"excerpt":{"rendered":"
[ad_1] To count the number of columns with awk you can use the NF variable: $ cat file ABC|12345|EAR PQRST|123|TWOEYES ssdf|fdas,sdfsf $ awk -F\\| ‘NF!=3’ file ssdf|fdas,sdfsf However, this does not seem to cover all the possible ways the data could be corrupted based on the various revisions of the question and the comments. A … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[320],"tags":[445,1042,1043],"class_list":["post-8903","post","type-post","status-publish","format-standard","hentry","category-solved","tag-awk","tag-bash","tag-shell"],"yoast_head":"\n[Solved] Shell scripting to find the delimiter - JassWeb<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/jassweb.com\/solved\/solved-shell-scripting-to-find-the-delimiter\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"[Solved] Shell scripting to find the delimiter - JassWeb\" \/>\n<meta property=\"og:description\" content=\"[ad_1] To count the number of columns with awk you can use the NF variable: $ cat file ABC|12345|EAR PQRST|123|TWOEYES ssdf|fdas,sdfsf $ awk -F| 'NF!=3' file ssdf|fdas,sdfsf However, this does not seem to cover all the possible ways the data could be corrupted based on the various revisions of the question and the comments. A ... Read more\" \/>\n<meta property=\"og:url\" content=\"https:\/\/jassweb.com\/solved\/solved-shell-scripting-to-find-the-delimiter\/\" \/>\n<meta property=\"og:site_name\" content=\"JassWeb\" \/>\n<meta property=\"article:published_time\" content=\"2022-09-15T23:12:31+00:00\" \/>\n<meta name=\"author\" content=\"Kirat\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Kirat\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-shell-scripting-to-find-the-delimiter\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-shell-scripting-to-find-the-delimiter\\\/\"},\"author\":{\"name\":\"Kirat\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#\\\/schema\\\/person\\\/65c9c7b7958150c0dc8371fa35dd7c31\"},\"headline\":\"[Solved] Shell scripting to find the delimiter\",\"datePublished\":\"2022-09-15T23:12:31+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-shell-scripting-to-find-the-delimiter\\\/\"},\"wordCount\":158,\"publisher\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#organization\"},\"keywords\":[\"awk\",\"bash\",\"shell\"],\"articleSection\":[\"Solved\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-shell-scripting-to-find-the-delimiter\\\/\",\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-shell-scripting-to-find-the-delimiter\\\/\",\"name\":\"[Solved] Shell scripting to find the delimiter - JassWeb\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#website\"},\"datePublished\":\"2022-09-15T23:12:31+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-shell-scripting-to-find-the-delimiter\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-shell-scripting-to-find-the-delimiter\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-shell-scripting-to-find-the-delimiter\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"[Solved] Shell scripting to find the delimiter\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#website\",\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/\",\"name\":\"JassWeb\",\"description\":\"Build High-quality Websites\",\"publisher\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#organization\",\"name\":\"Jass Web\",\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/jassweb.com\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/jass-website-logo-1.png\",\"contentUrl\":\"https:\\\/\\\/jassweb.com\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/jass-website-logo-1.png\",\"width\":693,\"height\":132,\"caption\":\"Jass Web\"},\"image\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#\\\/schema\\\/person\\\/65c9c7b7958150c0dc8371fa35dd7c31\",\"name\":\"Kirat\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/wp-content\\\/litespeed\\\/avatar\\\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777008400\",\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/wp-content\\\/litespeed\\\/avatar\\\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777008400\",\"contentUrl\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/wp-content\\\/litespeed\\\/avatar\\\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777008400\",\"caption\":\"Kirat\"},\"sameAs\":[\"http:\\\/\\\/jassweb.com\"],\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/author\\\/jaspritsinghghumangmail-com\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"[Solved] Shell scripting to find the delimiter - JassWeb","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/jassweb.com\/solved\/solved-shell-scripting-to-find-the-delimiter\/","og_locale":"en_US","og_type":"article","og_title":"[Solved] Shell scripting to find the delimiter - JassWeb","og_description":"[ad_1] To count the number of columns with awk you can use the NF variable: $ cat file ABC|12345|EAR PQRST|123|TWOEYES ssdf|fdas,sdfsf $ awk -F| 'NF!=3' file ssdf|fdas,sdfsf However, this does not seem to cover all the possible ways the data could be corrupted based on the various revisions of the question and the comments. A ... Read more","og_url":"https:\/\/jassweb.com\/solved\/solved-shell-scripting-to-find-the-delimiter\/","og_site_name":"JassWeb","article_published_time":"2022-09-15T23:12:31+00:00","author":"Kirat","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Kirat","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/jassweb.com\/solved\/solved-shell-scripting-to-find-the-delimiter\/#article","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/solved-shell-scripting-to-find-the-delimiter\/"},"author":{"name":"Kirat","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31"},"headline":"[Solved] Shell scripting to find the delimiter","datePublished":"2022-09-15T23:12:31+00:00","mainEntityOfPage":{"@id":"https:\/\/jassweb.com\/solved\/solved-shell-scripting-to-find-the-delimiter\/"},"wordCount":158,"publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"keywords":["awk","bash","shell"],"articleSection":["Solved"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/jassweb.com\/solved\/solved-shell-scripting-to-find-the-delimiter\/","url":"https:\/\/jassweb.com\/solved\/solved-shell-scripting-to-find-the-delimiter\/","name":"[Solved] Shell scripting to find the delimiter - JassWeb","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/#website"},"datePublished":"2022-09-15T23:12:31+00:00","breadcrumb":{"@id":"https:\/\/jassweb.com\/solved\/solved-shell-scripting-to-find-the-delimiter\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/jassweb.com\/solved\/solved-shell-scripting-to-find-the-delimiter\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/jassweb.com\/solved\/solved-shell-scripting-to-find-the-delimiter\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/jassweb.com\/solved\/"},{"@type":"ListItem","position":2,"name":"[Solved] Shell scripting to find the delimiter"}]},{"@type":"WebSite","@id":"https:\/\/jassweb.com\/solved\/#website","url":"https:\/\/jassweb.com\/solved\/","name":"JassWeb","description":"Build High-quality Websites","publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/jassweb.com\/solved\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/jassweb.com\/solved\/#organization","name":"Jass Web","url":"https:\/\/jassweb.com\/solved\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/","url":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","contentUrl":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","width":693,"height":132,"caption":"Jass Web"},"image":{"@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31","name":"Kirat","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777008400","url":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777008400","contentUrl":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777008400","caption":"Kirat"},"sameAs":["http:\/\/jassweb.com"],"url":"https:\/\/jassweb.com\/solved\/author\/jaspritsinghghumangmail-com\/"}]}},"_links":{"self":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/8903","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/comments?post=8903"}],"version-history":[{"count":0,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/8903\/revisions"}],"wp:attachment":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/media?parent=8903"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/categories?post=8903"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/tags?post=8903"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}