{"id":9801,"date":"2022-09-20T19:48:01","date_gmt":"2022-09-20T14:18:01","guid":{"rendered":"https:\/\/jassweb.com\/solved\/solved-r-computing-not-so-fast\/"},"modified":"2022-09-20T19:48:01","modified_gmt":"2022-09-20T14:18:01","slug":"solved-r-computing-not-so-fast","status":"publish","type":"post","link":"https:\/\/jassweb.com\/solved\/solved-r-computing-not-so-fast\/","title":{"rendered":"[Solved] R computing not so fast"},"content":{"rendered":"

[ad_1]
\n<\/p>\n

\n
\n
<\/div>\n
\n
\n

This isn’t quite right, but maybe gives some indication of how to make this type of operation faster. Here’s the data<\/p>\n

url <- \"http:\/\/pastebin.com\/raw.php?i=hsGACr2L\"\ndfi <- read.csv(url)\n<\/code><\/pre>\n
I calculate the product and cumulative sum of the product of price and volume. The calculation is vectorized so fast.<\/p>\n
pv <- with(dfi, Price * Volume)\ncpv <- cumsum(pv)\nvol_range <- 100000\n<\/code><\/pre>\n
My strategy was to figure out how to group the data in a relatively efficient way. I did this by creating a logical vector that will have ‘TRUE’ when a new group starts (I think the actual calculation is wrong below, and that there are edge cases that will fail; probably the strategy needs to be re-thought, but the notion is to minimize the non-vectorized data modification)<\/p>\n
grp <- logical(nrow(dfi))\ni <- 1\nrepeat {\n    grp[i] <- TRUE\n    ## find first index evaluating to 'TRUE'\n    i <- which.max(cpv - (cpv[i] - pv[i]) > vol_range)\n    ## prevent fails when, e.g., any(diff(cvp) > vol_range)\n    if (i > 1L && grp[i] == TRUE)\n        i <- i + 1L\n    if (i == 1L)   # no TRUE values, so FALSE is max, and elt 1 is first FALSE\n        break\n}\n<\/code><\/pre>\n
cumsum(grp)<\/code> divides the data into the first, second, … groups, and I add this to the data frame<\/p>\n
dfi$Group <- cumsum(grp)\n<\/code><\/pre>\n
For the output, the basic strategy is to split Price (etc.) by Group, and apply a function to each group. There are a number of ways to do this, tapply<\/code> is not particularly efficient (data.table excels at these types of calculations, but does not provide any particular benefit up to this point) but for the scale of data is likely to be sufficient.<\/p>\n
dfo <- with(dfi, {\n    data.frame(\n        open = tapply(Price, Group, function(x) x[1]),\n        high = tapply(Price, Group, max),\n        low = tapply(Price, Group, max),\n        close = tapply(Price, Group, function(x) x[length(x)]),\n        volume = tapply(Volume, Group, sum),\n        pv = tapply(Price * Volume, Group, sum))\n})\n<\/code><\/pre>\n
This takes a fraction of a second for the 10,000 row sample data.<\/p>\n<\/p><\/div>\n
<\/div>\n<\/div>\n
            <\/span> <\/p><\/div>\n<\/div>\n
[ad_2]<\/p>\n
solved R computing not so fast <\/p>\n","protected":false},"excerpt":{"rendered":"
[ad_1] This isn’t quite right, but maybe gives some indication of how to make this type of operation faster. Here’s the data url <- “http:\/\/pastebin.com\/raw.php?i=hsGACr2L” dfi <- read.csv(url) I calculate the product and cumulative sum of the product of price and volume. The calculation is vectorized so fast. pv <- with(dfi, Price * Volume) cpv … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[320],"tags":[321],"class_list":["post-9801","post","type-post","status-publish","format-standard","hentry","category-solved","tag-r"],"yoast_head":"\n[Solved] R computing not so fast - JassWeb<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/jassweb.com\/solved\/solved-r-computing-not-so-fast\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"[Solved] R computing not so fast - JassWeb\" \/>\n<meta property=\"og:description\" content=\"[ad_1] This isn’t quite right, but maybe gives some indication of how to make this type of operation faster. Here’s the data url <- "http:\/\/pastebin.com\/raw.php?i=hsGACr2L" dfi <- read.csv(url) I calculate the product and cumulative sum of the product of price and volume. The calculation is vectorized so fast. pv <- with(dfi, Price * Volume) cpv ... Read more\" \/>\n<meta property=\"og:url\" content=\"https:\/\/jassweb.com\/solved\/solved-r-computing-not-so-fast\/\" \/>\n<meta property=\"og:site_name\" content=\"JassWeb\" \/>\n<meta property=\"article:published_time\" content=\"2022-09-20T14:18:01+00:00\" \/>\n<meta name=\"author\" content=\"Kirat\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Kirat\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-r-computing-not-so-fast\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-r-computing-not-so-fast\\\/\"},\"author\":{\"name\":\"Kirat\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#\\\/schema\\\/person\\\/65c9c7b7958150c0dc8371fa35dd7c31\"},\"headline\":\"[Solved] R computing not so fast\",\"datePublished\":\"2022-09-20T14:18:01+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-r-computing-not-so-fast\\\/\"},\"wordCount\":215,\"publisher\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#organization\"},\"keywords\":[\"r\"],\"articleSection\":[\"Solved\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-r-computing-not-so-fast\\\/\",\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-r-computing-not-so-fast\\\/\",\"name\":\"[Solved] R computing not so fast - JassWeb\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#website\"},\"datePublished\":\"2022-09-20T14:18:01+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-r-computing-not-so-fast\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-r-computing-not-so-fast\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-r-computing-not-so-fast\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"[Solved] R computing not so fast\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#website\",\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/\",\"name\":\"JassWeb\",\"description\":\"Build High-quality Websites\",\"publisher\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#organization\",\"name\":\"Jass Web\",\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/jassweb.com\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/jass-website-logo-1.png\",\"contentUrl\":\"https:\\\/\\\/jassweb.com\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/jass-website-logo-1.png\",\"width\":693,\"height\":132,\"caption\":\"Jass Web\"},\"image\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#\\\/schema\\\/person\\\/65c9c7b7958150c0dc8371fa35dd7c31\",\"name\":\"Kirat\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/wp-content\\\/litespeed\\\/avatar\\\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777008400\",\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/wp-content\\\/litespeed\\\/avatar\\\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777008400\",\"contentUrl\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/wp-content\\\/litespeed\\\/avatar\\\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777008400\",\"caption\":\"Kirat\"},\"sameAs\":[\"http:\\\/\\\/jassweb.com\"],\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/author\\\/jaspritsinghghumangmail-com\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"[Solved] R computing not so fast - JassWeb","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/jassweb.com\/solved\/solved-r-computing-not-so-fast\/","og_locale":"en_US","og_type":"article","og_title":"[Solved] R computing not so fast - JassWeb","og_description":"[ad_1] This isn’t quite right, but maybe gives some indication of how to make this type of operation faster. Here’s the data url <- \"http:\/\/pastebin.com\/raw.php?i=hsGACr2L\" dfi <- read.csv(url) I calculate the product and cumulative sum of the product of price and volume. The calculation is vectorized so fast. pv <- with(dfi, Price * Volume) cpv ... Read more","og_url":"https:\/\/jassweb.com\/solved\/solved-r-computing-not-so-fast\/","og_site_name":"JassWeb","article_published_time":"2022-09-20T14:18:01+00:00","author":"Kirat","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Kirat","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/jassweb.com\/solved\/solved-r-computing-not-so-fast\/#article","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/solved-r-computing-not-so-fast\/"},"author":{"name":"Kirat","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31"},"headline":"[Solved] R computing not so fast","datePublished":"2022-09-20T14:18:01+00:00","mainEntityOfPage":{"@id":"https:\/\/jassweb.com\/solved\/solved-r-computing-not-so-fast\/"},"wordCount":215,"publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"keywords":["r"],"articleSection":["Solved"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/jassweb.com\/solved\/solved-r-computing-not-so-fast\/","url":"https:\/\/jassweb.com\/solved\/solved-r-computing-not-so-fast\/","name":"[Solved] R computing not so fast - JassWeb","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/#website"},"datePublished":"2022-09-20T14:18:01+00:00","breadcrumb":{"@id":"https:\/\/jassweb.com\/solved\/solved-r-computing-not-so-fast\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/jassweb.com\/solved\/solved-r-computing-not-so-fast\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/jassweb.com\/solved\/solved-r-computing-not-so-fast\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/jassweb.com\/solved\/"},{"@type":"ListItem","position":2,"name":"[Solved] R computing not so fast"}]},{"@type":"WebSite","@id":"https:\/\/jassweb.com\/solved\/#website","url":"https:\/\/jassweb.com\/solved\/","name":"JassWeb","description":"Build High-quality Websites","publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/jassweb.com\/solved\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/jassweb.com\/solved\/#organization","name":"Jass Web","url":"https:\/\/jassweb.com\/solved\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/","url":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","contentUrl":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","width":693,"height":132,"caption":"Jass Web"},"image":{"@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31","name":"Kirat","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777008400","url":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777008400","contentUrl":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777008400","caption":"Kirat"},"sameAs":["http:\/\/jassweb.com"],"url":"https:\/\/jassweb.com\/solved\/author\/jaspritsinghghumangmail-com\/"}]}},"_links":{"self":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/9801","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/comments?post=9801"}],"version-history":[{"count":0,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/9801\/revisions"}],"wp:attachment":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/media?parent=9801"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/categories?post=9801"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/tags?post=9801"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}