{"id":12027,"date":"2022-09-29T09:31:05","date_gmt":"2022-09-29T04:01:05","guid":{"rendered":"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/"},"modified":"2022-09-29T09:31:05","modified_gmt":"2022-09-29T04:01:05","slug":"solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js","status":"publish","type":"post","link":"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/","title":{"rendered":"[Solved] How to use promise.all with a specified number of parallel processing at Node.js"},"content":{"rendered":"<p> [ad_1]<br \/>\n<\/p>\n<div id=\"answer-70469327\" class=\"answer js-answer accepted-answer js-accepted-answer\" data-answerid=\"70469327\" data-parentid=\"70469239\" data-score=\"2\" data-position-on-page=\"1\" data-highest-scored=\"1\" data-question-has-accepted-highest-score=\"1\" itemprop=\"acceptedAnswer\" itemscope itemtype=\"https:\/\/schema.org\/Answer\">\n<div class=\"post-layout\">\n<div class=\"votecell post-layout--left\"><\/div>\n<div class=\"answercell post-layout--right\">\n<div class=\"s-prose js-post-body\" itemprop=\"text\">\n<blockquote>\n<p>I have a list of promises: [pr1, pr2, pr3, pr4, pr5]<\/p>\n<\/blockquote>\n<p>If you have a list of 5 promises you already have 5 things processing in parallel. There is no way to stop it since you have already triggered the processes by creating the promises.<\/p>\n<p>If you want to process only two of them at once you need to <strong>NOT CREATE THE PROMISES<\/strong>. Therefore what you need is a list of 5 <strong>functions that return promises<\/strong> instead of 5 promises.<\/p>\n<p>What you need is an array of <code>[f1, f2, f3, f4, f5]<\/code> where <code>f1<\/code> will return <code>pr1<\/code>, <code>f2<\/code> will return <code>pr2<\/code> etc.<\/p>\n<p>Once you have this all you need to do is to <code>Promise.all()<\/code> two promises at a time:<\/p>\n<pre><code>const tasks = [f1, f2, f3, f4, f5];\n\nconst BATCH_IN_PARALLEL = 2;\n\nasync function batchTasks() {\n    for (let i=0; i&lt;tasks.length;) {\n        let promises = [];\n\n        \/\/ create two promises at a time:\n        for (let j=0; j&lt;BATCH_IN_PARALLEL &amp;&amp; i&lt;tasks.length; i++,j++) {\n            let t = tasks[i];\n            promises.push(t()); \/\/ create the promise here!\n        }\n\n        await Promise.all(promises); \/\/ wait for the two promises\n    }\n}\n<\/code><\/pre>\n<p>If you need the result of the promises just collect them in an array:<\/p>\n<pre><code>async function batchTasks() {\n    let result = [];\n\n    for (let i=0; i&lt;tasks.length;) {\n        let promises = [];\n\n        \/\/ create two promises at a time:\n        for (let j=0; j&lt;BATCH_IN_PARALLEL &amp;&amp; i&lt;tasks.length; i++,j++) {\n            let t = tasks[i];\n            promises.push(t()); \/\/ create the promise here!\n        }\n\n        result.push(await Promise.all(promises));\n    }\n\n    return result;\n}\n<\/code><\/pre>\n<p>The above is a basic implementation of batching. It only processes two async functions at a time but it waits for both to complete before processing two more. You can get creative and process another function as soon as one is done but the code for that is a bit more involved.<\/p>\n<p>The <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.npmjs.com\/package\/async-q\">async-q<\/a> library has a function that does just this: <code>asyncq.parallelLimit<\/code>:<\/p>\n<pre><code>const asyncq = require('async-q');\n\nconst tasks = [f1, f2, f3, f4, f5];\n\nlet result = asyncq.parallelLimit(tasks, 2);\n<\/code><\/pre>\n<hr>\n<h3>Additional answer<\/h3>\n<p>Here is some old code I found in one of my projects that continuously processes two tasks in parallel. It uses a recursive function to process the tasks array until empty. As you can see, the code is a bit more complicated but not too difficult to understand:<\/p>\n<pre><code>function batch (tasks, batch_in_parallel) {\n    let len = tasks.length;\n    \n    return new Promise((resolve, reject)=&gt;{\n        let counter = len;\n\n        function looper () {\n            if (tasks.length != 0) {\n                \/\/ remove task from the front of the array:\n                \/\/ note: alternatively you can use .pop()\n                \/\/       to process tasks from the back\n                tasks.shift()().then(()=&gt;{\n                    counter--;\n                    if (counter) { \/\/ if we still have tasks\n                        looper();  \/\/ process another task\n                    }\n                    else {\n                        \/\/ if there are no tasks left we are\n                        \/\/ done so resolve the promise:\n                        resolve();\n                    }\n                });\n            }\n        }\n\n        \/\/ Start parallel tasks:\n        for (let i=0; i&lt;batch_in_parallel; i++) {\n            looper();\n        }\n    });\n}\n\n\/\/ Run two tasks in parallel:\nbatch([f1, f2, f3, f4, f5], 2).then(console.log('done');\n<\/code><\/pre>\n<p>Note that the above function does not return results. You can modify it to collect the result in an array then return the result by passing it to <code>resolve(result)<\/code> but making sure the result is in the same order as the tasks is not trivial.<\/p>\n<p>Nowdays I&#8217;d just use <code>asyncq.parallelLimit()<\/code> unless I really don&#8217;t want to import the entire <code>async-q<\/code> library or my boss\/client does not trust it.<\/p>\n<\/p><\/div>\n<div class=\"mt24\"><\/div>\n<\/div>\n<p>            <span class=\"d-none\" itemprop=\"commentCount\"><\/span> <\/p><\/div>\n<\/div>\n<p>[ad_2]<\/p>\n<p>solved How to use promise.all with a specified number of parallel processing at Node.js <\/p>\n","protected":false},"excerpt":{"rendered":"<p>[ad_1] I have a list of promises: [pr1, pr2, pr3, pr4, pr5] If you have a list of 5 promises you already have 5 things processing in parallel. There is no way to stop it since you have already triggered the processes by creating the promises. If you want to process only two of them &#8230; <a title=\"[Solved] How to use promise.all with a specified number of parallel processing at Node.js\" class=\"read-more\" href=\"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/\" aria-label=\"More on [Solved] How to use promise.all with a specified number of parallel processing at Node.js\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[320],"tags":[333,902,1163,2982],"class_list":["post-12027","post","type-post","status-publish","format-standard","hentry","category-solved","tag-javascript","tag-node-js","tag-parallel-processing","tag-promise"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>[Solved] How to use promise.all with a specified number of parallel processing at Node.js - JassWeb<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"[Solved] How to use promise.all with a specified number of parallel processing at Node.js - JassWeb\" \/>\n<meta property=\"og:description\" content=\"[ad_1] I have a list of promises: [pr1, pr2, pr3, pr4, pr5] If you have a list of 5 promises you already have 5 things processing in parallel. There is no way to stop it since you have already triggered the processes by creating the promises. If you want to process only two of them ... Read more\" \/>\n<meta property=\"og:url\" content=\"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/\" \/>\n<meta property=\"og:site_name\" content=\"JassWeb\" \/>\n<meta property=\"article:published_time\" content=\"2022-09-29T04:01:05+00:00\" \/>\n<meta name=\"author\" content=\"Kirat\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Kirat\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/\"},\"author\":{\"name\":\"Kirat\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31\"},\"headline\":\"[Solved] How to use promise.all with a specified number of parallel processing at Node.js\",\"datePublished\":\"2022-09-29T04:01:05+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/\"},\"wordCount\":333,\"publisher\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\"},\"keywords\":[\"javascript\",\"node.js\",\"parallel-processing\",\"promise\"],\"articleSection\":[\"Solved\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/\",\"url\":\"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/\",\"name\":\"[Solved] How to use promise.all with a specified number of parallel processing at Node.js - JassWeb\",\"isPartOf\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#website\"},\"datePublished\":\"2022-09-29T04:01:05+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/jassweb.com\/solved\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"[Solved] How to use promise.all with a specified number of parallel processing at Node.js\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/jassweb.com\/solved\/#website\",\"url\":\"https:\/\/jassweb.com\/solved\/\",\"name\":\"JassWeb\",\"description\":\"Build High-quality Websites\",\"publisher\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/jassweb.com\/solved\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\",\"name\":\"Jass Web\",\"url\":\"https:\/\/jassweb.com\/solved\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png\",\"contentUrl\":\"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png\",\"width\":693,\"height\":132,\"caption\":\"Jass Web\"},\"image\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31\",\"name\":\"Kirat\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586\",\"contentUrl\":\"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586\",\"caption\":\"Kirat\"},\"sameAs\":[\"http:\/\/jassweb.com\"],\"url\":\"https:\/\/jassweb.com\/solved\/author\/jaspritsinghghumangmail-com\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"[Solved] How to use promise.all with a specified number of parallel processing at Node.js - JassWeb","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/","og_locale":"en_US","og_type":"article","og_title":"[Solved] How to use promise.all with a specified number of parallel processing at Node.js - JassWeb","og_description":"[ad_1] I have a list of promises: [pr1, pr2, pr3, pr4, pr5] If you have a list of 5 promises you already have 5 things processing in parallel. There is no way to stop it since you have already triggered the processes by creating the promises. If you want to process only two of them ... Read more","og_url":"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/","og_site_name":"JassWeb","article_published_time":"2022-09-29T04:01:05+00:00","author":"Kirat","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Kirat","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/#article","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/"},"author":{"name":"Kirat","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31"},"headline":"[Solved] How to use promise.all with a specified number of parallel processing at Node.js","datePublished":"2022-09-29T04:01:05+00:00","mainEntityOfPage":{"@id":"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/"},"wordCount":333,"publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"keywords":["javascript","node.js","parallel-processing","promise"],"articleSection":["Solved"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/","url":"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/","name":"[Solved] How to use promise.all with a specified number of parallel processing at Node.js - JassWeb","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/#website"},"datePublished":"2022-09-29T04:01:05+00:00","breadcrumb":{"@id":"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/jassweb.com\/solved\/solved-how-to-use-promise-all-with-a-specified-number-of-parallel-processing-at-node-js\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/jassweb.com\/solved\/"},{"@type":"ListItem","position":2,"name":"[Solved] How to use promise.all with a specified number of parallel processing at Node.js"}]},{"@type":"WebSite","@id":"https:\/\/jassweb.com\/solved\/#website","url":"https:\/\/jassweb.com\/solved\/","name":"JassWeb","description":"Build High-quality Websites","publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/jassweb.com\/solved\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/jassweb.com\/solved\/#organization","name":"Jass Web","url":"https:\/\/jassweb.com\/solved\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/","url":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","contentUrl":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","width":693,"height":132,"caption":"Jass Web"},"image":{"@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31","name":"Kirat","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/image\/","url":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586","contentUrl":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586","caption":"Kirat"},"sameAs":["http:\/\/jassweb.com"],"url":"https:\/\/jassweb.com\/solved\/author\/jaspritsinghghumangmail-com\/"}]}},"_links":{"self":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/12027","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/comments?post=12027"}],"version-history":[{"count":0,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/12027\/revisions"}],"wp:attachment":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/media?parent=12027"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/categories?post=12027"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/tags?post=12027"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}