Repository navigation
Worker pool? #31
Description
Activity
This could probably be adapted (and simplified) to suit:
https://github.com/proxy/gist.github.com/developit/65a2212731f6b00a8aaa55d70c594f5cFWIW, it's best to only invoke greenlet once per function - generally at definition time. Dynamically creating workers to do single tasks will always have a nontrivial performance overhead.
Just going to clarify for folks finding this issue - Greenlet does re-use the worker thread for all calls, but only if you hoist the greenlet() definition so it's being called one time for the given async function:
// BAD: this creates a new worker every time getPage() is called: export function getPage(url) { const getJson = greenlet(async url => { return await (await fetch(url)).json(); }); return getJson(url); }
// GOOD: this uses the same worker for each call to getPage(): const getJson = greenlet(async url => { return await (await fetch(url)).json(); }); export function getPage(url) { return getJson(url); }
Reacted by Spencer Levitthey @developit, I wanted to offload some functions to a worker and I was thinking if this 👇 solution is good enough? (the alternative is using those postMessage thingies by hand, which I hate):
let dispatch = greenlet(async (action) => { switch (action.type) { case 'fetchGH': { let url = `https://github.com/proxy/api.github.com/users/${action.username}` let res = await fetch(url) let profile = await res.json() return profile.name } case 'sayHi': { return "Hi " + action.name } default: { throw new Error(`Unknown action ${action.type}`) } } })
@developit I love workerize(-loader)! But it didn't play well with Next.js, so I ended up using
worker-plugin(which is also by you? :o) along withcomlink.So, I guess, if anyone is trying to bundle workers in Next.js, use that combo, it works perfectly. Otherwise, stick to greenlet and/or workerize.
(you should probably mark the last 3 comments as off-topic :) )
Oh I didn't mean the loader - there is a non-loader version of workerize that uses the same internals as Greenlet, works at runtime.
I needed to import the AWS SDK, that's why it wasn't an option :(
PS. If anyone needs to bundle web workers in Next.js, use Parcel (microbundle had a babel-related error) to build your workers to the
/publicfolder. Then use comlink.Otherwise use greenlet/workerize.
Creating a new worker for every invocation can be rather slow when dispatching small tasks. How about adding a pool of workers?