Feature: requestIdleCallback(cb)

[isaacs]

This seems interesting and useful, and probably wouldn't be terrifically hard to add to node: https://w3c.github.io/requestidlecallback/

It just landed in Chrome Canary: https://plus.google.com/+IlyaGrigorik/posts/bPNjgMwcMKs

Could be worth waiting a bit to see how people find it in browsers, or if V8 has any sort of special support for it in upcoming versions.

[Fishrock123]

cc @ofrobots

[benjamingr]

Why not userland? What information about the server load isn't available to the userland and if it is not available - wouldn't it be better to expose it instead and let userland solutions evolve (at least first)?

[isaacs]

@benjamingr By definition, it's impossible for userland JS to know when V8 isn't busy doing anything, because when it's running userland JS, it's busy doing that.

You could sort of fake it with setTimeouts or by writing a C++ addon that uses the v8::IdleTimeout stuff, but it would be tricky to get exactly the same logic as in Chrome.

[benjamingr]

Well, if you schedule a macrotask (setTimeout) wouldn't it run after node is "no longer busy"?

How do we define idle? Idle in a UI setting is different from idle on a server. I definitely see the usages of "running low priority background work" in node - but what do we consider idle here?

[pmq20]

@isaacs What is v8::IdleTimeout?

[pmq20]

@benjamingr I searched the code in vain for a ready-to-use definition of idle-ness. libuv has a uv_idle_t but it's not what we are looking for:

Despite the name, idle handles will get their callbacks called on every loop iteration,
not when the loop is actually "idle".

Therefore I think we need to first extend libuv to provide an API that could register callbacks for when the loop is really "idle".

[bnoordhuis]

A browser's event loop works at discrete intervals so there is a useful definition of "idle": no more work for this quantum, next quantum hasn't started yet, remaining time is idle time.

Node's event loop however is continuous and it's only truly idle when it's waiting for timers to expire and nothing else. As soon as there is any I/O involved, it can go from idle to busy at any time without warning.

How do browsers process requestIdleCallback() when there is an in-progress XMLHttpRequest?

[pmq20]

@bnoordhuis I think there is one more idle situation on server, that is, when I/O is in progress, i.e. when the event loop polls for i/o but there are no pending callbacks yet.

[bnoordhuis]

@pmq20 Waiting for I/O is not idleness in the "predictable quantum of time" definition from the W3C document. Outside of the "timers only" scenario, node is very much like Chuck Norris in that it never sleeps, it just waits.

(Yes, I slipped a Chuck Norris joke in there. I apologize for nothing.)

Tangential: the concept of idle callbacks has some relation with the idle GC feature that was removed three years ago in commit d607d85.

The idea was to run the garbage collector proactively at quiescent times but it never worked all that well and for pretty much the same reasons that make requestIdleCallback() tricky.

[isaacs]

@bnoordhuis That's a good point.

Doing GC at those times turned out to be a bad idea, I do recall.

I'd just as soon tell people to do "long-running work" using child procs (or web workers on browsers). It makes more sense on browsers, since you do have some kinds of work that must block the same thread as UI (updating the dom, etc) and doing that when idle is less likely to impact the user.

If there's no way to do this in a way that makes sense, then that's reasonable. But I can expect that someone will ask for it at some point.

[tjconcept]

Tangential: the concept of idle callbacks has some relation with the idle GC feature that was removed three years ago in commit d607d85.

Is it related to what has later been described here? http://v8project.blogspot.dk/2015/08/getting-garbage-collection-for-free.html

[Fishrock123]

Closing since it doesn't seem very feasible. Maybe we can re-evaluate in the future.

[sebmarkbage]

Let me describe my use case and maybe that can inform a plausible design.

Marshaling between workers is too expensive and add too much overhead. Especially when you have lots of little memoized data shared between multiple parts of the app. It is also difficult to break a part the app at a reasonable seam that is stable over time. The easiest way to get good scheduling is to break it apart into small slices of work - which is why the I/O model of Node is good to begin with. This is why rearchitected React (Fiber) to use cooperative scheduling on the client, powered by requestIdleCallback. It lets us gather all input from all the I/O in the beginning of the frame and then use the rest of the available time to prioritize and do the highest priority work.

In my Node use case I have a HTTP request/response model where I need to on demand fetch some data from the backend. Different segments of the response has a dependency on different data from the backend. The segments can be computed out of order but each segment has a priority associated with it. If I have data available from one of the lower priority segments but not higher priority segment, I don't want to block. I want to use the idle CPU time to compute some of the lower priority segments.

E.g.:

Request data for Segment 1.
Request data for Segment 2.
Data returns for Segment 2.
Callback fires for Segment 2. Process Segment 2.
Data returns for Segment 1.
Callback fires for Segment 1. Process Segment 1.

If the data for Segment 1 returns just after I've started processing Segment 2, then I've probably done the wrong choice because now Segment 1 gets delayed a bit even though it has higher priority. However, in most cases it's a better use of time. My goal is to minimize these cases.

If I understand the Node architecture well enough, it is possible to get into situations like this:

Request data for Segment 1.
Request data for Segment 2.
Request data for Segment 3.
Data returns for Segment 2.
Callback fires for Segment 2. Process Segment 2.
Data returns for Segment 3. (Still in the middle of processing Segment 2.)
Data returns for Segment 1. (Still in the middle of processing Segment 3.)
Callback fires for Segment 3. Process Segment 3.
Callback fires for Segment 1. Process Segment 1.

In this case we have already received data for Segment 1 by the time we start processing Segment 3, but because they returned in the order I can't change the priority.

I'd like to have a way to wait for all my I/O processing to be done and built up my own priority queue.

Request data for Segment 1.
Request data for Segment 2.
Request data for Segment 3.
Data returns for Segment 2.
Callback fires for Segment 2. Add to priority queue. Request Idle.
Idle: Process Segment 2.
Data returns for Segment 3. (Still in the middle of processing Segment 2.)
Data returns for Segment 1. (Still in the middle of processing Segment 3.)
Callback fires for Segment 3. Add to priority queue. Request Idle.
Callback fires for Segment 1. Add to priority queue.
Idle: Process Segment 1. Process Segment 3. (Processing occurs in priority order in this batch of work.)

So, for my use case it's not quite requestIdleCallback because there is no "deadline" associated with the callback. Instead, it's just used as a way to schedule something after we've exhausted everything already queued in the buffers.

EDIT: I've been told setImmediate might do this already. Is that correct?