Repository navigation
Possible bug in streams between node 8.1.4 and 8.2.0 #15029
Description
Activity
@nodejs/streams
That seems like a good candidate.
Our code for this particular case is really hairy, so if it was an actual bug, and the fix is backported to 6, I'll with great joy remove our current handling of it 🙂
Can you please verify with 8.4.0?
8.2.0, 8.2.1, 8.3.0 and 8.4.0 all behave the same
6.11.2, 7.11.1, 8.0.0, 8.1.0 and 8.1.4 also behave the same, FWIW. So something changed between 8.1.4 and 8.2.0
So fare I see: Under the hood, pump uses end-of-stream. In version 8.1.4 and older end-of-stream get a
closeevent from the underlaying stream (which then are used to throw an Error further up). In version 8.2.0 and newer end-of-stream get afinishevent from the underlaying stream.I've pushed usage of request instead of fetch. Never used got (it's get from http). Can look into removing express on Monday
I meant, can you just show this using node core?
Here is a stipped down version:
const { get, createServer } = require('http'); let external; // Http server createServer((req, res) => { res.writeHead(200); setTimeout(() => { external.abort(); res.end('Hello World\n'); }, 1000); }).listen(3000); // Proxy server createServer((req, res) => { get('http://127.0.0.1:3000', inner => { res.on('close', () => { console.log('response writable:', res.writable); }); inner.pipe(res); }); }).listen(3001, () => { external = get('http://127.0.0.1:3001'); });
What happens here is that we (
external) do a request to a server (proxy server) which stream data through from a second server (http serve) and we terminate the request before the stream is done. A real use case for this is ex a user hitting the stop button in the browser while the server is still piping data to it.If one run the above in node 8.1.4 or older,
response writable: truewill be printed. If one run the above in node 8.2.0 or newer,response writable: falsewill be printed.In other words; in node 8.1.4 and older it seems like the writable response stream was writable after a
closeevent. This would again create a problem where the stream could be written too after it was closed.To me this looks like a fixed bug in node 8.2.0 and not a new bug. What make it looks like a bug is that libraries such as end-of-stream and pump suddenly stopped providing error objects on the above scenario between two versions of node.
If anyone can confirm that the above behaviour is correct (the response should not be writable after a close event), I'll be happy to report this to end-of-stream and pump so they can account for this change between the node versions.
4 remaining items
The stream are terminated prematurelly in all versions. In node 8.1.4 and older
eos(pumphave this "error" inheritated fromeos) ends with an error object telling us that. In 8.2.0eosends with no error so in newer versions we really do not know that the stream was terminated prematurelly.I am not sure how
eoswill be able to figure this out now. I created an issue on it: mafintosh/end-of-stream#14@trygve-lie I understand the problem now.
pumpis calling.end()on the response, causingres.writableto go tofalse. This preventseosto call the callback with the error.I think it's a bug in core.
stream.Writablesetsws.writable = falseafter emitting'finish', while OutgoingMessage setsws.writable = truebefore emitting'finish'.Reacted by Simen Bekkhus and Tuan Anh TranWhat's happening is clear to me
I'm curios, could you expand on that?
EDIT: Done in #15404 (comment), so no need to do so here. Thanks!
- added a commit that references this issue
on Sep 20, 2017 - added 2 commits that reference this issue
on Sep 21, 2017 - added a commit that references this issue
on Jul 27, 2026 - added a commit that references this issue
on Aug 22, 2026
We have some code which uses
pump, and between 8.1.4 and 8.2.0 the handler in certain cases stopped being called with an error. We have a test that our error handler is invoked correctly, which fails on newer versions of node.In 8.1.4 we hit https://github.com/mafintosh/end-of-stream/blob/0ae1658b8167596fafbb9195363ada3bc5a3eaf2/index.js#L44, whilst in 8.2.0 (and later) we don't.
I've created a repository with reproduction code.
https://github.com/SimenB/node-stream-repro