Repository navigation
CreateReadStream doesn't close the file #1180
Description
Activity
Can you show the relevant code, or at least a minimal, reproducible example?
There is a known problem when you pipe a stream to a response and the client aborts request. Try this package: https://github.com/jshttp/on-finished
@mscdex
here is a video with low quality:
https://www.dropbox.com/s/3avchxknfc2nrxj/nodejs-iojs-stream-problem.mov
Example: https://www.dropbox.com/s/fl3ic54zw3zdglb/iojs-stream-problem.zip@vkurchatkin the problem still persists, but the simulation is harder. Here is a video:
https://www.dropbox.com/s/p0soq6289yn5o6h/nodejs-iojs-problem-destroy-onfinished.mov
Example: https://www.dropbox.com/s/5wga3la8j10zq7y/iojs-stream-problem-harder.zip@vkurchatkin I tested this problem more. I destroyed the stream via
stream.close()andstream.destroy(), but the file has been still opened.Thanks!
please don't ask people to download zip files to test your code. just show us a gistfile.
the second example is incorrect.
var destroy = require('destroy'); var http = require('http'); var onFinished = require('on-finished'); var fs = require('fs'); http.createServer(function (req, res) { res.writeHead(200, { 'Content-Type': 'image/jpeg' }); var stream = fs.createReadStream('bg.jpg'); stream.pipe(res); onFinished(res, function(err) { res.on('finish', function() { destroy(stream); }); }); }).listen(1337, '127.0.0.1'); console.log(process.argv[0] + ': ' + process.pid + '\nhttp://127.0.0.1:1337');
you're basically doing something like:
res.on('finish', => res.on('finish', => destroy(stream) ) )
which isn't going to work because
finishis only going to be destroyed once. you just have to do this:onFinished(res, function () { destroy(stream) })
if the above doesn't work, please file an issue with
on-finishedanddestroy.see the bottom example: https://github.com/jshttp/on-finished#example
@jonathanong ohh sorry, my mistake.
I'll test it today and I'll give you a feedback. Thanks!I have tested it and with
on-finishedit works correctly. But I wonder why the stream is not closed after the end of the reading (withautoClose: true)? So I detected: the stream must be manually closed after the pipe is closed.I think that it's a bug. The test result with
onFinished:- streams opened: 311 504
- still opened: 7 files
Can you explain to me, why does it behaving like that?
Thanks.This has been dormant for a long time. Does anyone know if it is a confirmed bug? If so, any idea if it's been fixed?
Without further confirmation it's difficult to say. I'd recommend closing and we can reopen later if necessary.
Going to close given the lack of any further confirmation or discussion. Can reopen if necessary.
i believe this is the correct issue to post under...
i have a read stream piping a binary file to the response object,
var rstream = fs.createReadStream(testFilePath, { encoding: 'base64' });on first call it sends the complete file to the client but doesn't close the stream, on the second call it fails but raises the close event. once the close event has been fired, the following call will succeed and the subsequent call will fail in the same way.
at no time are the 'error', 'end' or 'finish' events raised,rstream .on('error', function (e) { console.log('rstream error'); }) .on('end', function (e) { console.log('rstream end'); }) .on('finish', function (e) { console.log('rstream finish'); }) .on('close', function (e) { console.log('rstream close'); }) ; rstream .pipe(res) .on('error', function (e) { console.log('pipe error'); }) .on('end', function (e) { console.log('pipe end'); }) .on('finish', function (e) { console.log('pipe finish'); }) .on('close', function (e) { console.log('pipe close'); }) ;in the logs, the only line is "pipe close" from the second call.
this only happens if encoding is set to base64, when i transport the file without transformation it works perfectly.
alright, i've figured out why it was only happening every second call - that's because the client's ajax calls weren't disconnecting properly (jquery). the only thing that still concerns me is that the end and finish events aren't raised, otherwise the close event is being raised just fine.
You should really use http://npm.im/pump to pipe things, or those situations can happen.
Reacted by Adam Fisheralright, i used pump, and it's easier to use (thanks!) but i have the same result. the only difference is that i'm able to catch the error properly on the second call (or any other call made using the jquery ajax function):
send complete Error: premature close
at ServerResponse.onclose (C:\myproject\node_modules\end-of-stream\index.js:44:54)
at emitNone (events.js:91:20)
at ServerResponse.emit (events.js:185:7)
at Socket.onServerResponseClose (_http_server.js:131:44)
at emitOne (events.js:101:20)
at Socket.emit (events.js:188:7)
at Pipe._handle.close [as _onclose] (net.js:498:12)Can you please upload a full example to demonstrate your issue?
var rstream = fs.createReadStream(testFilePath, { encoding: 'base64' }); pump(rstream, res, function (err) { console.log('send complete', err); });the first call returns the file data, but doesn't close the pipe. the second call fails and produces the following error:
send complete Error: premature close at ServerResponse.onclose (C:\proj\node_modules\end-of-stream\index.js:44:54) at emitNone (events.js:91:20) at ServerResponse.emit (events.js:185:7) at Socket.onServerResponseClose (_http_server.js:131:44) at emitOne (events.js:101:20) at Socket.emit (events.js:188:7) at Pipe._handle.close [as _onclose] (net.js:498:12)this is due to the jquery ajax call not completing properly, but clearly the stream should be closing regardless? again, no "end" of "finish" events raised.
You mention an Ajax call, can you upload a full example to demonstrate your problem?
$.ajax({ cache: false, url: fileAccessUrl, method: 'GET', headers: {}, success: function (data, textStatus, jqXHR) { var contentType = jqXHR.getResponseHeader("content-type"); console.log('received data with content type ' + contentType + ' and length ' + data.length); }, error: function (jqXHR, textStatus, errorThrown) { console.log(jqXHR.status + ':'); console.log(textStatus); console.log(errorThrown); } });@therightstuff I need something I can run on my laptop to help you. That's what I mean by a "Full" example.
without sharing proprietary code i would have to write it new, thanks anyway.
- added a commit that references this issue
on Dec 23, 2021
I found a big problem in streams. I performed multiple tests on Ubuntu 14.04 LTS and OSX too and I see multiple opened files through
lsof -p PID. I useautoClose: true(in default) and I destroy the stream manually with this module https://github.com/stream-utils/destroy.This problem is not recorded for every request, but only if I performed multiple requests at the same time. My website falls down.
How can I prevent this problem? Is there a bug? How to close opened files? Extending
ulimitis not a solution.Thanks for any advice.
PS: io.js v1.5.1
I reported the same problem on node.js GitHub.