Skip to content

Lifetime of a Buffer in a C++ addon #3222

Description

@sgerace

I'm working on an asynchronous C++ addon that is going to be processing buffers piped in from a stream, and I'm wondering the best way to persist the buffer data from the main event-loop thread into my worker thread. I'm basing much of my design on the way the Node.js zlib module is structured in the sense that I'm exposing a process function of my class that looks something like:

void Processor::ProcessAsync(const FunctionCallbackInfo<Value> &args) {
    Isolate* isolate = args.GetIsolate();

    // Unwrap processor object
    Processor *obj = ObjectWrap::Unwrap<Processor>(args.Holder());

    // Reset callback
    Local<Function> callback = Local<Function>::Cast(args[1]);
    obj->callback_.Reset(isolate, callback);

    // Get current buffer data...if this is set on obj, will is persist?
    unsigned char *buf = (unsigned char *)node::Buffer::Data(args[0]);
    size_t length = node::Buffer::Length(args[0]);

    // Build up the work request
    uv_work_t *request = &(obj->request_);
    uv_queue_work(uv_default_loop(), request, Processor::Process, Processor::After);

    // Return value
    args.GetReturnValue().Set(Undefined(isolate));
}

My question is whether or not the unsigned char *buf will persist throughout the lifetime of my class, or as I suspect, I will need to create a local copy of the data. I'm wondering if there is a way to force v8 to persist the data until I release it, but I'm just getting into the deep-end (or maybe the shallow end...) of interacting with v8 and would appreciate any guidance on how stuff like this is typically accomplished.

Activity

  1. bnoordhuis commented on Oct 6, 2015

    @bnoordhuis
    Member

    You need to either keep a reference around to the buffer object or make a copy of the data. It looks like you're already using a v8::Persistent<T> or v8::Global<T> for the callback, I'd just extend that to the buffer object.

  2. sgerace commented on Oct 6, 2015

    @sgerace
    Author

    Ahh, yeah, makes sense. So in my class something like:

    v8::Persistent<v8::Object> buffer_;
    v8::Persistent<v8::Function> callback_; // I already have this one
    

    And then set it in my function the same way I'm setting the callback:

    obj->buffer_.Reset(isolate, args[0]);
    obj->callback_.Reset(isolate, callback);
    

    Am I then correct in assuming that when my class's destructor is called these two persistent references are both released, so there isn't anything else I would need to do?

    Now that I know where to look, it seems obvious; I'm guessing that I can also make a call to buffer_.Reset() (with no arguments) in my class' cleanup function (to force cleanup independent of GC) to release the handles without having to rely on my destructor being called, correct?

  3. bnoordhuis commented on Oct 6, 2015

    @bnoordhuis
    Member

    v8::Persistent<T> doesn't Reset() in its destructor (by default) but v8::Global<T> does. If you want to play it safe, call Reset() explicitly in your destructor.

  4. added
    bufferIssues and PRs related to the buffer subsystem.
    c++Issues and PRs that require attention from people who are familiar with C++.
    questionIssues asking questions about Node.js.
    on Oct 6, 2015
  5. sgerace commented on Oct 7, 2015

    @sgerace
    Author

    Ahh, yep, once again, knowing what to look for helps. For future reference, I found the relevant section in Google's v8 Embedder's Guide: https://developers.google.com/v8/embed#handles-and-garbage-collection

    Specifically where they state:

    A Persistent<SomeType> can be constructed with its constructor, but must be explicitly cleared with Persistent::Reset.

    Thanks for the help!

  6. sgerace commented on Oct 7, 2015

    @sgerace
    Author

    @bnoordhuis Upon further reading, I found this comment in the same guide:

    During the garbage collection process the garbage collector often moves objects to different locations in the heap. When the garbage collector moves an object the garbage collector also updates all handles that refer to the object with the object's new location.

    Seeing as though I cannot interact with v8 in my uv worker function, do I need to be concerned about my data moving if I store a persistent handle at my class level and then pass a pointer to my data into my worker function? In other words, does the persistent handle guarantee that the pointer I get from node::Buffer::Data will remain valid?

  7. reopened this on Oct 7, 2015
  8. bnoordhuis commented on Oct 7, 2015

    @bnoordhuis
    Member

    The memory that the buffer object points to won't move but the object itself can. Rule of thumb: don't interact with node or V8 when on a different thread. That means you shouldn't call e.g. node::Buffer::Data() from the thread pool.

  9. sgerace commented on Oct 7, 2015

    @sgerace
    Author

    Makes sense, so by that logic if I store a pointer to the data (i.e., as a char * obtained from node::Buffer::Data()) in my class, it should be safe to consume that data in a worker thread as long as the Buffer itself is contained in a Persistent handle (also as a class variable). Does that sound correct?

  10. bnoordhuis commented on Oct 7, 2015

    @bnoordhuis
    Member

    Yes, 100% correct.

  11. sgerace commented on Oct 7, 2015

    @sgerace
    Author

    @bnoordhuis Awesome, thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bufferIssues and PRs related to the buffer subsystem.c++Issues and PRs that require attention from people who are familiar with C++.questionIssues asking questions about Node.js.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions