Skip to content

Stop test after first assertion failure, and handle exceptions thrown from callbacks - #177

Closed
gilad61 wants to merge 1 commit into
caolan:masterfrom
gilad61:master
Closed

gilad61 wants to merge 1 commit into
caolan:masterfrom
gilad61:master

Conversation

@gilad61

@gilad61 gilad61 commented Aug 1, 2012

Copy link
Copy Markdown

This pull request contains two suggested fixes:

  1. Currently when an exception is thrown from an asynchronous callback the process crashes and subsequent tests don't run. Instead we should catch the exception and fail the test gracefully with test.done(exception).
  2. Currently when any assertion fails the test continues and all other assertions are called. This is not a common behavior. The test should fail and stop after the first assertion failed, and move on to the next test.

@Sannis

Sannis commented Sep 20, 2012

Copy link
Copy Markdown
Contributor

It would be good improvement.

@epall

epall commented Sep 26, 2012

Copy link
Copy Markdown

+1

1 similar comment
@dbaba

dbaba commented Feb 2, 2013

Copy link
Copy Markdown

+1

@spiffytech

Copy link
Copy Markdown

Can we please get this merged in? This functionality is crucial for me in a testing library.

@fxa

fxa commented Feb 12, 2013

Copy link
Copy Markdown

to 1. of course +1
to 2. I am torn. When writing new tests, I want to stop on the first error. When running my unit tests during a build, I don't expect an error. When my code (or my tests) are erroneous, the first thing I want to have is an overview.
For example, when ALL tests fail, propably I made an infrastructure thingy false. If I only see, that the first test fails, I would be mislead and investigate in this test.
I think, the best way would be an OPTION, whether to fail fast or not.
failpoint: firstFailedAssert | firstFailedTest | never

@spiffytech

Copy link
Copy Markdown

@fxa: (2) is not a request to abort the entire test suite when an error is encountered. It is a request to abort a particular test function when an assertion fails in that function. All other test functions in the test suite would still be run like normal.

@fxa

fxa commented Feb 12, 2013

Copy link
Copy Markdown

@ spiffytech
Sorry for my misunderstanding!
Forget my previous comment and it is a
+1 to all from me

@gregnavis

Copy link
Copy Markdown

👍

@mreinstein

Copy link
Copy Markdown
Collaborator

I think this is a good change. One thing to keep in mind is it's breaking the public API, introducing behavior that people may be depending on. Let's tentatively plan on doing this in the 1.0.0 release.

@ctalkington

Copy link
Copy Markdown

👍 this would be awesome

@fidanov

fidanov commented Apr 19, 2013

Copy link
Copy Markdown

+1 from me too, if this is not accepted soon I am changing to something else

@isacssouza

Copy link
Copy Markdown

+1, it's been 10 months since this pull request, is it ever going to be merged?

@mreinstein

Copy link
Copy Markdown
Collaborator

@isacssouza I'm not intentionally ignoring this merge. The problem is I'm not super familiar with nodeunit internals, so I'm hesitant to just merge in code without having a deeper understanding. I get the 2 points of the merge; to basically fix the test handling on uncaught exceptions and failing tests.

Can you please add some unit tests for this? I'll feel a bit better about the merge if I see some basic tests in place.

@isacssouza

Copy link
Copy Markdown

Thanks for the follow up, @mreinstein Unfortunately I'm also not familiar with this code. @gilad61, can you help us here?

I'll try to look into this when I have time.

@mlogan

mlogan commented Jul 18, 2013

Copy link
Copy Markdown

+1

(I normally hate to "me too" things, but since this has been open for a while...)

@mreinstein

Copy link
Copy Markdown
Collaborator

I'm all for doing this, still waiting for unit tests!

@zippo445

Copy link
Copy Markdown

+1 I would really need this feature. I might not be understanding the problem correctly but the following solution would work for me: https://github.com/proxy/gist.github.com/zippo445
just added two try/catch around fn.call(context, test) in the wrapTest function in core.js file....
should I fork and make a pull request out of this?

@mreinstein

Copy link
Copy Markdown
Collaborator

Again, I'm all for merging this. I'm still waiting for some unit tests for this feature before including it.

@zippo445

Copy link
Copy Markdown

I wrote this little test: https://github.com/proxy/gist.github.com/zippo445/6733044 it could be added to test/test-testcase.js hope it's enough to go with the merge. let me know.

@isacssouza

Copy link
Copy Markdown

Hey folks, sorry for not following up on this.

I was not able to create reliable tests for this patch, I have tried some
approaches but the tests failed in some scenarios. I really should have
reported this back then. Sorry about that.

On Fri, Sep 27, 2013 at 3:31 PM, phil notifications@github.com wrote:

I wrote this little test: zippo445 / gist:6733044 it could be added to
test/test-testcase.js hope it's enough to go with the merge. let me know.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/177#issuecomment-25266629
.

Isac.

@zippo445

Copy link
Copy Markdown

so..... is the simple test above ok? if not, can you please explain would be missing to make it robust enough?

@FGRibreau

Copy link
Copy Markdown

+1

5 similar comments
@stefanjudis

Copy link
Copy Markdown

+1

@markdb314

Copy link
Copy Markdown

+1

@bsrik77

bsrik77 commented Jan 14, 2014

Copy link
Copy Markdown

+1

@danielflippance

Copy link
Copy Markdown

+1

@glukki

glukki commented Feb 6, 2014

Copy link
Copy Markdown

+1

@YuukanOO

Copy link
Copy Markdown

Totally need this +1

@Granjow

Granjow commented Mar 13, 2014

Copy link
Copy Markdown

+42

Would be quite helpful to get err in this example instead of just the «could not be created» message when err is not null!

    user.save( function ( err, user ) {
            test.ok( err === null, err );
            test.ok( user != null, 'User could not be created' );
            test.done();
    } );

@glukki

glukki commented Mar 14, 2014

Copy link
Copy Markdown

@Granjow sorry, why don't you use test.ifError( err )?

@Granjow

Granjow commented Mar 14, 2014

Copy link
Copy Markdown

@glukki Valid point. Changed it, thanks for the hint. If err is an error, I still get the message of the second test for user != null though.

@caolan

caolan commented Mar 28, 2014

Copy link
Copy Markdown
Owner

This should probably be done with domains today. Not sure how best to handle this in the browser case.

@caolan caolan closed this Mar 28, 2014
@statico

statico commented Mar 28, 2014

Copy link
Copy Markdown

@caolan Did you mean to close this issue if it's not fixed?

@caolan

caolan commented Mar 30, 2014

Copy link
Copy Markdown
Owner

@statico close the pull request, yes... the issue itself still needs attention :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.