This repository was archived by the owner on May 7, 2022. It is now read-only.
Repository navigation
This repository was archived by the owner on May 7, 2022. It is now read-only.
acceptance tests #11
Description
was thinking about how node created breaking changes in bugfixes very often and how to avoid it. how about a test infrastructure that tests popular packages on a nightly build? i.e. create a container with the nightly build, install express/hapi/nan/whatever and run npm test. might catch some tiny bugs before releasing.
Activity
othiym23 commentedon Jan 18, 2015
I would love this, not least for npm itself. It's only diligence on the part of @wraithan that we caught what would have completely broken npm when used with Node.js 0.11.15 had it not been caught beforehand. npm, express, leveldown, gulp, hapi, and 5-10 others would be a good place to start.
wraithan commentedon Jan 18, 2015
A few months back I brought up this idea with @nvcexploder. Back then we were talking about hapi and the node-newrelic module. He came up with the idea that there could be a repo of tests that you just PR your module into, which if this suite existed for io.js folks could do that.
An alternative, to borrow from the rust community, would be something like http://rust-ci.org/ The idea being that you have nightly builds that you push into a PPA. Then you have a travis-ci build that pulls from that PPA and runs the tests. Finally, to make sure it gets ran on each nightly, once the PPA is updated rust-ci triggers a rebuild of the last build of each travis-ci job it knows about.
The choice to build and push to a PPA means that rust doesn't have to be built in every single job and instead gets built once and everyone tests against the same exact builds.
I'm not sure that the rust-ci solution is the best, since it means other people maintaining the suite and having to make sure that the failures are indeed because of io.js instead of because their suite sucks. But it does mean that greater coverage is more achievable. Significant drops in the number of successful builds would be blatant indicators of breaking something.
Just my 2 cents.
mikeal commentedon Jan 24, 2015
This should be logged as a feature request on https://github.com/iojs/build
There's no contention here, I think everyone wants this, it's just a matter of prioritizing it against build's other work.