npm run test:* fails with error: "Unable to find path to phantomjs"

[da70]

The automated Selenium tests do not work in current develop and master branches of new clones of the repo. Clones made before Feb 18, 2016 might not have this bug, even if fast-forwarded to current branch heads. The bug only manifests if node_modules/ contains only dependencies in current package.json.

This issue is a Bug

Related issue(s) and/or pull request(s)

See #567.

Expected Behaviour

Run tests successfully.

Observed behaviour

Instead, running the tests fails with error:

> parallelshell "node build-config/selenium_webdriver.js -a 127.0.0.1 -p 8080 -c-1 ./dist/cloud-reader" "npm run mocha --readium-js-viewer:MODE=firefox"

http-server arguments: 
[ '-a', '127.0.0.1', '-p', '8080', '-c-1', './dist/cloud-reader' ]
SELENIUM ...
undefined:67
    throw new Error('Unable to find path to phantomjs, please run npm install and find the relative path for your system. Current location: [' + __dirname + ']' +

[...SNIPPED...]

Steps to reproduce

Make a new clone of readium-js-viewer and check out either develop or master branches. If using an old clone made before Feb 18, 2016, fast forward to current develop or master, delete node_modules/, and recreate it so that it matches package.json dependencies exactly.

Then run any of the following (might need to do npm run dist first?):

• npm run test
• npm run test:chrome
• npm run test:firefox

Product

The automated test suite that comes with the readium-js-viewer repo.

Additional information

Why it's happening:

Line 42 of build-config/selenium_webdriver.js:

                    var func = eval("( function(){"+fileContents+"; return {start: start, stop: stop}; } )");

...is slurping in this version of grunt-selenium-webdriver's node_modules/grunt-selenium-webdriver/tasks/selenium_webdriver.js, which is part of version 0.2.482 packaged for NPM. Lines 56-72 of this file:

...

    phantomLoc = __dirname,

...

// installed as module or locally?
if ( fs.existsSync( phantomLoc + "/../node_modules/phantomjs/bin/phantomjs") ) {
//    console.log ('branch 1b');
    // mac as grunt plugin test
    phantomLoc += "/../node_modules/phantomjs/bin";
} else if (fs.existsSync(phantomLoc + '/../../phantomjs/bin/phantomjs')) {
    // mac, module use
//    console.log ('branch 2b');
    phantomLoc += "/../../phantomjs/bin";
} else {
    // if adding new cases please identify environment so that changes can be maintained
    throw new Error('Unable to find path to phantomjs, please run npm install and find the relative path for your system. Current location: [' + __dirname + ']' +
        'Environment: [' + os.type() + ',' +
        os.platform() + ',' +
        os.arch() + ',' +
        os.release() + ']');
}

...

Note that because this file is being slurped in and eval-ed by build-config/selenium_webdriver.js, that phantomLoc is assigned build-config/ (by __dirname). Hence, this grunt-selenium-webdriver task is looking for phantomjs in the main node_modules/ for readium-js-viewer, which is in fact not there anymore, as it was replaced by phantomjs-prebuilt in the main package.json here: 2dc9bb0ee2d22f7644e60ff291afe221c576b136.

A recent commit to grunt-selenium-webdriver seems to fix this problem:
e9f63af80c8e6738b00812370c6f1ba13c5dc817. This commit is for version 0.2.530, though, and the NPM repository has only the 0.2.482 package. It is unclear when the 0.2.530 NPM release will happen: Github issue #29: Please release 0.2.530.

Temporary workaround options:

• Add a step to npm prepare that creates a symlink node_modules/grunt-selenium-webdriver/node_modules/phantomjs -> node_modules/phantomjs:

    "prepare": "npm run prepare:submodule && npm run prepare:local && cd node_modules && ln -s ../node_modules/grunt-selenium-webdriver/node_modules/phantomjs phantomjs",

This will match this grunt-selenium-webdriver condition:

if ( fs.existsSync( phantomLoc + "/../node_modules/phantomjs/bin/phantomjs") ) {

• Change the grunt-selenium-webdriver dependency in package.json from this:

    "grunt-selenium-webdriver": "latest",

...to this:

    "grunt-selenium-webdriver": "levexis/grunt-selenium-webdriver#e9f63af80c8e6738b00812370c6f1ba13c5dc817",

I've tested both these workarounds, and will submit a PR for the first, as that seems to be the more appropriate solution.

Even after using these workarounds, I was unable to get tests working. Please note that there is currently a problem of integration between Selenium and Firefox:

• Firefox driver in 2.53.4 not working with Firefox 48 #2559
• Selenium 2.53 not working on Firefox 47
• Selenium 2.53.1 does not work on FireFox 48 (duplicate)
• Firefox 47 - Unable to connect to host 127.0.0.1 on port 7055 after 45000 ms. #2110
• Firefox Developer Edition (47.0a2) crashes using Java Selenium 2.53.0 (and 2.52.0, 2.48.2) #1862

The solution at the moment seems to be either to use Firefox 47.0.1 (not 47.0 or 48.0), or to use the new, in-development GeckoDriver a.k.a. Marionette. At the time I did research on this driver, it didn't seem like it was robust enough for serious use.
I've been using a parallel Firefox 47.0.1 with success with Selenium scripts for other projects. However, because the Readium tests use grunt-selenium-webdriver, I was unable to figure out a way to specify an alternative Firefox or Selenium.

The Chrome tests also seem to be failing. As I looked over the tests/ directory I noticed that things seemed to be in flux, so decided not to pursue this any farther for the time being.

[danielweck]

Heads-up:
a really dirty workaround, currently in the develop branch:
2c349d0

[danielweck]

Unfortunately, I believe that the symlink fix is not portable (notably on MS Windows):
ln -s ../node_modules/grunt-selenium-webdriver/node_modules/phantomjs phantomjs

[danielweck]

So, grunt-selenium-webdriver is not an active project, and the version deployed at NPM is behind the GitHub master branch.
It looks like WebDriver.io would make it easier to launch Readium tests from NodeJS:
https://github.com/webdriverio/webdriverio/

[danielweck]

I updated the grunt-selenium-webdriver dependency to match the tip of their GitHub master branch:
096fa4c
Because ReadiumJS currently has tests designed to execute in "real" browsers (user-installed, accessed through WebDriver instances, rather than based on old WebKit / PhantomJS), I am closing this issue as resolved.

[da70]

Yes, I've been using WebDriver.io for writing tests against our ReadiumJS build, and so far have been happy with it. It has an optional test runner that is synchronous by default, which I find convenient for writing tests. I also like to use real browsers, too, as opposed to PhantomJS and the like. Thanks for following up.