With Travis - Behat fails, selenium not running.

[roperto]

It seems like a Java problem (surprise surprise)

Workaround: add sudo: required to .travis.yml

[stronk7]

As commented @ 4f97e21, I've tried with sudo: required and it helped with behat executions but, at the same time, introduced a really annoying behavior with random failures all over the place. See:

• https://travis-ci.org/stronk7/moodle-local_codechecker/builds/330949987

So, particularly, I'm back to sudo: false + disabled behat (it's not critical in my plugin). And at least that way I get more consistent results. See:

• https://travis-ci.org/stronk7/moodle-local_codechecker/builds/331009937

Just sharing, not really sure which the selenium / java problem is. Annoying. Ciao :-)

[roperto]

Hi @stronk7

I am also getting inconsistent problems updating packages and such.

I disable while developing, but when merging I still want the peace of mind of behat tests passing

Please let us know if you find a workaround, Eloy ...

Thanks

[mackensen]

I'm seeing the same issues with my plugins. Behat's important to me, so I'm setting sudo: true and re-running failures. This is with version 2 of the plugin.

[polothy]

Do you guys have more example of build failures? Eloy's failure has nothing to do with Behat, it's failing to install Java8 (oracle-java8-installer, oracle-java8-set-default) because for whatever reason, some of the builds didn't find oracle-java8-set-default package, while others did. This is only needed for the mustache linting IIRC. You could try Java9, but betting that wont work.

[roperto]

hi @mrmark .. I have more examples but that's exactly the same problem.

We currently have a custom docker image to run moodle-plugin-ci to be used in our GitLab, but it has a lot of custom code.

If we make it more flexible do you think it would be a approach to running it Travis? Hopefully would speedup part of initialisation (it would only pull changes in Moodle since the image was built, composer init, npms, etc) and less prone to Travis infrastructure changes. The initialisation process would still run as phpunit/behat init is required after the plugin is added.

If I went ahead, would you be interested in that or do you think its a dead end?

[mackensen]

@mrmark Sure. This is for the https://github.com/LafColITS/moodle-tool_hidecourses plugin:

• Selenium doesn't start with sudo: false: https://travis-ci.org/LafColITS/moodle-tool_hidecourses/builds/327216012
• Selenium starts with sudo: true: https://travis-ci.org/LafColITS/moodle-tool_hidecourses/builds/329856531

[scara]

@mackensen @mrmark here is my working example w/ sudo: false: https://travis-ci.org/scara/moodle-local_twittercard/builds/330854428 .

HTH,
Matteo

[stronk7]

Hi,

just note that only @javascript scenarios require selenium + real browser to be working, so it's normal, for people not having such a tag to get behat passing.

In fact it was working ok for my plugin, but because of a little restriction with checkboxes I needed to add the @javascript tag to one of my features.. and there is where the problem started to happen consistently.

Just to explain/clarify why tests seem to work to some people without problem. It's not magic or luck. Only @javascript scenarios are affected by selenium not working

[stronk7]

OT: just came to my brain... could it be some problem with IPv4/IPv6 ? I've had a flash that some time ago I had to force IPv4 in some places (-Djava.net.preferIPv4Stack=true) because not all the stack was able to work with IPv6 properly (and the system insisted to use IPv6 by default).

As said... just a blind shot!

[polothy]

If we make it more flexible do you think it would be a approach to running it Travis? Hopefully would speedup part of initialisation (it would only pull changes in Moodle since the image was built, composer init, npms, etc) and less prone to Travis infrastructure changes. The initialisation process would still run as phpunit/behat init is required after the plugin is added.

That was proposed in #57 to use moodle-docker, but I was worried about performance. I'd likely only want to move to a core supported/maintained container if we ever went in that direction. I do agree though, there have been a lot of build breaks due to Travis updates, which is too bad.

Thanks everyone for all the examples, feedback!

[scara]

It's not magic or luck. Only @javascript scenarios are affected by selenium not working

TNX @stronk7!
My bad for not thinking at it: shame on Matteo 😊 !

[polothy]

Having lots of troubling figuring out the problem. Have a build I'm debugging here: https://travis-ci.org/moodlerooms/moodle-plugin-ci/jobs/334948327

• PHP Server seems to be OK: https://travis-ci.org/moodlerooms/moodle-plugin-ci/jobs/334948327#L615
• Behat failure: https://travis-ci.org/moodlerooms/moodle-plugin-ci/jobs/334948327#L904
• And output from Selenium command: https://travis-ci.org/moodlerooms/moodle-plugin-ci/jobs/334948327#L1207

Anyone know what's going on? Any guesses?

[roperto]

Hi @mrmark , as you said PHP server (behat site) looks fine.

I'd investigate towards the selenium server, I am guessing a driver/selenium problem from "UnsupportedCommandException" at https://travis-ci.org/moodlerooms/moodle-plugin-ci/jobs/334948327#L1278 -- maybe a mismatch of firefox version and selenium driver?

[scara]

@mrmark @roperto Bad request looks like coming from a 400 HTTP Status: what about SeleniumHQ/selenium#2074 ?

HTH,
Matteo

[polothy]

Thanks @scara, that might be the problem, I commented in travis-ci/travis-ci#8711

There is cat /etc/hosts with sudo false/true:

• sudo false: https://travis-ci.org/moodlerooms/moodle-plugin-ci/builds/335223258#L487
• sudo true: https://travis-ci.org/moodlerooms/moodle-plugin-ci/builds/335233134#L498

If it is indeed a problem with /etc/hosts then there is nothing we can do about it because in order to fix it, we would need sudo, so might as well just use sudo: true in your build file until Travis fixes the container.

[scara]

Great @mrmark!
At the end @stronk7 was right about IPv4: we need to force the binding to IPv4 since IPv6 support is broken which means -Djava.net.preferIPv4Stack=true on the Java command line and php -S 127.0.0.1:8080 to force the builtin-in server bind to IPv4 i.e. no localhost here.
You could try first w/ PHP and then if it won't work force IPv4 at Java side too.

For the record see another example using sudo: required - which is slower among the other cons -, read carefully the commit message: mrkn/ruby@ebffe78.

HTH,
Matteo

[polothy]

Thanks for the continued feedback, but doesn't appear to work unfortunately, https://travis-ci.org/moodlerooms/moodle-plugin-ci/builds/335718087

[scara]

Hi @mrmark,
to recap, the failure is now related to the new settings towards a "pure IPv4 net env":

• 127.0.0.1 instead of localhost: 1f28688 and fe9db9a. Unrelated but 127.0.0.1 is missing here: https://github.com/moodlerooms/moodle-plugin-ci/blob/fe9db9abb0c848998c388cfa7173b16939649b6f/src/Command/BehatCommand.php#L140
• -Djava.net.preferIPv4Stack=true: 071895a
• -Djava.net.preferIPv6Addresses=false: ffdd994

Before leaving this path, what about give -Djava.net.preferIPv4Addresses=true also a try?
That could exclude any IPv6 attempt since from your logs it's pretty clear that the (first) failure happens in opening a session against Selenium.

After more digging the web I found SeleniumHQ/selenium-google-code-issue-archive#3280 and SeleniumHQ/selenium@1ada6dd: it looks like there's no need to force IPv4 but IIUC to force the hosts preference to localhost.
Guessing that Bad Request here is coming from a mismatching between the Host HTTP header coming with the request and the info taken by the Firefox built-in web server.
If that is true, I'm not sure how to test it since we need to inject a value for the (new) pref webdriver_firefox_allowed_hosts.

HTH,
Matteo

[scara]

FYI yesterday I've also filed travis-ci/travis-ci#9167 which could solve our issue here if the Bad Request is actually due to the archived issue I've found.

At the moment I'm not sure if we'd need some IPv4 tuning after that fix: will see.

[polothy]

Thanks @scara for your help, I think I'll wait for the Travis issue to get some traction before I try anything else on this.

[polothy]

With travis-ci/travis-ci#9167 fixed, still doesn't work :(

Re-ran my master branch with no luck and same for the debugging one. /etc/hosts now looks like this:

::1	 ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
172.17.0.4	travis-job-moodlerooms-moodle-plugin-335718089.travisci.net travis-job-moodlerooms-moodle-plugin-335718089
127.0.0.1 localhost localhost 

[scara]

Hi @mrmark,
I'm out of smart ideas than trying to revert fe9db9a#diff-0 to let Selenium be accessed via localhost and not via 127.0.0.1, since we found that this didn't helped in the attempt of forcing IPv4 and maybe this revert together with the recent change in /etc/hosts could help the resolution of the host in the Firefox built-in web server.

[polothy]

Minor update:

First, as of 2.3.0, recommending sudo: true for builds that run Behat, which I think most of you are already doing.

Second, think I sort of verified that the problem in the container build system is with /etc/hosts, in particular, this line: ::1 ip6-localhost ip6-loopback. My funky reproduction was to do this:

1. I switched my Travis file to use sudo: true, build passes.
2. I edited the /etc/hosts to add ::1 ip6-localhost ip6-loopback, build fails.
3. I edited the /etc/hosts to add ::1 localhost ip6-localhost ip6-loopback, build passes.

From what I can tell, ::1 localhost was removed in travis-ci/travis-ci#8780. Unsure if we will be able to get it back.

[mudrd8mz]

Thanks @mrmark for the update. I am affected by this too (e.g. https://travis-ci.org/mudrd8mz/moodle-block_course_contents/jobs/336000695).