PrometheusMetricsConfig can't run in distributed mode

[GaspardZeBear]

Hi,

Bug ? : Jmeter 5.2.1, plugin 0.6.0

Issue : When remotly launched, the prometheus plugin 0.6.0 generates an exception when trying to increment in jsr223 script counters defined as metrics . It works when launched locally. The counters defined into a prometheus listener work in both cases..

Illustration with the plugin demo scenario :

https://github.com/johrstrom/jmeter-prometheus-plugin/blob/master/docs/examples/simple_prometheus_example.jmx

Hereunder details

Thanks for help ! Your plugin is great

Local launch

curl -vvv http://local:9270 2>&1 | grep jsr223 | grep total
< # HELP jsr223_can_fail_total success ratio of the can_fail_sampler
< # TYPE jsr223_can_fail_total counter
< jsr223_can_fail_total 16.0
< # HELP jsr223_assertions_total default help string
< # TYPE jsr223_assertions_total counter
< jsr223_assertions_total{label="less than 2kB",} 16.0
< # HELP jsr223_count_total the total number of samplers
< # TYPE jsr223_count_total counter
< jsr223_count_total{label="can_fail_sampler",} 16.0
< # HELP jsr223_animals_total default help string
< # TYPE jsr223_animals_total counter
< jsr223_animals_total{color="G",size="S",mammal="N",} 1.0
< jsr223_animals_total{color="R",size="L",mammal="N",} 1.0
< jsr223_animals_total{color="R",size="S",mammal="Y",} 2.0
< jsr223_animals_total{color="B",size="M",mammal="Y",} 1.0
< jsr223_animals_total{color="R",size="M",mammal="Y",} 1.0
< # HELP jsr223_success_total the total number of successful samplers
< # TYPE jsr223_success_total counter
< jsr223_success_total{label="can_fail_sampler",} 10.0

Remote launch : jsr223_animals_total not present

curl -vvv http://remote: 9270 2>&1 | grep jsr223 | grep total
< # HELP jsr223_can_fail_total success ratio of the can_fail_sampler
< # TYPE jsr223_can_fail_total counter
< jsr223_can_fail_total 32.0
< # HELP jsr223_count_total the total number of samplers
< # TYPE jsr223_count_total counter
< jsr223_count_total{label="can_fail_sampler",} 32.0
< # HELP jsr223_success_total the total number of successful samplers

Traces

2020-10-21 08:50:20,174 WARN c.g.j.l.PrometheusListener: Prometheus server has not yet been initialized, doing it now
2020-10-21 08:50:20,175 ERROR c.g.j.c.CollectorElement: Didn't create new collector because of error,
java.lang.NullPointerException: null
at com.github.johrstrom.collector.CollectorElement.makeNewCollectors(CollectorElement.java:70) [prometheus-plugin-0.6.0.jar:?]
at com.github.johrstrom.config.PrometheusMetricsConfig.testStarted(PrometheusMetricsConfig.java:44) [prometheus-plugin-0.6.0.jar:?]
at com.github.johrstrom.config.PrometheusMetricsConfig.testStarted(PrometheusMetricsConfig.java:59) [prometheus-plugin-0.6.0.jar:?]
at org.apache.jmeter.engine.StandardJMeterEngine.notifyTestListenersOfStart(StandardJMeterEngine.java:208) [ApacheJMeter_core.jar:5.2.1]
at org.apache.jmeter.engine.StandardJMeterEngine.run(StandardJMeterEngine.java:381) [ApacheJMeter_core.jar:5.2.1]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_262]
......
2020-10-21 08:50:23,367 ERROR o.a.j.p.j.s.JSR223Sampler: Problem in JSR223 script JSR223 Sampler, message: javax.script.ScriptException: java.lang.NullPointerException: Cannot invoke method labels() on null object
javax.script.ScriptException: java.lang.NullPointerException: Cannot invoke method labels() on null object
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:324) ~[groovy-all-2.4.16.jar:2.4.16]
at org.codehaus.groovy.jsr223.GroovyCompiledScript.eval(GroovyCompiledScript.java:72) ~[groovy-all-2.4.16.jar:2.4.16]
at javax.script.CompiledScript.eval(CompiledScript.java:92) ~[?:1.8.0_262]
at org.apache.jmeter.util.JSR223TestElement.processFileOrScript(JSR223TestElement.java:223) ~[ApacheJMeter_core.jar:5.2.1]
at org.apache.jmeter.protocol.java.sampler.JSR223Sampler.sample(JSR223Sampler.java:71) [ApacheJMeter_java.jar:5.2.1]
at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:627) [ApacheJMeter_core.jar:5.2.1]
at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:551) [ApacheJMeter_core.jar:5.2.1]
at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:490) [ApacheJMeter_core.jar:5.2.1]
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:257) [ApacheJMeter_core.jar:5.2.1]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_262]
Caused by: java.lang.NullPointerException: Cannot invoke method labels() on null object
at org.codehaus.groovy.runtime.NullObject.invokeMethod(NullObject.java:91) ~[groovy-all-2.4.16.jar:2.4.16]
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:47) ~[groovy-all-2.4.16.jar:2.4.16]
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) ~[groovy-all-2.4.16.jar:2.4.16]
at org.codehaus.groovy.runtime.callsite.NullCallSite.call(NullCallSite.java:34) ~[groovy-all-2.4.16.jar:2.4.16]
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) ~[groovy-all-2.4.16.jar:2.4.16]
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116) ~[groovy-all-2.4.16.jar:2.4.16]
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:144) ~[groovy-all-2.4.16.jar:2.4.16]
at Script41.run(Script41.groovy:10) ~[?:?]
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:321) ~[groovy-all-2.4.16.jar:2.4.16]

[johrstrom]

Thanks for the ticket! It's strange that this would happen to you. Looking back through the code, if this were an issue it would mean nobody could run distributed tests, which doesn't seem to be the case.

Can you comment out the skip tg and see if it runs in a distributed mode then? I think there could be an issue with the fact that there are multiple listeners and the way JMeter initialized everything on the client side before starting the test on the remove server.

[GaspardZeBear]

Hi ..
I have disabled the skip tg and launched in distributed mode, the issue is still present

2020-10-21 17:12:11,339 ERROR c.g.j.c.CollectorElement: Didn't create new collector because of error,
java.lang.NullPointerException: null
at com.github.johrstrom.collector.CollectorElement.makeNewCollectors(CollectorElement.java:70) [prometheus-plugin-0.6.0.jar:?]
at com.github.johrstrom.config.PrometheusMetricsConfig.testStarted(PrometheusMetricsConfig.java:44) [prometheus-plugin-0.6.0.jar:?]
at com.github.johrstrom.config.PrometheusMetricsConfig.testStarted(PrometheusMetricsConfig.java:59) [prometheus-plugin-0.6.0.jar:?]
at org.apache.jmeter.engine.StandardJMeterEngine.notifyTestListenersOfStart(StandardJMeterEngine.java:208) [ApacheJMeter_core.jar:5.2.1]
at org.apache.jmeter.engine.StandardJMeterEngine.run(StandardJMeterEngine.java:381) [ApacheJMeter_core.jar:5.2.1]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_262]
2020-10-21 17:12:11,781 INFO o.a.j.e.StandardJMeterEngine: Starting ThreadGroup: 1 : listener tg
2020-10-21 17:12:11,781 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group listener tg.
2020-10-21 17:12:11,781 INFO o.a.j.e.StandardJMeterEngine: Thread will continue on error
2020-10-21 17:12:11,781 INFO o.a.j.t.ThreadGroup: Starting thread group... number=1 threads=1 ramp-up=1 delayedStart=false
2020-10-21 17:12:11,781 INFO o.a.j.t.ThreadGroup: Started thread group number 1
2020-10-21 17:12:11,781 INFO o.a.j.e.StandardJMeterEngine: Starting ThreadGroup: 2 : config tg
2020-10-21 17:12:11,781 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group config tg.
2020-10-21 17:12:11,781 INFO o.a.j.e.StandardJMeterEngine: Thread will continue on error
2020-10-21 17:12:11,781 INFO o.a.j.t.ThreadGroup: Starting thread group... number=2 threads=1 ramp-up=1 delayedStart=false
2020-10-21 17:12:11,782 INFO o.a.j.t.JMeterThread: Thread started: 10.0.95.21-listener tg 1-1
2020-10-21 17:12:11,783 INFO o.a.j.t.ThreadGroup: Started thread group number 2
2020-10-21 17:12:11,783 INFO o.a.j.e.StandardJMeterEngine: All thread groups have been started
2020-10-21 17:12:11,783 INFO o.a.j.t.JMeterThread: Thread started: 10.0.95.21-config tg 2-1
2020-10-21 17:12:11,814 INFO o.a.j.p.j.s.J.can_fail_sampler: sleeping for 102 ms
2020-10-21 17:12:11,939 INFO o.a.j.p.j.s.J.can_fail_sampler: sleeping for 1371 ms
2020-10-21 17:12:13,311 INFO o.a.j.p.j.s.J.can_fail_sampler: sleeping for 100 ms
2020-10-21 17:12:13,412 INFO o.a.j.p.j.s.J.can_fail_sampler: sleeping for 207 ms
2020-10-21 17:12:13,620 INFO o.a.j.p.j.s.J.can_fail_sampler: sleeping for 1283 ms
2020-10-21 17:12:14,797 ERROR o.a.j.p.j.s.JSR223Sampler: Problem in JSR223 script JSR223 Sampler, message: javax.script.ScriptException: java.lang.NullPointerException: Cannot invoke method labels() on null object
javax.script.ScriptException: java.lang.NullPointerException: Cannot invoke method labels() on null object

[johrstrom]

OK, this could be due to me setting config elements as the running version, an API I don't fully understand and probably just blindly copied from another config element as an example.

jmeter-prometheus-plugin/src/main/java/com/github/johrstrom/config/PrometheusMetricsConfig.java

Line 43 in cbd51a1

this.setRunningVersion(true);

You can probably comment PrometheusMetricsConfig and it should run. Meaning you've discovered some bug in Prometheus Metrics config elements and how they can't run in distributed mode. Apparently I invented it for my use case which was functional testing and didn't need to run in distributed mode and nobody else uses this 😞 .

In any case, this does seem to be a bug with the config element. You seem to be just testing the workflow out, so commenting that should element should allow you to move forward, though it removes that functionality should you need it.

[GaspardZeBear]

Ok thanks. I'll try to have a more detailed look at this issue ... but I need some time to enter the internals of jmeter.
Regards