Topics

What failed?


Randall Wood <rhwood@...>
 

From the following output, which “feature” (or whatever the language of the integration tests uses) failed?

rhwood@Calculus JMRI % mvn test-compile failsafe:integration-test
[INFO] Scanning for projects...
[INFO
[INFO] ---------------------------< org.jmri:jmri >----------------------------
[INFO] Building JMRI 4.19.7-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO
[INFO] --- build-helper-maven-plugin:3.0.0:timestamp-property (timestamp-property) @ jmri ---
[INFO
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven) @ jmri ---
[INFO
[INFO] --- jacoco-maven-plugin:0.8.5:prepare-agent (prepare-agent) @ jmri ---
[INFO] surefire.argLine set to -javaagent:/Users/rhwood/.m2/repository/org/jacoco/org.jacoco.agent/0.8.5/org.jacoco.agent-0.8.5-runtime.jar=destfile=/Users/rhwood/Developer/JMRI/JMRI/rhwood/JMRI/target/jacoco-unit.exec,append=true
[INFO
[INFO] --- jacoco-maven-plugin:0.8.5:prepare-agent (prepare-agent-integration) @ jmri ---
[INFO] failsafe.argLine set to -javaagent:/Users/rhwood/.m2/repository/org/jacoco/org.jacoco.agent/0.8.5/org.jacoco.agent-0.8.5-runtime.jar=destfile=/Users/rhwood/Developer/JMRI/JMRI/rhwood/JMRI/target/jacoco-it.exec,append=true
[INFO
[INFO] --- javacc-maven-plugin:2.6:jjtree-javacc (default) @ jmri ---
[INFO] Skipping - all parsers are up to date
[INFO
[INFO] --- buildnumber-maven-plugin:1.4:create-metadata (generate) @ jmri ---
[INFO] ShortRevision tag detected. The value is '7'.
[INFO] Executing: /bin/sh -c cd '/Users/rhwood/Developer/JMRI/JMRI/rhwood/JMRI' && 'git' 'rev-parse' '--verify' '--short=7' 'HEAD'
[INFO] Working directory: /Users/rhwood/Developer/JMRI/JMRI/rhwood/JMRI
[INFO
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ jmri ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2397 resources
[INFO] Copying 1 resource
[INFO
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ jmri ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 4736 source files to /Users/rhwood/Developer/JMRI/JMRI/rhwood/JMRI/target/classes
[INFO] /Users/rhwood/Developer/JMRI/JMRI/rhwood/JMRI/java/src/jmri/util/JmriJFrame.java: Some input files use or override a deprecated API.
[INFO] /Users/rhwood/Developer/JMRI/JMRI/rhwood/JMRI/java/src/jmri/util/JmriJFrame.java: Recompile with -Xlint:deprecation for details.
[INFO
[INFO] --- build-helper-maven-plugin:3.0.0:add-test-source (add-test-source) @ jmri ---
[INFO] Test Source directory: /Users/rhwood/Developer/JMRI/JMRI/rhwood/JMRI/java/acceptancetest/step_definitions added.
[INFO
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ jmri ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 469 resources
[INFO
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ jmri ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 4310 source files to /Users/rhwood/Developer/JMRI/JMRI/rhwood/JMRI/target/test-classes
[INFO] /Users/rhwood/Developer/JMRI/JMRI/rhwood/JMRI/java/test/jmri/jmrit/operations/OperationsTestCase.java: Some input files use or override a deprecated API.
[INFO] /Users/rhwood/Developer/JMRI/JMRI/rhwood/JMRI/java/test/jmri/jmrit/operations/OperationsTestCase.java: Recompile with -Xlint:deprecation for details.
[INFO
[INFO] --- maven-failsafe-plugin:2.22.0:integration-test (default-cli) @ jmri ---
[INFO
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running jmri.implementation.DefaultCabSignalIT
Using org.netbeans.jemmy.drivers.APIDriverInstaller driver installer
Warning: Nashorn engine is planned to be removed from a future JDK release
Warning: Nashorn engine is planned to be removed from a future JDK release
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.249 s - in jmri.implementation.DefaultCabSignalIT
[INFO] Running jmri.jmrix.can.cbus.CbusCabSignalIT
Warning: Nashorn engine is planned to be removed from a future JDK release
Warning: Nashorn engine is planned to be removed from a future JDK release
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.459 s - in jmri.jmrix.can.cbus.CbusCabSignalIT
[INFO] Running jmri.jmrix.loconet.LnCabSignalIT
Warning: Nashorn engine is planned to be removed from a future JDK release
Warning: Nashorn engine is planned to be removed from a future JDK release
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.396 s - in jmri.jmrix.loconet.LnCabSignalIT

0 Scenarios
0 Steps
0m9.815s


[INFO] Running JUnit Vintage
.Warning: Nashorn engine is planned to be removed from a future JDK release
....Warning: Nashorn engine is planned to be removed from a future JDK release
....Warning: Nashorn engine is planned to be removed from a future JDK release
....Warning: Nashorn engine is planned to be removed from a future JDK release
....Warning: Nashorn engine is planned to be removed from a future JDK release
....Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile LocoNet_Simulator.3f1121a3 [main] apps.Apps.<init>()
ERROR - Did not find class "Open JMRI System Console" [main] apps.startup.StartupActionModelUtil.isSystemConnectionAction()
F.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile EasyDcc__Simulator.3f1121a3 [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile TMCC_Simulator.3f0b414a [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile Prevent_Init_Loop.3f01644a [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile Sprog_Simulator.3d40b3b8 [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile LocoNet_Simulator.3f1121a3 [main] apps.gui3.Apps3.configureProfile()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile EasyDcc__Simulator.3f1121a3 [main] apps.gui3.Apps3.configureProfile()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile TMCC_Simulator.3f0b414a [main] apps.gui3.Apps3.configureProfile()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile Prevent_Init_Loop.3f01644a [main] apps.gui3.Apps3.configureProfile()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile Grapevine_Simulator.3f339028 [main] apps.gui3.Apps3.configureProfile()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile Sprog_Simulator.3d40b3b8 [main] apps.gui3.Apps3.configureProfile()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile LocoNet_Simulator.3f1121a3 [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile EasyDcc__Simulator.3f1121a3 [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile TMCC_Simulator.3f0b414a [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile Prevent_Init_Loop.3f01644a [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile Grapevine_Simulator.3f339028 [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile Sprog_Simulator.3d40b3b8 [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile LocoNet_Simulator.3f1121a3 [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile EasyDcc__Simulator.3f1121a3 [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile TMCC_Simulator.3f0b414a [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile Prevent_Init_Loop.3f01644a [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile Grapevine_Simulator.3f339028 [main] apps.Apps.<init>()
.Warning: Nashorn engine is planned to be removed from a future JDK release
...INFO  - Starting with profile Sprog_Simulator.3d40b3b8 [main] apps.Apps.<init>()
...INFO  - Starting with profile LocoNet_Simulator.3f1121a3 [main] apps.AppsBase.configureProfile()
...INFO  - Starting with profile EasyDcc__Simulator.3f1121a3 [main] apps.AppsBase.configureProfile()

Failed scenarios:
java/acceptancetest/features/apps/ApplicationStart.feature:28 # Application Start

30 Scenarios (1 failed, 29 passed)
118 Steps (118 passed)
3m49.789s

java.lang.AssertionError: Unexpected ERROR or higher messages emitted: Did not find class "Open JMRI System Console"
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.junit.Assert.assertFalse(Assert.java:65)
at jmri.util.JUnitUtil.tearDown(JUnitUtil.java:344)
at apps.ApplicationTestAcceptanceSteps.lambda$new$7(ApplicationTestAcceptanceSteps.java:98)
at cucumber.runtime.java8.Java8HookDefinition.lambda$execute$0(Java8HookDefinition.java:51)
at cucumber.runtime.Timeout.timeout(Timeout.java:16)
at cucumber.runtime.java8.Java8HookDefinition.execute(Java8HookDefinition.java:47)
at cucumber.runner.HookDefinitionMatch.runStep(HookDefinitionMatch.java:16)
at cucumber.runner.TestStep.executeStep(TestStep.java:65)
at cucumber.runner.TestStep.run(TestStep.java:50)
at cucumber.runner.TestCase.run(TestCase.java:50)
at cucumber.runner.Runner.runPickle(Runner.java:49)
at cucumber.runtime.junit.PickleRunners$NoStepDescriptions.run(PickleRunners.java:146)
at cucumber.runtime.junit.FeatureRunner.runChild(FeatureRunner.java:68)
at cucumber.runtime.junit.FeatureRunner.runChild(FeatureRunner.java:23)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at cucumber.runtime.junit.FeatureRunner.run(FeatureRunner.java:73)
at cucumber.api.junit.Cucumber.runChild(Cucumber.java:123)
at cucumber.api.junit.Cucumber.runChild(Cucumber.java:65)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at cucumber.api.junit.Cucumber$RunCucumber.evaluate(Cucumber.java:147)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:43)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:82)
at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:73)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:132)
at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:142)
at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:117)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:383)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:344)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:417)


[ERROR] Tests run: 31, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 229.899 s <<< FAILURE! - in JUnit Vintage
[ERROR] Application Start  Time elapsed: 7.94 s  <<< FAILURE!
java.lang.AssertionError: Unexpected ERROR or higher messages emitted: Did not find class "Open JMRI System Console"

[INFO
[INFO] Results:
[INFO
[ERROR] Failures: 
[ERROR] apps.RunCucumberIT.Application Start
[INFO]   Run 1: PASS
[INFO]   Run 2: PASS
[INFO]   Run 3: PASS
[INFO]   Run 4: PASS
[INFO]   Run 5: PASS
[ERROR]   Run 6: RunCucumberIT Unexpected ERROR or higher messages emitted: Did not find class "Open JMRI System Console"
[INFO]   Run 7: PASS
[INFO]   Run 8: PASS
[INFO]   Run 9: PASS
[INFO]   Run 10: PASS
[INFO]   Run 11: PASS
[INFO]   Run 12: PASS
[INFO]   Run 13: PASS
[INFO]   Run 14: PASS
[INFO]   Run 15: PASS
[INFO]   Run 16: PASS
[INFO]   Run 17: PASS
[INFO]   Run 18: PASS
[INFO]   Run 19: PASS
[INFO]   Run 20: PASS
[INFO]   Run 21: PASS
[INFO]   Run 22: PASS
[INFO]   Run 23: PASS
[INFO]   Run 24: PASS
[INFO]   Run 25: PASS
[INFO]   Run 26: PASS
[INFO]   Run 27: PASS
[INFO]   Run 28: PASS
[INFO]   Run 29: PASS
[INFO]   Run 30: PASS
[INFO
[INFO
[ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0
[INFO
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  06:59 min
[INFO] Finished at: 2020-05-21T06:35:29-04:00
[INFO] ------------------------------------------------------------------------
rhwood@Calculus JMRI % 

(This kind of stuff needs to be significantly more explicit—I can’t tell where I am supposed to begin looking into a failure with these integration tests, and without that kind of input, I can only disable these tests and ignore them as completely untraceable and therefor meaningless.)

Randall


Paul Bender
 




On May 21, 2020, at 6:55 AM, Randall Wood via groups.io <rhwood@...> wrote:

From the following output, which “feature” (or whatever the language of the integration tests uses) failed?


From the log, the scenario that failed was:

Failed scenarios:
java/acceptancetest/features/apps/ApplicationStart.feature:28 # Application Start


Which looking at line 28 in the specified feature file is:

   | apps.DecoderPro.DecoderPro | java/test/apps/PanelPro/profiles/LocoNet_Simulator | DecoderPro | DecoderPro version |

Which is trying to start DecoderPro with the profile located in java/test/apps/PanelPro/profiles/LocoNet_Simulator.

More specifically, The test fails because that configuration is trying to open the system console and can’t find that as a startup action:

java.lang.AssertionError: Unexpected ERROR or higher messages emitted: Did not find class "Open JMRI System Console"

Which seems like it could be an indicator of a real bug.

Paul