mercredi 5 août 2015

Nunit-Console fails some tests when with configuration file

Nunit-console is running on our build server and some tests concerning registry access are failing on the build server.

The command line used by the build server is similar to this :

nunit-console.exe AllTests.nunit -xml=TestResult.xml /nologo /nodots

When tested directly on the build server it produces this output:

ProcessModel: Default    DomainUsage: Default
Execution Runtime: net-3.5

Tests run: 15, Errors: 0, Failures: 4, Inconclusive: 0, Time: 0.995385377191794 seconds
  Not run: 2, Invalid: 0, Ignored: 2, Skipped: 0

The 4 failures here correspond to the failures reported by the CI build.

The content of the AllTests.nunit is similar to this:

<?xml version="1.0" encoding="utf-8"?>
<NUnitProject>
   <Settings activeconfig="Release"/>
   <Config name="Release">
      <assembly path="Assembly1\bin\Release\Assembly1.dll"/>
      <assembly path="Assembly2\bin\Release\Assembly2.dll"/>
      <assembly path="Assembly3\bin\Release\Assembly3.dll"/>
   </Config>
</NUnitProject>

Nevertheless, when I run nunit-console with the assembly where concerned tests are failing, I have no more errors.

nunit-console.exe assembly1\bin\Release\assembly1.dll /nologo /nodots

Output is as folow:

ProcessModel: Default    DomainUsage: Single
Execution Runtime: net-3.5

Tests run: 8, Errors: 0, Failures: 0, Inconclusive: 0, Time: 0.134831871089761 seconds
  Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0

The question is why test results vary when run with assembly or with the nunit-console configuration file ?

Aucun commentaire:

Enregistrer un commentaire