Running tests

AppVeyor has tight integration with the following testing frameworks:

This does not, however, mean you can’t run your own favourite testing frameworks (like Jasmine, JSLint, Pester) within AppVeyor, but rather that AppVeyor provides automatic discovery, execution and real-time reporting for the frameworks listed above.

In most cases you are good to go with the default Auto testing mode. This mode tells AppVeyor to recursively search the build folder for test assemblies referencing known frameworks, run tests with corresponding test runners, and push results back to the build console.

Selecting assemblies to test

By default, Auto mode scans the entire build folder. For large projects this could be a time-consuming operation.

In the Test assemblies box you can specify one of the following:

  1. Exact path to an assembly relative to build root folder, for example myproject\bin\debug\myassembly.dll.
  2. Assembly file name without a path - this case AppVeyor will perform recursive search of all assemblies with the given name.
  3. Wildcard. If wildcard is used it should be full relative path. For example, to scan all folders recursively for test-assembly.dll specify **\*.test-assembly.dll.

To match an assembly in a specific folder:


or if build configuration is set in environment variable:


You can substitute any existing environment variable

To match all assemblies ending with .tests.dll:


Configuring tests in appveyor.yml:

  # assemblies to test - optional
    - test-assembly-A.dll
    - '**\*.tests.dll'

  # categories to test - optional
    - A
    - B

# run custom scripts before tests
  - script 1
  - script 2

# run custom scripts after tests
  - script 1
  - script 2

If assembly path in appveyor.yml starts with * surround the value with single quotes to make YAML parser happy:

    - '**\*.tests.dll'

See appveyor.yml reference for complete syntax.

Selecting categories to test

By default, AppVeyor runs all tests from found assemblies.

You can include or exclude certain test categories from tests run on Tests tab of project settings on in appveyor.yml.

To run tests from only specified categories:

      - A
      - B

To run tests from all categories except specified ones:

      - A
      - B

Visual Studio unit tests (C#)

Applying category to a test method:

[TestMethod, TestCategory("A")]
public void TestA()
} (C#)

Applying category to a test method:

[Fact, Trait("Category", "A")]
public void MyTest()

NUnit (C#)

Applying category to a test fixture:

[TestFixture, Category("LongRunning")]
public class LongRunningTests

Applying category to a test method:

public class SuccessTests
    [Test, Category("Long")]
    public void VeryLongTest()

MSpec (C#)

Applying category to MSpec test with Tags attribute:

[Subject(typeof(Account), "Funds transfer")]
public class when_transferring_between_two_accounts : AccountSpecs

Calling test runners from your custom scripts

The AppVeyor build environment includes runners for MSTest, NUnit and frameworks that are integrated with the build console to push real-time results while running tests.

You can install and use your own build runners for frameworks above, communication with the build console must be done via text output from those runners.

Visual Studio Unit Testing framework

To run unit tests for Visual Studio test framework with real-time reporting use command:

vstest.console /logger:Appveyor <assembly> [options]

[options] are standard vstest.console.exe command-line options.

NUnit 2.x

To run NUnit tests with real-time reporting use command:

nunit-console <assembly> [options]

or for x86 assemblies:

nunit-console-x86 <assembly> [options]

NUnit 3.x

To run NUnit tests with reporting test results to AppVeyor use command:

nunit3-console <assembly> [options] --result=myresults.xml;format=AppVeyor

To run tests with real-time reporting use command:

xunit.console <assembly> /appveyor

To run unit tests which target .NET 4.0 and later, use command:

xunit.console.clr4 <assembly> /appveyor


To run MSpec tests with real-time reporting use command:

mspec [options] <assemblies>

Pushing real-time test results to build console

AppVeyor build console has Tests tab with test results updated in a real-time as tests run:

build messages

So, how to get there from a script if you use your own test runner or a different non-supported testing framework?

Build Worker API

You can use Build Worker API REST methods, command-line utility and PowerShell scripts to push test results in a real-time one-by-one or in batch.

Uploading XML test results

Testing frameworks can produce XML report with test results. Upload these XML files from your build script to Test results endpoint and they will be parsed and test results added to Tests tab of build console.

Test results endpoint URL has the following format:{resultsType}/{jobId}


  • resultsType - test framework name to parse test results; supported parsers:
    • mstest
    • xunit
    • nunit
    • nunit3
    • junit.
  • jobId - build job ID that is currently running; can be read from APPVEYOR_JOB_ID environment variable.

Example build script in PowerShell that runs tests and then uploads results in XML format:

# run tests
xunit.console .\path\to\test-assembly.dll /xml .\xunit-results.xml

# upload results to AppVeyor
$wc = New-Object 'System.Net.WebClient'
$wc.UploadFile("$($env:APPVEYOR_JOB_ID)", (Resolve-Path .\xunit-results.xml))

XML files must be uploaded as multipart/form-data.

See also:

Did you know that you can edit this page on GitHub and send us a Pull Request?