AppVeyor adds support for GitHub Enterprise, Atlassian Stash and any external repository

Posted on January 22, 2015

We’ve just made AppVeyor available to even more developer teams! We are thrilled to announce new additions to AppVeyor supported source control providers:

  • GitHub Enterprise
  • Atlassian Stash
  • Git
  • Mercurial
  • Subversion

Now you can use AppVeyor practically with any repository out there whether it is hosted in a cloud or on your own premises!

GitHub Enterprise

GitHub logo

All beloved GitHub features such as branch builds, Pull Requests and webhooks are available in GitHub Enterprise integration. AppVeyor can use both OAuth and Personal Access tokens to authenticate against your GitHub Enterprise repositories.

Atlassian Stash

Atlassian Stash logo

AppVeyor provides complete and seamless integration with Atlassian Stash. Both OAuth and Basic authentications are supported. Whenever you add a new project in AppVeyor webhook and repository SSH key are automatically set.

Git, Mercurial and Subversion repositories

Git - Mercurial - SVN logo

Now you can specify a URL to any repository hosted on the Internet! AppVeyor supports credentials and SSH authentication against those repositories.

Do we support your repository now?

Send us a message if you want to re-evaluate AppVeyor!

Having repository behind the firewall?

Of course, the repository should be accessible over the Internet. However, hold on! As a next milestone we are going to release AppVeyor on-premise edition which you can install behind your firewall. On-premise edition will provide even more tight integration with GitHub Enterprise and Stash such as “Sign in with GHE or Stash” buttons.

Other news

In case you missed that:

Test image with Visual Studio 2015 CTP and SDK

Posted on January 20, 2015

We’ve just added a new build worker image with Visual Studio 2015 CTP 5 installed!

Both Visual Studio Ultimate 2015 CTP and Visual Studio 2015 SDK CTP were installed from official download page.

Build worker image is called Visual Studio 2015 CTP. You can select it on Environment tab of project settings (if you configure project on UI):

project-environment-tab

or specify in appveyor.yml:

os: Visual Studio 2015 CTP

Please note builds using this image run on Azure environment which means there is a few minutes delay before build starts required to provision build worker VM.

Add this command to install section of your build config if you need msbuild command to call MSBuild 14.0 by default:

set PATH=C:\Program Files (x86)\MSBuild\14.0\Bin;%PATH%

There is an image with previous Visual Studio 2015 release called Visual Studio 2015 Preview. Starting from today we will be updating only new Visual Studio 2015 CTP image.

Enjoy!

Building private GitHub repositories with sub-modules

Posted on January 08, 2015

The following article was written specifically for GitHub, but some of these techniques could be applied to other Git hosting platforms as well.

How AppVeyor is cloning private repos

AppVeyor uses SSH to clone private Git repositories. When you add a project in AppVeyor a new RSA key-pair is generated which consists of private and public keys. Public key is deployed to a remote Git repository using GitHub (or BitBucket) API and private key is pushed to build worker during the build. For SSH protocol to work on Windows private key should be located in %USERPROFILE%\.ssh\id_rsa file.

The problem with private sub-modules

Git has submodules support and this is a wonderful tool for organizing large projects or reusing some code. While building your solution on AppVeyor you need to checkout sub-modules as part of your build. Well, you can use the following command during install phase which occurs between clone and build:

git submodule update --init --recursive

The problem arises when sub-modules refer private Git repositories which cannot be cloned without authentication and as a result you get stalled build. This is because sub-module repository does not contain SSH public key used to authenticate main repo, so Git is asking for credentials:

sub-modules-stalled-build

The solution

A custom SSH key could be used to checkout repository private sub-modules. The rest of this article explains how to generate SSH key and setup AppVeyor project to use it.

Check sub-modules path

First of all you have to check sub-modules URLs in .gitmodules to make sure they are in SSH format. For GitHub it should be something like:

url = git@github.com:{owner}/{repo}.git

Generate SSH key

Now, let’s generate a new SSH key that will be used to fetch sub-modules.

In command prompt type the following command:

ssh-keygen -t rsa

When prompted enter key file name, say submodules and empty passphrase.

ssh-keygen.exe utility is part of Git installation for Windows and is typically located in C:\Program Files (x86)\Git\bin directory.

In the current directory you’ll find two files: submodules which contains private key and submodules.pub with public key.

Add SSH public key to GitHub

If you have only one sub-module in your main repository you can add public key directly to sub-module repo, however if there are multiple dependencies GitHub won’t allow you to add the same key again.

Open submodules.pub file and copy its contents to clipboard.

Navigate to SSH Keys under your GitHub profile and add a new SSH Key with contents from clipboard and any title.

Configure AppVeyor project to use SSH key

Next, during the build on the worker machine we have to put private key contents into %USERPROFILE%\.ssh\id_rsa before running git submodule update --init --recursive command.

We’ll store contents of private key in environment variable.

UI

Open “Environment” tab of project settings in AppVeyor and add a new environment variable called priv_key. Open submodules file with private key and copy base-64 body of the key between -----BEGIN RSA PRIVATE KEY----- and -----END RSA PRIVATE KEY----- into clipboard:

rsa-private-key

Paste contents of clipboard into value field of environment variable. New lines will be changed to spaces - that’s OK - we’ll turn them back to new lines with PowerShell script shown below.

Mark variable as “secure” by clicking “lock” icon next to it - this will prevent it from being decoded during pull requests (see explanation below).

In Install script field paste the following code:

$fileContent = "-----BEGIN RSA PRIVATE KEY-----`n"
$fileContent += $env:priv_key.Replace(' ', "`n")
$fileContent += "`n-----END RSA PRIVATE KEY-----`n"
Set-Content c:\users\appveyor\.ssh\id_rsa $fileContent
git submodule -q update --init --recursive

appveyor.yml

Copy the contents of private key to clipboard as shown above and open Encrypt data tool in AppVeyor. Encrypt the value of clipboard using that page.

Add this to your appveyor.yml:

environment:
  priv_key:
    secure: <encryped-value>

install:
  - ps: $fileContent = "-----BEGIN RSA PRIVATE KEY-----`n"
  - ps: $fileContent += $env:priv_key.Replace(' ', "`n")
  - ps: $fileContent += "`n-----END RSA PRIVATE KEY-----`n"
  - ps: Set-Content c:\users\appveyor\.ssh\id_rsa $fileContent
  - git submodule update --init --recursive

Security considerations

“Secure” variables means you can safely put them into appveyor.yml that is visible to others. Other than that they are just regular environment variables in a build session that could be easily displayed in a build log by simple Get-ChildItem env:.

However, secure variables are not decoded during Pull Request builds which prevents someone from submitting PR with malicious build script displaying those variables. In more controlled environment through with a trusted team and private GitHub repositories there is an option on General tab of project settings to allow secure variables for PRs.

If you accidentally submitted any sensitive information into public repo or displayed it in a public build log don’t wait - invalidate/change/re-generate that data immediately!

AppVeyor Deployment and YAML improvements

Posted on December 16, 2014

We are continuously improving AppVeyor platform and doing a couple of changes/deployments during the week. To give you more information about ongoing/upcoming changes and status updates we introduced a new “Technical updates” mailing list. It will be sent approximately two-three times a week.

All existing customers can subscribe to this mailing list on Profile page. If you decide not to subscribe to technical updates you’ll still be receiving this monthly newsletter.

Now, back to deployment improvements. Deployment has always been a strong part of AppVeyor and we are committed to make AppVeyor a single shop for your entire continuous delivery. Also, YAML configs worked amazingly well for AppVeyor customers and we continue to invest into this area with a new features based on your feedback.

New SQL Database deployment provider

Your AppVeyor builds may produce SSDT packages (.dacpac files) describing application database changes. Publishing SSDT project from Visual Studio is a trivial task, but it’s always been a challenge of doing that on a build server. Most common tools for synchronizing DACPAC packages were SqlPackage.exe and MSDeploy.exe with built-in DacFx provider.

Now AppVeyor offers a new SQL Database deployment provider for incremental publishing of SSDT packages to a local SQL Server instance, remote SQL Server or Azure SQL databases.

SQL database provider settings

SQL Database provider uses SQL Server Data-tier Application Framework (DacFx) and as most of AppVeyor deployment providers it can be used during the build for staging deployment as well as a new “environment” for production deployments. Read more

SFTP support

We added SFTP (SSH File Transfer Protocol) support into FTP deployment provider. Don’t mess it with FTPS which is also supported - it’s a completely different thing though it organically complements a new “unified” FTP provider. Read more

Install MSI packages with Deployment Agent

With the improved AppVeyor Deployment Agent it’s now possible to install MSI packages on staging and production environments behind the firewall. With MSI added you can use Agent to deploy various types of workloads: web applications, windows services, console apps, SQL Databases and MSI packages. Read more

New GitHub Releases provider

This is definitely a great news for open-source projects hosted on GitHub and using AppVeyor for their CI! GitHub deployment provider allows to publish build artifacts as assets to your repository release. Read more

YAML configuration validation

We re-factored appveyor.yml configuration parser to make it work in “strict” mode, so you get immediate feedback if there is something wrong with project config and as a bonus there is a new page for validating appveyor.yml instead of try-and-fail process:

Validate YAML

Export project configuration in YAML

You can easily switch your projects to YAML and benefit from portable and versioned configuration. There is a new tab on AppVeyor project settings which allows you to see how project changes made through UI would look in appveyor.yml:

Export YAML

New REST API for configuring project with YAML

It’s been a challenge to configure project settings through REST API as their request/response JSON format was, well, derived from UI and not suitable for processing by humans. With all these parsing and exporting improvements in YAML config we also added two new API calls: get project settings in YAML and update project setting in YAML.

Holidays are coming and we would like to wish all our customers more green builds and less bugs! Merry Christmas and Happy New Year!

Custom text on SVG build status badges

Posted on November 26, 2014

SVG is great! We’ve just added a small, but very neat feature that allows you customizing badge text.

This looks really great for batch-specific status badges put in one line (these are statuses for Grunt.js project):

master branch badge osx-travis branch badge legacy-log branch badge

To customize SVG badge titles for pending, failing and passing states add pendingText, failingText and passingText query parameters respectively.

For example:

https://ci.appveyor.com/api/projects/status/32r7s2skrgm9ubva?svg=true&passingText=master%20-%20OK&failingText=master%20-%20Fails

Read more about status badges in AppVeyor documentation.

Enjoy!