Converting a Mercurial repository to Git on Windows

Posted on February 23, 2014

The vast majority of AppVeyor customers use Git, specifically GitHub. Historically, we’ve been using Mercurial for AppVeyor source control, but recently due to the growing popularity and ecosystem of Git and GitHub we thought maybe it’s time to jump Git train and start using it for AppVeyor projects.

The first problem we faced with was converting existing Mercurial repositories to Git with preserving all history and authors. If you search in Google for converting mercurial to git on Windows you will find some (mostly similar) posts on SO, but none of them worked for us if followed “as is” on Windows. After many trials we found the way that 100% works and could be easily reproduced - everything was done from scratch on a clean Windows Server 2012 machine.

This article specifically describes migration of project from BitBucket’s Mercurial repository (original) to GitHub (migrated) using hg-fast-export tool.

Install required software

We assume you start from a clean Windows Server 2012 machine.

Open command line prompt and make sure all tools are available in the PATH: hg, git, python.

Migration

We will be doing everything in c:\projects directory.

Open command prompt.

cd c:\projects

Clone source Mercurial repository:

hg clone https://bitbucket.org/appveyor/demoapp

Create an empty GitHub repository and clone it to demoapp_git directory:

git clone https://github.com/AppVeyor/DemoApp.git demoapp_git

Clone hg-fast-export repository:

git clone http://repo.or.cz/r/fast-export.git

Open c:\projects\fast-export\hg-fast-export.py in Notepad and replace highlighted region with the code below:

hg-fast-export

#!/usr/bin/env python

# Copyright (c) 2007, 2008 Rocco Rutte <pdmef@gmx.net> and others.
# License: MIT <http://www.opensource.org/licenses/mit-license.php>

import sys

# import mercurial libraries from zip:
sys.path.append(r'C:\Program Files (x86)\Mercurial\library.zip')

from mercurial import node
from hg2git import setup_repo, fixup_user, get_branch, get_changeset
from hg2git import load_cache, save_cache, get_git_sha1, set_default_branch, set_origin_name
from optparse import OptionParser
import re
import os

Copy content of fast-export to demoapp_git ignoring .git folder.

Switch to demoapp_git directory:

cd demoapp_git

If you need to map authors to new repository with different name/email create authors.txt with one mapping per line (old=new), like below:

Feodor Fitsner <feodor@appveyor.com>=Feodor Fitsner <feodor@fitsner.com>

Run Python script to import Mercurial repo into Git one (you are running this script from Git repository directory):

hg-fast-export.sh -r c:\projects\demoapp -A authors.txt

Checkout HEAD to check that everything looks good:

git checkout HEAD

Remove conversion files:

git clean -f
del /Q hg2git.pyc

Rename .hgignore to .gitignore:

ren .hgignore .gitignore
git add .gitignore
git commit -m ".hgignore renamed to .gitignore"

Push Git repo to GitHub:

git push -u origin master

Enjoy!

NuGet support in AppVeyor CI

Posted on February 21, 2014

NOTE: NuGet support is available in AppVeyor CI 2.0 which is currently in beta. Please see this post for AppVeyor 2.0 announcement and sign up information.

AppVeyor CI has native NuGet support which becomes de-facto a packaging standard for .NET libraries and applications.

Every AppVeyor account comes with following built-in NuGet feeds:

  • Account feed - password-protected feed aggregating NuGet packages from all projects with support of publishing of your own packages
  • Project feeds - collect all NuGet packages pushed to artifacts during the build

Account NuGet feed

Account NuGet feed aggregates packages from all project feeds and allows publishing of your custom packages. All account feeds are password-protected. You can find account feed URL and its API key on Account → NuGet page:

nuget-account

You can use your AppVeyor account email/password to access password-protected NuGet feeds although we recommend creating a separate user account just for these purposes (Account → Team).

If you use GitHub or BitBucket button to login AppVeyor you can reset your AppVeyor account password using the Forgot password link.

For publishing your own packages to account feed use the command:

nuget push <your-package.nupkg> -ApiKey <your-api-key> -Source <feed-url>

Replace <your-api-key> and <feed-url> with values from Account  NuGet page.

Project NuGet feed

Project feed collects all NuGet packages pushed to artifacts during the build. Project feed is password-protected if the project references private GitHub or BitBucket repository; otherwise project feed has public access:

nuget-project-feed

Automatic publishing of NuGet projects

You can enable automatic publishing of NuGet packages during the build on Build tab of project settings. When it is enabled AppVeyor calls nuget pack for every project in the solution having .nuspec file in the root and then publishes NuGet package artifact in both project and account feeds.

To generate a .nuspec file for your project run the following command from project root directory:

nuget spec

Pushing NuGet packages from build scripts

To push NuGet package as artifact and publish it in both project and account feeds use this command anywhere in your build script:

appveyor PushArtifact <your-nugetpackage.nupkg>

When you delete a project in AppVeyor its corresponding NuGet feed is deleted, however all NuGet packages from that feed remain published in account feed.

Configuring private NuGet feed on your development machine

Visual Studio

To configure custom feed in Visual Studio open Tools → Options → Package Manager → Package Sources and add new feed.

When you first open Manage NuGet packages dialog you will be presented with a dialog box asking for credentials:

nuget-visualstudio-auth

Command line

To configure private NuGet feed on your development machine run this command:

nuget sources add -Name <friendly-name> -Source <feed-url> -UserName <username> -Password <pass>

Configuring private feed to work with NuGet Package Restore

You may use account feed to publish your external packages that can be further referenced during AppVeyor builds.

To configure AppVeyor project to use private NuGet feed during build you can use the following approach:

  1. Create a separate AppVeyor account for accessing NuGet feed.
  2. On Environment tab of project settings add two environment variables nuget_user and nuget_password: nuget-environment-variables
  3. Into Install script box add this command:

     nuget sources add -Name MyAccountFeed -Source <feed-url> -UserName %nuget_user% -Password %nuget_password%
    

Replace <feed-url> with URL of your account feed.

Explicit NuGet package restore before the build

To restore Visual Studio solution NuGet packages before build add this command to Before build script box on Build tab of project settings (provided your .sln file or packages.config are in the root of repository):

nuget restore

otherwise if project solution is in sub-directory:

nuget restore <solution-folder>\<solution.sln>

AppVeyor 2.0: dedicated build VMs, parallel testing, NuGet, deployment and more

Posted on February 19, 2014

After 4 months of intensive development we are excited to announce a public beta of AppVeyor 2.0! The new release provides you with whole new experience: build environment that is under your full control, large projects support with build matrix and parallel testing, scriptless deployment and release management!

You could see those great features in flagman continuous integration services for Linux such as Travis CI, but not for Windows up until now. Today, I’m really proud to say that AppVeyor is the only CI solution for Windows that offers dedicated build machines with admin access, build matrix with jobs parallelization and integrated deployment.

Wireframe Screenshot

What’s new at a glance

Builds run on dedicated Virtual Machines

Azure logo

In AppVeyor 2.0 we are moving away from shared build servers to dedicated VMs. Every build job runs on pristine VM with admin rights! This was probably the main reason for kicking-off this release.

  • With dedicated build VM you get guaranteed performance (CPU and I/O).
  • Increased security with confidence that your code downloaded to isolated environment that is immediately decommissioned after build completes.
  • Having admin rights on build server gives you unlimited possibilities: from installing additional software with Chocolatey or Web PI for supporting your build and headless browser testing to deploying to the same build server for integration testing and BVTs.
  • Being hosted on Windows Azure AppVeyor can offer you build infrastructure with unlimited scale.

NuGet hosting

AppVeyor 2.0 has built-in hosting for private and public NuGet feeds.

Nuget logo

Every account comes with a private password-protected NuGet feed aggregating packages from all projects and enabling publishing of your custom packages.

Projects have separate NuGet feeds with all NuGet packages pushed as artifacts.

Read more about NuGet support

Build matrix

Build matrix

Easily build/test for multiple configurations. Specify which operating systems, build configurations and platforms you would like to include into build matrix and AppVeyor will start a build with multiple jobs for all combinations.

Build matrix supports the following dimensions:

  • Operating system
  • Environment variables
  • Platform, e.g. x86, x64, AnyCPU
  • Configuration, e.g. Build, Debug
  • Test categories

Parallel testing

Parallel testing

Large projects can contain hundreds and thousands of tests that could run for hours. AppVeyor 2.0 allows to split your tests into groups by categories, assemblies or custom criteria and run them as build jobs in parallel thus drastically reducing overall build time.

Read more about parallel testing

Deployment

Deploy

AppVeyor 2.0 has scriptless, repetitive one-click deployment to multiple environments! Deploy as part of the build or promote releases later - manually or through API.

Supported deployment providers:

Read more about deployment

Fine-grained control over build configuration

YAML

Great Windows software must provide user interface for any function it has. AppVeyor 2.0 follows this tradition and further extends project settings, so you can control build environment and inject custom script logic on any stage of build pipeline without ever touching your repository!

For command-line gurus or those coming from Linux we added fancy YAML configuration support! Add appveyor.yml with project configuration into root of your repository and next time you fork the repo just add a new project in AppVeyor.

Read more about build configuration

All new refreshed, responsive and real-time UI

AppVeyor 2.0 has completely re-designed UI to get results faster and on the go!

AngularJS, SignalrR and Foundation helped us to build great experience we’re really proud of:

  • Adaptive layout and improved navigation.
  • Real-time build console.
  • Real-time reporting of MSBuild errors and warnings.
  • Real-time reporting of test results (MSTest, NUnit and xUnit).
  • Optimized for mobile screens.

Try AppVeyor now!

Sign up now

  • Free plan with support of public repositories only. Let us know if you need private repositories support or want to play with parallel testing and we’ll be happy to enable them for your account.

Documentation

https://www.appveyor.com/docs

Dicsussions

http://help.appveyor.com

Enjoy the beta!

Feodor Fitsner, AppVeyor founder and developer

AppVeyor CI overview presentation

Posted on October 01, 2013

In this short presentation I’m talking about AppVeyor CI features and benefits, comparing it to the existing continuous integration solutions and giving some hints about future plans.

Enjoy!

AppVeyor update and new pricing

Posted on September 25, 2013

It’s been a while since we last talked as we were working hard implementing really cool stuff and processing user requests to make AppVeyor more stable, secure and fast. Today we are thrilled to announce a huge update to AppVeyor CI service and start of its commercial availability!

Summary of what we prepared for you

  • Deployment automation as part of CI process
  • Kiln integration
  • GitHub integration improvements
  • Custom build scripts with integrated logging/testing/packaging support
  • PSake support
  • Custom storage for build artifacts, public access to artifacts, deployment to Azure
  • Go language support
  • API PowerShell library
  • Status badges
  • Other small improvements and bug fixes based on community feedback
  • New pricing

Deployment script

We spent long nights polishing our new PowerShell deployment framework - AppRolla. AppRolla is extensible framework tightly integrated with AppVeyor for automating deployment of distributed .NET applications to multi-server environments.

Just some of its features and benefits:

  • Deploy Web applications to standalone IIS servers and web clusters
  • Deploy Windows services to standalone back-end servers and application clusters
  • Deploy Azure Cloud Services from blob storage (see custom artifacts storage below)
  • Modify application configuration files upon deployment
  • Extensible with your own custom deployment tasks

Checkout web application project deployment article to learn how to:

  • Setup automatic deployment of web application project to Staging server as part of CI process.
  • Promote project release to Production environment with a click of a button.
  • Rollback, remove deployment on Production or Staging interactively from command line.

Kiln integration

We got numerous requests from our customers to integrate AppVeyor with Kiln source control from FogCreek Software. Though we are not affiliated with them Kiln looks compelling for projects with private repositories because of its per-user pricing (yes, you can create any number of repositories) and cool “Kiln harmony” feature when every single repository can be used from both Git and Mercurial, simultaneously. Integration with AppVeyor does not require an additional Kiln user - you just create a new “access token” and use it when creating a new project in AppVeyor.

GitHub integration improvements

We added GitHub organizations support and revised OAuth scopes required to access your GitHub account and repositories. AppVeyor makes sure only the minimum set of GitHub permissions is requested just to do the job.

Build scripts

In addition to high-level “Visual Studio solution” and “MSBuild” scenarios we added custom build scripts with maximum control over CI process. You can author build script in PowerShell or as a batch file and use any build tools like MSBuild, PSake, NUnit, xUnit, Node.js in your custom workflow.

While running a custom build script AppVeyor provides you integration points for:

  • communicating MSBuild results back to AppVeyor
  • communicating testing results back to AppVeyor
  • pushing artifacts to build artifacts storage

See creating custom build scripts article for more details.

Custom storage for build artifacts

Out of the box AppVeyor pushes build artifacts in a cloud storage which is GEO-replicated and shared between tenants. We added custom storage support which allows you to configure either Azure blob storage or Amazon S3 account to store build artifacts for your projects. Custom storage that is configured per-account gives you additional benefits:

  • You can have your own folders structure and file naming.
  • Artifacts can be accessed using selected cloud storage API or URLs.
  • Security policy in your organization prevents you from storing build artifacts on a shared storage.
  • Deploy to Windows Azure cloud services directly from a blob storage.
  • Control public access to artifacts.
  • You won’t be affected by artifacts retention policy (it’s not yet implemented, but coming in the future) - store as many versions as you need.

See packaging artifacts article for more details.

Go language support

Go language from Google is gaining momentum and if you are interested in learning/using Go and coming from Windows world AppVeyor is a great platform for doing CI for your next Go project. Take a look at this sample Go repository (thanks Nathan Youngman for his help) to see simple Go project structure and build.cmd to kick-off testing in AppVeyor CI environment.

API

AppVeyor is a single page application (SPA) built with AngularJS (and we promise to tell about its internals in our blog) and all its functionality is available through REST API. We created lightweight AppVeyor API PowerShell library which is currently used by AppRolla deployment script and shows how to authenticate API calls and get details about your projects. We will be adding more functions into it over time.

Status badges

You can put the image on your project website displaying the status of the last build. URL of this image and markdown snippet could be found on “Status badges” tab of project settings.

Start of commercial availability

We processed a lot of feedback from our customers which allowed us to substantially improve AppVeyor CI, increase its security and stability. We have a strong feeling that AppVeyor is ready for prime time and today we announce its commercial availability.

What that means for you? Starting today (September 25th), we offer you a free 30-day trial with unlimited private repositories. During the trial period we will keep in touch with you closely to assist you with the new functionality and work out the best plan for your current needs.

New pricing

Our goal is to provide you outstanding service with competitive and reasonable pricing. We gave our pricing a second thought and came up with 3 simple paid plans:

  • Express - $19/month (1 private repository)
  • Professional - $39/month (10 private repositories)
  • Premium - $79/month (unlimited private repositories)

Compare to $67 per month AWS or Azure small instance running TeamCity or Jenkins!

AppVeyor is free for open-source projects with public repositories.

We listen you!

That’s a ton of new information that could be hard to digest. We are going to publish educational articles in our blog on various topics ranging from deployment automation to AppVeyor development secrets. We’ve been quite active on Twitter (and this is really cool support tool), so make sure to follow us to stay updated.

For now, if you have any questions please do not hesitate to drop us a message at team@appveyor.com, start a new discussion on our forums or submit a feature request on UserVoice.

Best regards, Feodor Fitsner, AppVeyor founder and developer