Some Order in (Not) Chaos

It has been a long while (2013) since I promised writing about Psake and using Powershell for build scripts.

But sometimes, something unexpected has to happen for old things to become current again. That something was my curiosity in using AppVeyor as a automated deployment platform.

Hosted CI++

It is not now that I “discovered” AppVeyor (I created a working barebones NMoneys build long time ago, when Google Code was still alive) but it has been lately that I was interested in its potential for automating deployments.
I am not going to be analyzing AppVeyor: go ahead, browse their magnificent documentation and don’t take my word on how easy it is to get going. Do try it.

AppVeyor is a hosted Continuous Integration/Delivery (CI/CD) for the Windows platform so you don’t have to have your own server and go through the process of setting up a server.
Best of all it is free for OSS projects.

CI/CD 101

The basic idea of CI is “listening” to your source code repository and when new commits are made, perform some actions (usually compiling the source code, running unit tests, perform some sort of static analysis and some more stuff).
The basic idea of CD is build upon CI in order to automatically (or with minimal human intervention) push your software to the environments that needs to be deployed for people to use.

It can get more deep and complicated than this, but it is basically automation performed somewhere else than your development machine.

NMoneys Build History

In the beginning of time NMoneys was built using nothing but MSBuild scripts. I hated it, so whenever I had to extend it (automating API documentation) I migrated and enhanced the script using Psake as the task framework and Powershell as the language instead of XML. The main purpose for this build was (and is) to generate the artifacts (the various Nuget packages) whenever a new release was due.

A new release was a matter of executing the build script to get the freshly built NuGet packages after making sure everything was ok (tests). This is, but not for every build there is a corresponding deployment, a bunch of manual steps existed like: pushing the packages to NuGet and creating a binary release with unsigned and singed assemblies.

nmoneys_build

  • Clean: clean solution and recreate the /release folder, that will contain everything we are likely to deploy
  • Compile: invoke msbuild.exe to compile the solution
  • Sign: use IL trickery to sign an assembly once built
  • Document: use ImmDoc.Net and the MS HTML Help Compiler to generate a .chm with the API documentation
  • Test: invoke nunit-console.exe to run all tests of the solution and create visual reports on their execution
  • Copy Artifacts: copy binaries, source files, test reports, documentation and package manifest to the /release folder
  • Build Artifacts: invoke nuget.exe to pack the different packages and zip.exe to generate the bin and signed artifacts to be released

    Could we do better? I would not be writing if we couldn’t Winking smile

    CI with AppVeyor

    Translating the process to AppVeyor was not difficult at all, but extracting a common set of script functionality that can be invoked both by the local Psake build and the remote engine took a bit longer.
    All I can say that the scripts are so much cleaner and easy to follow than before and because of that only, the process was worth it.

    Appveyor has a rich and complex build pipeline and their default build and test stages make redundant the local Compile and Test tasks, as well was the Clean, because, for every build that is started, a brand new machine is provisioned, so there is nothing to clean. Of their complete pipeline we use the following steps:

    appveyor_build

    The equivalency between the two is roughly like this:

    • Init + BuildCompile + Sign + Document
    • TestTest + CopyArtifacts + BuildArtifacts

    CI Build?

    Isn’t it weird to have a Deploy stage in a CI pipeline?
    Only if the deploy has the meaning of “pushing components to another environment for their execution”.

    Appveyor has two types of deployment: Inline (the last stage of the pipeline) and Environment. Whereas the Inline deployment is executed for each build execution, the Environment deployment is asynchronously triggered by “human intervention”.
    The Deploy stage we refer to in our graphic is, thus, an inline deployment and in our case the “only” thing we do is “promote” artifacts from the /release folder to AppVeyor artifacts for them to be deployed later on in en Environment deployment.
    The manual promotion (as opposed of the artifact definition as configuration of the build) is provoked by the similarity in package naming which would make impossible to deploy only one package to a given environment.

    Details, please

    All these changes have turned the old NMoneys build process into a very powerful, lean and streamlined deployment pipeline.

    Details matter. Details are fun. I have learned a ton of Powershell in the refactoring process, so… why not sharing it? But this post is already long, so there will be another one with those beloved details.

  • Kategorier: Udvikling

    Tagged as: , , ,