I’m a tinkerer, always wanting to know how something works internally. Growing up I was always pulling apart old telephones, radios, anything I could get my hands on, to see how they worked inside, what would happen when I poked at things in there and testing the limits. This is one of the reasons I got into software development, I like looking at problems, try to break them down, see what makes them tick and finally come up with my own solution.
I’ve been working with Umbraco for about 7 years now and working with ASP.NET for 10 years so when I approach Umbraco I have always come at it from an ASP.NET developer mindset.
When I look at the differences between how I develop Umbraco applications compared to ASP.NET applications there’s one thing that always bugs me.
Deployment.
How we deploy ASP.NET
When I think about how we deploy an ASP.NET application, be it WebForms, MVC or a WebAPI SPA there’s a few common themes:
-
Source Control is king
-
A build server compiles
-
A deployment tool deploys
While over the years the tooling has changed (VSS, TFS, SVN, CruiseControl.NET, TeamBuild, TeamCity, robocopy, FTP, MSDeploy, Octopus Deploy, so and so forth) these three points stay the same.
How we deploy Umbraco
When looking at how we deploy Umbraco there's a slightly different approach. Sure we keep things in our VCS but we can't keep everything there, after all, Umbraco has a lot of stuff within its database.
So maybe we have side-by-side browser windows to compare Document Types, maybe it's exported as a package, maybe we rely on Courier or maybe you do what we tried once and seed each database with offset ID's and use a SQL Data Compare tool to diff and merge (and that went just as well as you can imagine!).
The problem of the database
So much of Umbraco relies on what’s in the database. This makes a problem for deployments, there's really important information in there that I simply don't control the create/edit process for, let alone try and get back to one of my original goals of Source Control is king.
Over the years there has been many ideas on how this can be solved and for a while Code First was a hot topic. I’ve built a Code First Umbraco tool, in fact I've built 3 different Umbraco code generators, but ultimately it comes down to not really solving the problem in the best way possible. As soon as you're Code First you make the back office less appealing, no more tweaking Document Types in there, you have to re-code, compile and see what's what.
The other approach that is quite popular is uSync. The idea behind this is that you export everything to files on disk and they get imported into another environment. This feels a lot more in align with goal of Source Control is king, but there's one downside, the deployment. While uSync will be automated in its importing it does that when the site starts up, so it's not really handled by a deployment tool.
Rethinking the solution
Back to the story from the start, I like to tinker, to find out how things work. So rather than picking at everything I decided to have tackle this problem myself. What I wanted to do was:
-
Have everything in source control
-
Be able to run something outside of the web context
-
I don't want to run the same step twice on an environment
This whole idea comes from what I'm familiar with when working with SQL, EF Code First Migrations or dbup are examples of what I want to do, a number of sequential steps to go from schema 1 to schema n, in this case the schema is our Document Types, Macro Definitions, etc.
The real key here for me is that I don’t want to have the start the server to run this, I don't want the first request to pay the price of doing my migration.
Getting down and dirty
So I started to tinker and tinker with the most out-there idea, a console application that controls Umbraco. Thanks to the service APIs that were introduced in Umbraco 6 and expanded on in Umbraco 7 there's much less of a reliance on the HttpContext object.
My next stop was to dive into the source of Umbraco to have a look at how it sets up the services and with a few tricks on the BootManager (which are beyond the scope of this article) you can setup quite a large part of Umbraco without using a web component. A few more tricks with the .NET AppDomain (again, beyond what I want to cover here) and then we've got our starting point.
Your Chauffeur arrives
From this foundation came my idea, Chauffeur, which is a bit of a joke on Courier, it delivers things, just in a classier fashion.
Everything is a Deliverable
The core of Chauffeur is a concept called a Deliverable (yes, I have bad jokes everywhere in Chauffeur). A Deliverable is something that Chauffeur does and it's built on an extensibility model as I have ideas of Chauffeur should do but I didn't want to restrict others having ideas too.
If we start with the basic Deliverable, help, we can find everything that Chauffeur can do for us.
Each of these are a Deliverable themselves and we can run help against them to find out what they can do.
Let's start with settings
You can see here Chauffeur knows things about your application, where is the website on disk, the Umbraco folder and your connection string (this is a SQLCE install).
And there are a bunch of other Deliverables in the default install that we'll look at in a bit but first I want to look at the concept of a Deliverable a bit more.
Making a Deliverable
Everyone's application is different so as I mentioned I wanted Chauffeur to be extensible, so when it starts up it goes out and finds all the Deliverables, either baked in, 3rd party extensions or ones you’ve written yourself, and they are pretty simple to implement. First up you want to inherit from the Deliverable base class:
using Chauffeur;
namespace Chauffeur.Samples
{
public class HelloWorldDeliverable : Deliverable
{
}
}
Then you’re going to need a constructor. Because Chauffeur can read/write to the console, but I wanted it to be testable, you don't use Console.ReadLine or Console.WriteLine, instead you receive a TextReader and TextWriter to work with, which are exposed as an In and Out property from the base class:
using System.IO;
using Chauffeur;
namespace Chauffeur.Samples
{
public class HelloWorldDeliverable : Deliverable
{
public HelloWorldDeliverable(TextReader reader, TextWriter writer)
: base (reader, writer)
{
}
}
}Lastly we need to add an attribute to name our Deliverable. This is the command that you will execute from Chauffeur when you want to call your deliverable. Each Deliverable has a single name but you can also add aliases to it so you can call it by an abbreviation, like you'll see in the help screenshot.
using System.IO;
using Chauffeur;
namespace Chauffeur.Samples
{
[DeliverableName("hello-world")]
public class HelloWorldDeliverable : Deliverable
{
public HelloWorldDeliverable(TextReader reader, TextWriter writer)
: base(reader, writer)
{
}
}
}Now you're done! Sure, your Deliverable does nothing, but that's all you really have to implement! To actually do something there's a Run method that you would implement, which is an asynchronous method that you can do most anything you want.
Wait, weren’t we talking about deployments?
Well done astute reader, yes we were talking about deployments. You see, with Chauffeur I wanted to start with the basic building blocks and then I am in a position to meet my original goals.
The first thing I wanted to be able to install the Umbraco database. This makes sense, it might be the first time deploying into an environment, I have blown it away from local for some reason, etc.
So I created an install Deliverable:
This looks at your Umbraco connection string, determines the database type, if it’s SQL CE and doesn't already exist it'll create the database file (for SQL Server it requires the database to exist) and then uses the Umbraco setup engine to create all the tables, keys, etc. Internally this is just using the Umbraco setup engine, just like when you use it in the browser wizard (without the starter kit), but I haven't even started IIS Express.
The default user
When you install Umbraco you are prompted for the default user's password. This is set during the browser-based setup wizard, but with Chauffeur we can also change that, in fact you could change anyone’s password, meaning that you don't need to keep passwords in source control, you can have your deployment platform configure it:
Here we use the user Deliverable, calling the sub-command change-password, specifying the user ID (or username), the old password and the new password (fyi – the password setup by the Umbraco installer is 'default').
Delivering packages
The Umbraco package engine is really good at getting things from files into the Umbraco database. By things I mean Document Types, Templates, etc, it's quite good at that. So naturally Chauffeur should be good at that:
Here's Chauffeur importing the CWS starter kit. Now, it’s a little different to if you're installing a package from the package repository, Chauffeur wants the package.xml file from within a package, meaning you have to extract the ZIP and add the package.xml to source control. The reason for this is that any file should be restorable from either NuGet or in source control themselves, and Chauffeur will just add the stuff to the database.
Chauffeur Delivers!
What we've seen here is a number of bespoke commands run from a shell which works nicely when you can log into the shell but doesn't work so when in an automated deployment scenario.
But never fear, that's where Chauffeur's delivery command comes in. This is your way of saying to Chauffeur "I have a bunch of deliveries to be delivered, deliver what is undelivered." Ok, maybe that’s not clearer!
Think of this like an Entity Framework migration, it looks at the state of your system, looks at a collection of pre-written steps, and executes the ones that haven't previously been executed.
A delivery looks like so:
install y
user change-password 0 default my-secret-passwordThis is then in a file 001-Setup.delivery. I'd then have another one called 002-Setup_CWS.delivery that looks like:
Package CWS_PackageOn my new environment I run Chauffeur with the delivery command:
And when I run it again:
Chauffeur knows what has been delivered and won't deliver it again, you don’t want to sign for a package twice do you!
Chauffeur is ready to pick-up you
Chauffeur is on NuGet, that's the best way to get and it's broken down into two packages, Chauffeur Runner and Chauffeur. The reason for the breakdown is that the Chauffeur NuGet package is the core engine and what you would depend on if you're creating your own Deliverables.
The Chauffeur Runner package is the console application that starts Chauffeur, so it’s what you interact with. In theory Chauffeur can be hosted anywhere, even within a web context, so Umbraco could host Chauffeur, but I haven't tried that…
So what does the future hold? Well I believe the underlying platform is pretty rock solid, the code is up on GitHub and it has a CI platform that builds PR's and generates NuGet packages so you can immediately test yourself.
I'm currently working on better integration with uSync because while Chauffeur works with uSync it's a bit cumbersome and doesn't support all uSync features. The benefits of uSync used the Umbraco package format itself, the files it generates are usable from the Chauffeur package Deliverable.
All that's left if your feedback, so call your Chauffeur and ride in style.