Unit Testing Umbraco - Plausible by Sam Sussman

Unit testing is a common requirement in software development. Using third party libraries and systems often present challenges when writing unit tests. Umbraco is no exception.

Unlike many third party solutions, Umbraco has the advantage of being open source, providing less restrictions when coming up with solutions. I would like to demonstrate that unit testing code which utilizes the Umbraco API is both possible and valuable.

Example code for this article, as well as an alpha version of a unit testing engine, can be found at https://github.com/thantos/SAS.UmbracoUnitTesting.

Why is unit testing Umbraco difficult?

Umbraco’s development methods are great for securing an API from unintended modifications, allowing intended modifications, and providing convenience. These same methods, including internal constructors, resolvers, and extensions, can make testing difficult.

Internal constructors are often seen when a class is intended to be used outside of the Umbraco dlls, but not intended to be constructed. When all of a class's constructors are internal or at least the ones we’d need, creating an instance can be difficult.

Resolvers are used by Umbraco to discover classes that provide services like the Culture Dictionary or Property Type Resolvers. All of these resolvers follow a singleton pattern and have internal constructors. In some cases I have found a native Api method to initialize them, but each resolver has its own set of difficulties.

And finally, Extensions! Anyone who has done unit testing has run into issues testing extensions. Extensions appear to be members of a class, but are really members of an unrelated static class which libraries like Moq cannot affect in a way we’d need to for testing.

Prereqs

Umbraco 7.4.* (likely lower as well)
Moq - library for mocking interfaces.
Microsoft Unit Testing

Errors!

Time to write a test. As a good developer, I want to test the view models returned by my custom SurfaceController. Currently this custom SurfaceController has a single action which returns an empty view model. Nothing complicated, it should just work.

public class BasicTestSurfaceController : SurfaceController
{
    public PartialViewResult BasicTestAction()
    {
        return PartialView(new { });
    }
}

Test

[TestClass]
public class BasicTestSurfaceControllerTest
{
    [TestMethod]
    public void BasicActionTest()
    {
        var controller = new BasicTestSurfaceController();
        var res = controller.BasicTestAction();
        var model = res.Model as object;

        Assert.IsNotNull(model);
     }
}

Running the test results in an error, but we didn’t even write any code!

System.ArgumentNullException: Value cannot be null.
Parameter name: umbracoContext

The error was thrown by a parent class of the SurfaceController called PluginController. The PluginController requires a valid the UmbracoContext to exist. Using the empty constructor, SurfaceController attempts to pass in the UmbracoContext.Current singleton and finds it null.

Minimal Solution

Luckily the UmbracoContext has a handy method called EnsureContext testing. This allows the major dependencies of the UmbracoContext to be passed in and for the singleton to be set. While we are at it we will setup the ApplicationContext as the UmbracoContext requires it.

var appCtx = ApplicationContext.EnsureContext(
    new DatabaseContext(Mock.Of(), Mock.Of(), new SqlSyntaxProviders(new[] { Mock.Of() })),
    new ServiceContext(),
    CacheHelper.CreateDisabledCacheHelper(),
    new ProfilingLogger(
        Mock.Of(),
        Mock.Of()), true);

var ctx = UmbracoContext.EnsureContext(
    Mock.Of(),
    appCtx,
    new Mock(null, null).Object,
    Mock.Of(),
    Enumerable.Empty(), true);

Later we will use the mock objects to inject the values needed for our tests, but the base case doesn’t need anything special. Prepending the above two lines to our basic test will allow it to pass.

Difficult Operations

If you thought passing a simple empty action is as bad as this gets, you’d be wrong. Here are some operations which are non-trivial to test.

*As of version 0.0.1, I have not figured these operations out yet

SurfaceController.CurrentPage

A common operation is to use the CurrentPage property from the SurfaceController.

public PartialViewResult BasicCurrentPageAction()
{
    return PartialView(null, CurrentPage.Name);
}

If the CurrentPage property is referenced in some code we try to test, we’ll get a cryptic error back.

System.InvalidOperationException: Cannot find the Umbraco route definition in the route values, the request must be made in the context of an Umbraco request

This is because the current “request” doesn’t contain any valid route data. Here is the working test:

[TestMethod]
public void BasicCurrentPageTest()
{
    //SETUP UmbracoContext (ctx) and ApplicationContext (appCtx)

    var contentMock = new Mock();
    contentMock.Setup(s => s.Name).Returns("test");

    var content = contentMock.Object;

    //setup published content request. This sets the current content on the Umbraco Context and will be used later
    ctx.PublishedContentRequest = new PublishedContentRequest(new Uri("http://test.com"), ctx.RoutingContext,
        Mock.Of(section => section.UrlProviderMode == UrlProviderMode.AutoLegacy.ToString()),
        s => new string[] { })
    {
        PublishedContent = content
    };

    //The reroute definition will contain the current page request object and be passed into the route data
    var routeDefinition = new RouteDefinition
    {
        PublishedContentRequest = ctx.PublishedContentRequest
    };

    //We create a route data object to be given to the Controller context
    var routeData = new RouteData();
    routeData.DataTokens.Add(Constants.Web.UmbracoRouteDefinitionDataToken, routeDefinition);

    var controller = new BasicTestSurfaceController();
    //Setting the controller context will provide the route data, route def, publushed content request, and current page to the surface controller
    controller.ControllerContext = new System.Web.Mvc.ControllerContext(ctx.HttpContext, routeData, controller);

    var res = controller.BasicCurrentPageAction();
    var model = res.Model as string;

    Assert.AreEqual(content.Name, model);
}

In short, we need a RouteData instance which has a Data token defined for the Route Definition. The RouteDefinition needs a PublishedContentRequest (which we also give to the UmbracoContext). The PublishedContentRequest contains the “current page”, a IPublishedContent object which we mocked and gave a name. Finally, we give the RouteData to the controller by setting the ControllerContext.

If you run the test, you’ll see the test passes.

Umbraco Helper

If you, like me, prefer to pull most of your loading logic out of the razor views, that means you can now test those actions. Odds are the UmbracoHelper is being used in your code, but how do we get data into the UmbracoHelper so we can test?

Let’s take this simple controller action:

public PartialViewResult BasicPublishedContentAction()
{
    return PartialView(null, this.Umbraco.AssignedContentItem.Name);
}

This method uses the Umbraco property of the SurfaceController (or RenderMvcController, UmbracoApiController, etc.) and gets the AssignedContentItem. In most cases the AssignedContentItem is the same as the CurrentPage property from the SurfaceController (but with dependency injection magic, we aren’t always in the controller).

We easily create the UmbracoHelper for testing like...

var helper = new UmbracoHelper(ctx, mockContent.Object);

But how do we give that to the Controller?

The SurfaceController.Umbraco property is readonly, so we can’t simply set it. Fortunately the Umbraco controller base types (Surface, RenderMvc, and UmbracoApi) all have a constructor which takes in the UmbracoContext and UmbracoHelper.

Add a new constructor to the Controller which looks like this:

public BasicUmbracoApiController( /*extra parameters here*/, 
     UmbracoContext context, UmbracoHelper helper) 
     : base(context, helper) { }

This will allow us to pass in our UmbracoHelper instance.

[TestMethod]
public void BasicPublishedContent2Test()
{
   // SETUP UmbracoContext and ApplicationContext

    var mockContent = new Mock();
    mockContent.Setup(s => s.Name).Returns("test");

    var helper = new UmbracoHelper(ctx, mockContent.Object);

    var controller = new BasicTestSurfaceController(ctx, helper);
    var res = controller.BasicPublishedContentAction();
    var model = res.Model as string;

    Assert.IsNotNull(mockContent.Object.Name, model);

}

Dictionary Example

The UmbracoHelper can provide us with access to mocking some of the data we will need to test. Take the Dictionary for example.

public PartialViewResult BasicDictionaryAction()
{
    return PartialView(null,this.Umbraco.GetDictionaryValue("Test Key"));
}

There is a larger, more useful, constructor for the UmbracoHelper.

UmbracoHelper(UmbracoContext umbracoContext, IPublishedContent content, ITypedPublishedContentQuery typedQuery, IDynamicPublishedContentQuery dynamicQuery, ITagQuery tagQuery, IDataTypeService dataTypeService, UrlProvider urlProvider, ICultureDictionary cultureDictionary, IUmbracoComponentRenderer componentRenderer, MembershipHelper membershipHelper)

We will touch on the uses for some of the other parameters, but now the one we care about is the ICultureDictionary.

The above controller action can be tested like:

[TestMethod]
public void BasicDictionaryTest()
{
    //Setup ApplicationContext and UmbracoContext

    var test_value = "test";

    var valueDict = new Dictionary<string, string>() { { "Test Key", test_value } };

    var mockDict = new Mock();
    mockDict.Setup(s => s[It.IsAny()]).Returns(key => valueDict[key]);

    var helper = new UmbracoHelper(ctx,
        Mock.Of(), Mock.Of(),
        Mock.Of(),
        Mock.Of(),
        Mock.Of(),
        new UrlProvider(ctx, Mock.Of(section =>
            section.UrlProviderMode == UrlProviderMode.Auto.ToString()),
            new[] { Mock.Of() }),
        mockDict.Object, //(),
        new MembershipHelper(ctx, Mock.Of(), Mock.Of()));

    var controller = new BasicTestSurfaceController(ctx, helper);
    var res = controller.BasicDictionaryAction();
    var model = res.Model as string;

    Assert.AreEqual(test_value, model);
}

(mock) Configuration

From time to time when discovering ways to test different operations, I’d get an error:

System.Configuration.ConfigurationErrorsException: Could not load the Umbraco.Core.Configuration.UmbracoSettings.IUmbracoSettingsSection from config file, ensure the web.config and umbracoSettings.config files are formatted correctly

At first I added all of the App Settings and UmbracoConfiguration section to the App.config in my test project. However, I recently found that just adding the configSection and registering the UmbracoConfiguration section group with the Settings section in the app.config is enough to resolve the error.

<configSections>
    <sectionGroup name="umbracoConfiguration">
        <section name="settings" type="Umbraco.Core.Configuration.UmbracoSettings.UmbracoSettingsSection, Umbraco.Core" requirePermission="false" />
    </sectionGroup>
<configSections>

CoreBootManager

Some situations require inner workings of Umbraco that cannot be bypassed whatsoever. This includes IPublishedContent extensions like HasProperty. They rely on “Resolvers” to be initialized. These resolvers are singletons and the instances they provide all have internal constructors. Without access to MS Fakes, there really is no way to manually set the resolvers.

Unless... we can set the resolvers the same way Umbraco does it!

When Umbraco boots it calls on the WebBootManager. The WebBootManager initializes many of the components that Umbraco needs to run. Unfortunately the WebBootManager does not work outside of a request due to some conflicts with the HttpContext and having real values set or not. The web project uses some less than friendly HttpContext methods.

Luckily, WebBootManager inherits from CoreBootManager, which has less strict dependencies and still manages to set most of the resolvers we need (but not all, IPublishedContent.GetCulture() epects WebBootManager resolvers).

CoreBootManager cannot be used as is. Some slight modifications must be made by inheriting from it.

public class CustomBoot : CoreBootManager
{
    private readonly ServiceContext _servContext;

    public CustomBoot(UmbracoApplication app, ServiceContext context) : base(app)
    {
        _servContext = context;
    }

    protected override ServiceContext CreateServiceContext(DatabaseContext dbContext, IDatabaseFactory dbFactory)
    {
        return _servContext;
    }

    public override IBootManager Complete(Action afterComplete)
    {
        FreezeResolution();

        return this;
    }
}

The required changes include providing our own ServiceContext to allow mocking the services when needed and overriding the Complete method to prevent some breaking code from running (we still need to “FreezeResoluion” though).

Now we can initialize the resolvers by calling

var bm = new CustomBoot(new UmbracoApplication(), serviceContext);
bm.Initialize().Startup(null).Complete(null);

Complex Situation (HasProperty)

Now we “tie it all together” by testing one of the most complex operations I have resolved.

First the action:

public PartialViewResult BasicHasPropertyAction(int id, string property)
{
    var type = Umbraco.TypedContent(id);

    var hasProperty = type.HasProperty(property);

    return PartialView(null, hasProperty);
}

I know right? How could this innocent little block of code lead to so many problems??

First of all, HasProperty is an extension, so we immediately lose a great deal of control. Next it relies on the content to have a content type and that content type must have the property we are looking for. The PublishedContentType itself causes many of the of the issues. It doesn’t have a public constructor, but it does have a static method called Get, which returns a new PublishedContentType. The PublishedConentType.Get method has a great deal of dependencies including accessing a number of services (from ServiceContext) and the cache.

Still with me? There is more, but let’s just jump to the test.

[TestMethod]
public void BasicHasPropertyTest()
{
    //create a mock of the content type service
    var mockContentService = new Mock();
    //this time we will make our own service context, which can take in all of the umbraco services
    //Pass the context the mocked content service object
    //core boot manager requires Services.TextService to not be null (pass in mock of ILocalizedTextService)
    var serviceContext = new ServiceContext(contentTypeService: mockContentService.Object, localizedTextService: Mock.Of());

    var appCtx = ApplicationContext.EnsureContext(
        new DatabaseContext(Mock.Of(), Mock.Of(), new SqlSyntaxProviders(new[] { Mock.Of() })),
        serviceContext,
        CacheHelper.CreateDisabledCacheHelper(),
        new ProfilingLogger(
            Mock.Of(),
            Mock.Of()), true);

    //Setup UmbracoContext

    //Have to use an inherited instance of boot manager to remove methods we can't use
    var bm = new CustomBoot(new UmbracoApplication(), serviceContext);
    bm.Initialize().Startup(null).Complete(null);

    string ctAlias = "testAlias";
    string propertyName = "testProp";

    var mockContentType = new Mock();
    mockContentType.Setup(s => s.Alias).Returns(ctAlias);
    mockContentType.Setup(s => s.CompositionPropertyTypes).Returns(new PropertyType[] { new PropertyType(propertyName, DataTypeDatabaseType.Nvarchar, propertyName) });

    mockContentService.Setup(s => s.GetContentType(ctAlias)).Returns(mockContentType.Object);

    var ContentType = PublishedContentType.Get(PublishedItemType.Content, ctAlias);

    var contentId = 2;
    //get a mocked IPublishedContent
    var contentMock = new Mock();
    contentMock.Setup(s => s.ContentType).Returns(ContentType);

    var mockedTypedQuery = new Mock();
    mockedTypedQuery.Setup(s => s.TypedContent(contentId)).Returns(contentMock.Object);

    //give our dynamic query mock to the longer version of the UmbracoHelper constructor
    var helper = new UmbracoHelper(ctx,
        Mock.Of(),
        mockedTypedQuery.Object,
        Mock.Of(),
        Mock.Of(),
        Mock.Of(),
        new UrlProvider(ctx, Mock.Of(section => section.UrlProviderMode == UrlProviderMode.Auto.ToString()), new[] { Mock.Of() }),
        Mock.Of(),
        Mock.Of(),
        new MembershipHelper(ctx, Mock.Of(), Mock.Of()));

    var controller = new BasicTestSurfaceController(ctx, helper);
    var res = controller.BasicHasPropertyAction(contentId, propertyName);
    var model = (bool)res.Model;

    Assert.IsTrue(model);

    //clean up resolved so we can use this again...
    appCtx.DisposeIfDisposable();
}

Let's break this down a bit.

First we create a ServiceContext with a mocked IContentTypeService and a ILocalizedTextService. We will use the IContentTypeService.

I left the ApplicationContext call in to show us passing in the ServiceContext.

Next we start our custom CoreBootManager to setup the resolvers which PublishedContentType.Get uses.
Then we mock a IContentType, setting the alias and a collection of properties. At least one of the properties needs to be the property we are looking for.
We setup the IContentTypeService from before to return the IContentType we just mocked.
We call PublishedContentType.Get with the type of Content (if we had media or member, we’d need to use the corresponding type service in the last step) to get a PublishedContentType object back.
Now we mock a IPublishedContent object and give it the PublishedContentType we requested.
We mock a ITypedPublishedContentQuery, which provides the UmbracoHelper.TypedContent call used in the action.
Then we create a new UmbracoHelper, passing in the mocked typed query.
Finally we create the controller and call the action.

Test passes!

(Note: This test requires the app.config configuration discussed above.)

So much boiler plate!

By now you are thinking, “That’s great, but it isn’t worth the time to learn.” Which brings me to the self-promotion part of the article.

https://github.com/thantos/SAS.UmbracoUnitTesting

In the above link you will find example code which runs tests for all of the operations I have resolved against SurfaceController, RenderMvcController, and UmbracoApiController.

UmbracoUnitTestHelper

In addition, a helper, aptly named UmbracoUnitTestHelper, is included in the UmbracoUnitTesting assembly. The UmbracoUnitTestHelper greatly reduces the amount of boilerplate code required to have passing tests. This assembly also includes the MockServiceContext class, which mocks and initializes all Umbraco services and gives them to a ServiceContext object.

For example, here is the HasProperty test from before with the helper:

[TestMethod]
public void HelperHasPropertyTest()
{
    //Uses our special service context object (it mocks all services!!)
    var mockServiceContext = new MockServiceContext();

    var appCtx = UmbracoUnitTestHelper.GetApplicationContext(serviceContext: mockServiceContext.ServiceContext);
    var ctx = UmbracoUnitTestHelper.GetUmbracoContext(appCtx);

    UmbracoUnitTestHelper.StartCoreBootManager( UmbracoUnitTestHelper.GetCustomBootManager(serviceContext: mockServiceContext.ServiceContext));

    string propertyName = "testProp";
    UmbracoUnitTestHelper.SetupServicesForPublishedContentTypeResolution(mockServiceContext, new[] { UmbracoUnitTestHelper.GetPropertyType(alias: propertyName) });

    var contentId = 2;
    //get a mocked IPublishedContent           
    var contentMock = UmbracoUnitTestHelper.GetPublishedContentMock();

    var mockedTypedQuery = new Mock();
    mockedTypedQuery.Setup(s => s.TypedContent(contentId)).Returns(contentMock.Object);

    //give our dynamic query mock to the longer version of the UmbracoHelper constructor
    var helper = UmbracoUnitTestHelper.GetUmbracoHelper(ctx, typedQuery: mockedTypedQuery.Object);

    var controller = new BasicTestSurfaceController(ctx, helper);
    var res = controller.BasicHasPropertyAction(contentId, propertyName);
    var model = (bool)res.Model;

    Assert.IsTrue(model);

    //clean up resolved so we can use this again...
    UmbracoUnitTestHelper.CleanupCoreBootManager(appCtx);

That’s a 24 line improvement!

UmbracoUnitTestEngine (Alpha)

The helper is great and all, but it still requires some knowledge around what an operation needs to pass and of the inner workings of Umbraco. When testing, I really don’t want to have to think about each case separately, I just want to test the code I wrote.

Thats where the UmbracoUnitTestEngine comes in handy. The intention is to make simple english like calls requesting/setting the mock data our test Umbraco environment should provide.

The CurrentPage test becomes...

[TestMethod]
public void EngineCurrentPageTest()
{
    var content = _unitTestEngine.WithCurrentPage("Test");

    var controller = new BasicTestSurfaceController();
    _unitTestEngine.RegisterController(controller);
    var res = controller.BasicCurrentPageAction();
    var model = res.Model as string;

    Assert.AreEqual(content.Name, model);
}

And the HasProperty test becomes...

[TestMethod]
public void EngineHasPropertyTest()
{
    string propertyName = "testProp";
    var content = _unitTestEngine.WithPublishedContentPage(contentType: _unitTestEngine.WithPublishedContentType( propertyTypes: new[] { UmbracoUnitTestHelper.GetPropertyType( alias: propertyName) }));

    var controller = new BasicTestSurfaceController(_unitTestEngine.UmbracoContext, _unitTestEngine.UmbracoHelper);
    _unitTestEngine.RegisterController(controller);
    var res = controller.BasicHasPropertyAction(content.Id, propertyName);
    var model = (bool)res.Model;

    Assert.IsTrue(model);
}

What's missing

Since This is still in Alpha, there are several key features missing.

We still cannot start the WebBootManager
GetCulture and GetGridHtml can still not be tested.
There are many more operations which have not be tried yet!
More...

Next Steps

More examples, tests, and cases.
More versions (Currently only 7.4.1 and up are supported)
Creation of a nuget package.
Further expand the UmbracoUnitTestEngine and bring it out of alpha.

Contribute/Utilize

The goal of this project is to inspire and prove unit testing in Umbraco. I believe it will be a group effort to discover cases that are not currently handled and resolve them to benefit the community.

The best way to do that is to use the tools and write more tests!