Umbraco DevOps, Part 1: How to Configure Umbraco to Run on Azure by Simone Chiaretta

In this three-part series, you will learn how to optimize your Umbraco setup for Azure, applying the best practice provided by both Umbraco and Microsoft, and how to automate the deployment of both infrastructure and code using ARM templates and Github Actions.

At the end of the series, you will have a Umbraco setup with the following features:

load-balanced, with autoscaling frontend nodes and backend node;
automated installation of Azure resources and automatic configuration of the links between them;
support for multiple environments for development, testing, staging and production, as well as for running locally without using Azure;
complying with the best practices recommended by Umbraco HQ and Azure in terms of security and performance.

In the first article of the series, you are about to learn why running Umbraco on Azure PaaS (Platform as a Service) platform is different than running on-premises or on IaaS (Infrastructure as a Service), and what you need to do to address those differences.

I published a repository on Github with the code from this article, and tagged the various steps so that you can follow along as the article progresses. On the init tag you can find a standard installation of Umbraco 8.16 with the starter kit.

Why is Azure Different?

The documentation on Our Umbraco explains what you need to do to run Umbraco v8 on Azure App Service, but doesn't tell you why.

The answer to this question can be found on the App Service Documentation and, a bit more detail, on the wiki of Project Kudu, the engine behind Azure Web Sites.

The essential part to retain from the two links is that Azure has two types of files: Persisted Files and Temporary Files.

Persisted files are located under the home folder of the filesystem of your Web App and contain the code of your application. These files are persistent in the sense that they will never change unless you do it. They are also shared among all instances of the application. If you scale out to multiple instances, all the copies of the application will access exactly the same files (not a replica). Also, if Azure decides for any reason to move your application to a different worker, these files are available immediately once the app has restarted. All of this is achieved by mapping the home folder to a UNC network share.

Temporary files are stored on the local disk of the virtual machine running your application. But, unlike the persisted files, they are not shared among instances, and they might be reset to the initial state when restarting the application or if the application is moved to a different worker.

Each type of file has its advantages and disadvantages:

Persisted files: persistent, shared (both an advantage and a problem), but slower (on a network drive).
Temporary files: much faster but volatile.

Minimal Configuration for Running on Azure

Umbraco was developed before Cloud existed, and it stores everything, from cache files to indexes and temporary semaphore-like files in folders under the root of the site. You can immediately see why leaving the default behaviour will lead to performance and file locking problems.

The first set of settings, as explained on Our Umbraco is:

    <add key="Umbraco.Core.MainDom.Lock" value="SqlMainDomLock" />
    <add key="Umbraco.Core.LocalTempStorage" value="EnvironmentTemp" />

The first key, Umbraco.Core.MainDom.Lock sets to SqlMainDomLock, places the lock flag created by Umbraco during the startup on the SQL server instead of the filesystem to avoid file locking issues if multiple instances start at the same time.

The second key, Umbraco.Core.LocalTempStorage sets to EnvironmentTemp, instructs Umbraco to use the local temp folder for storing cache files and other temporary files.

You can find the code so far at the local-disk tag of the repo.

Move the Media Folder Outside of the Application Root

All files uploaded from the media section are stored by Umbraco under the \media folder, directly under the website's root. You might also want to move these files out of the root and into a storage account to get better performances and take advantage of all the features that come with Azure storage accounts.

For this, you'll need to install one NuGet package and add some configuration to the web.config file.

    Install-Package UmbracoFileSystemProviders.Azure.Media

The configuration added by the installer is the following:

<add key="AzureBlobFileSystem.ContainerName:media" value="media" />
<add key="AzureBlobFileSystem.RootUrl:media" value="https://[myAccountName].blob.core.windows.net/" />
<add key="AzureBlobFileSystem.ConnectionString:media" value="DefaultEndpointsProtocol=https;AccountName=[myAccountName];AccountKey=[myAccountKey]" />
<add key="AzureBlobFileSystem.MaxDays:media" value="365" />
<add key="AzureBlobFileSystem.UseDefaultRoute:media" value="true" />
<add key="AzureBlobFileSystem.UsePrivateContainer:media" value="false" />

Once installed and configured with the correct account name and access keys, all media files will be stored and read from the media container in the storage account.

You can read more information on this filesystem provider on its GitHub repository page.

After installing and configuring your project should be like the one on the azure-fs-provider tag.

Setting Up Examine

Examine index files are usually stored under the AppData folder, but since Lucene doesn't work very well with indexes stored on network folders, it must be configured to avoid the problem.

The configuration is different depending on the setup of the system. Load balancing setups need a different configuration than single instances.

Configuration Lucene indexes for single instances

The configuration that works with a single instance is:

<add key="Umbraco.Examine.LuceneDirectoryFactory" value="Examine.LuceneEngine.Directories.SyncTempEnvDirectoryFactory, Examine" />

This key instructs Examine to save the Lucene indexes on the local temp folder and synchronize them to the usual location for faster startup time when the local indexes get deleted.

Configuration Lucene indexes for load-balancing scenario

In a load-balancing setup, you need to set different values in the frontend and the backend.

The backend is configured with the SyncTempEnvDirectoryFactory like with the single instance
The frontend needs to be configured with the TempEnvDirectoryFactory, which stores the indexes only on the local temp folder.

This difference is needed because there might be multiple instances of the frontend, and having them synchronize to the same folder will cause file locking issues and inconsistencies.

If you recall from the beginning of the article, temporary folders are emptied every time the site is restarted, moved to another worker or "scaled out" to multiple instances. This means that the indexes on the frontend nodes need to be recreated from scratch every time. So this is a workable solution only if your index is small and quickly indexed without lengthy operations. You can see the status of the code by looking at the lucene-factory tag.

A better solution is using a centralized search engine. The version of Examine used in v8 is provider-based, and we can replace the default Lucene-based implementation with one based on Azure Cognitive Search.

Configuring Examine to use Azure Cognitive Search

The provider based on Azure Cognitive Search is a commercial package called ExamineX.

First, you need to install the package via NuGet.

Install-Package ExamineX.AzureSearch.Umbraco

Then a few lines of configuration are needed.

<add key="ExamineX.AzureSearchServiceName" value="your-azure-cognitive-search-service-name" />
<add key="ExamineX.SiteFriendlyName" value="Your site's Friendly name associated with your license" />
<add key="ExamineX.AzureSearchKey" value="YOUR-AZURE-COGNITIVE-SEARCH-ADMIN-SERVICE-KEY" />

ExamineX is highly configurable and customizable and supports multiple environments. For example, you can use one search service per environment and specify the environment name via configuration.

<add key="ExamineX.EnvironmentType" value="ByService" />
<add key="ExamineX.EnvironmentName" value="staging" />

And on local development machines you can disable ExamineX (and fall back to Lucene) with:

<add key="ExamineX.Disabled" value="true" /></code>

You can read more about ExamineX on its website: (https://examinex.online/).

The examineX tag in the repo shows all the configuration added so far.

Configuring Explicit Master Scheduling Selection

Now that all changes needed due to the Azure filetypes are covered let's see some other best practices configuration for Umbraco on Azure, starting with the explicit selection of the master scheduling server.

The master server needs write access to all Umbraco tables, while the replica servers only need access to 4 specific and non-content related tables. This way, we are also reducing the site's attack surface.

There are multiple ways of implementing this explicit selection. One is explained on the Our Umbraco documentation, but with time in our team, we developed a configuration-based approach that requires only one registrar.

public class ServerRoleRegistration : IUserComposer
{
    public void Compose(Composition composition)
    {
        composition.SetServerRegistrar(new ServerRoleRegistrar());
    }
}

public class ServerRoleRegistrar : IServerRegistrar
{
    public IEnumerable Registrations => Enumerable.Empty();

    public ServerRole GetCurrentServerRole()
    {
        switch (ConfigurationManager.AppSettings["My.Core.LoadBalancing.ServerRole"].ToLowerInvariant())
        {
            case "master":
                return ServerRole.Master;
            case "replica":
                return ServerRole.Replica;
            default:
                return ServerRole.Single;
        }
    }

    public string GetCurrentServerUmbracoApplicationUrl()
    {
        return null;
    }
}

With this registrar, a configuration key can decide the role of a node.

<add key="My.Core.LoadBalancing.ServerRole" value="Single" />

You can check the tag scheduling-master to verify the code till this point.

Configuring Application Insights

Another best practice to follow when running on Azure AppService is monitoring the site with Application Insights. This service helps a lot with the maintenance and management of the site. You get response times, exceptions, dependency charts, performance counters, AI-based alerts and loads of other information.

There are multiple ways of enabling Application Insights into a .NET-based applications, and you can read about them on the Azure Monitor documentation.

I prefer using the manual approach combined with some custom development to allow setting the instrumentation key via configuration (to allow the configuration of multiple environments).

As the first step, you need to install the NuGet package:

Install-Package Microsoft.ApplicationInsights.Web

This package installs all the various dependencies, creates a new ApplicationInsights.config configuration file and updates the web.config file with the registration for the needed httpModules.

From here, you have to configure the instrumentation key. The first option is to hard-code it inside the ApplicationInsights.config file. This option requires minimum changes but doesn't allow different keys per environment or disable tracking.

At the bottom of the file, you should have the following line:

<InstrumentationKey>your-instrumentation-key-here</InstrumentationKey>

Replace it with the key you find in the Azure portal, and you are good to go.

To set this key via an app setting, you need to add a new Composer to inject the key at runtime.

    public class ApplicationInsightsConfiguration : IUserComposer
    {
        public void Compose(Composition composition)
        {
            var AiEnabled = ConfigurationManager.AppSettings["My.Azure.AI.Enabled"].InvariantEquals("true");
            var AiInstrumentationConnectionString = ConfigurationManager.AppSettings["My.Azure.AI.InstrumentationConnectionString"];
            if (!AiEnabled)
            {
                Microsoft.ApplicationInsights.Extensibility.TelemetryConfiguration.Active.DisableTelemetry = true;
                return;
            }

            Microsoft.ApplicationInsights.Extensibility.TelemetryConfiguration.Active.ConnectionString = AiInstrumentationConnectionString;
        }
    }

Then you can add these 2 additional settings in your web.config file:

<add key="My.Azure.AI.Enabled" value="true" />
<add key="My.Azure.AI.InstrumentationConnectionString" value="InstrumentationKey=your-instrumentation-key-here;" />

Notice that we are using a connection string rather than the Instrumentation Key: this is now the recommended approach.

The installation of Application Insights doesn't always go smoothly: you can check the final step of this article on the app-insights tag.

Securing the Backend

You should probably do this in any deployments, not just on Azure, but it's better to configure the backend node to require HTTPS. This is done by updating an existing key in the web.config, setting it to true.

<add key="Umbraco.Core.UseHttps" value="true" />

Conclusions

At the end of this article, you have created a Umbraco site:

Optimized the configuration to run on the filesystem of Azure App Service.
Installed and configured the UmbracoFileSystemProvider for Azure.
Configured Examine to run by leveraging Azure Cognitive Search via ExamineX (or by optimizing Lucene DirectoryFactories).
Configured load-balancing with explicit master scheduling selection.
Installed and configured Application Insights.
Secured the backend.

To run the site, you need to create, configure and link together 11 Azure resources (per environment):

App Plan for the Frontend nodes
- App Service for the Fronted nodes
- Autoscaling for the Frontend nodes
App Plan for the Backend node
- App Service for the Backend node
Azure Storage Account
- Storage Blob Container
Azure Cognitive Search
Azure SQL Server
- Azure SQL Database
Application Insights

As you can imagine (or know if you have already done it before), using the Azure portal to configure everything is a tedious, error-prone and non-reproducible operation that cannot be automated.

In the next article of the series, you will learn the basics of the ARM template syntax. You will automatically create all the resources and automatically set all the references between them and the Umbraco site.

Articles in this series

Umbraco DevOps, Part 1: How to Configure Umbraco to Run on Azure
Umbraco DevOps, Part 2: Automating Azure Resources Creation with ARM Templates
Umbraco DevOps, Part 3: Implementing a CI/CD Workflow Using GitHub Actions

Become a Patron!