DFE 1.1.0 - 1 Understanding DAM For Episerver

Owned by Martin Ulisse
Jun 14, 2019

The Digizuite™ DAM for Episerver connector consists of a fairly large and advanced integration with many moving parts. The goal for this chapter is to give a bit of a basic understanding of how the parts fit together. Some concepts requires deeper explanation and examples and they are split into their own chapters, following this.

The DigizuiteClient

A central component to the integration is the DigizuiteClient. It's exposed through dependency injection using it's interface, IDigizuiteClient and is a singleton. It is the central point that all the other components of the integration use for the actual connection and configuration to the DAM Center. If you, as a developer, need to do code that for example queries the DAM for specific assets, you'll most likely find the need to use the ServiceLocator in Episerver to get an instance of IDigizuiteClient and use that as your starting point.

The DigizuiteClient initializes itself by connecting to various services in the DAM Center and fetches relevant configuration and settings defined in the DAM. For example which languages, media formats, sort orders, media types, metadata and crops that are available. It then also retrieves the DAM's "Media Manager Menu" tree structure which is exposed in Episerver as the folder structure of the assets. 

But - it's also in charge of numerous other important tasks:

  • Keeping track of the individual Service connectors to DAM Center and their assigned languages.
  • Caching relevant in-memory data
  • Client notifications
  • Connection status in case of resilience needed for unstable DAM connections
  • It contains the main methods used for all CRUD interactions with the DAM center.
  • It builds the proper Episerver Content objects for the corresponding Assets
  • Handling uploads
  • In-memory draft storage of asset metadata changes
  • Much, much more...

DigizuiteContentProvider

The content provider is the main class that plug into Episervers provider pattern to expose assets and folders in Digizuite as if they were Episerver Content.

By default it's registered programmatically by an initialization provider and attached to an entry point created below the main "Global Assets" folder in Episerver, called "Digizuite". However that can of course be customized - and you can even choose to register it your self. It contains the needed methods to return the proper content given a content reference, Guid or Url pointing to a Content element in this provider as well as methods to return directory listings and handle upload, save, deletes and many more operations.

For most of it's logic it will essentially encapsulate logic provided by the before mentioned DigizuiteClient.

ID's and GUIDs

In Episerver all content typically have both a ContentReference and a Guid - and in some cases a url. The DAM Center typically exposes only an AssetId (int) or a folderId (int). In order to easily map these over to Episerver ContentReference (that also has an int ID) the convention has been made that negative ID numbers are folders and positive ID numbers are assets. To ensure a uniform way of dealing with this - even if the future changes this approach, there is a helper class called ConversionHelper that provides methods to easily convert between the needed formats.

The GUIDs are slightly more tricky. For the content provider to work well, it both needs to be able to quickly identify if a given GUID belongs to the current provider, and if so - which corresponding asset-id should be looked up. As this lookup happens many times per request it isn't feasible to do a database mapping lookup every single time. Instead the integration uses a 'base' GUID defined in configuration and simply stores the int ID in the first 4 bytes (32-bit) of the GUID. The ConversionHelper also contains methods for dealing with this. However, under normal circumstances you shouldn't need to go to the level of complexity and to indicate that, those methods are in an "internal" namespace.

Drafts and versioning

The DAM Center doesn't support the same concept of draft/versioning/publish as Episerver does. In Episerver every single change results in a new draft version being created or saved, where as in the DAM Center changes are not saved unless you actively decide to click "Save". In order to support updating meta data fields in the DAM from Episerver, the DigizuiteClient will instead keep track of the changes in-memory, until the editor decides to Publish - at which point the changes will be sent back to the DAM Center and the asset updated.

Folder Hierarchy

Once again, the DAM Center and Episerver has slightly different opinions on the concept of a folder structure. Where it in Episerver is important that every single content item lives under exactly 1 parent, Digizuite DAM considers the folder nothing but a meta-data attribute for the asset that you can filter on. This means that all assets from Digizuite lives under the root (entry point in Episerver) but also can be visible in one or more child folders. 

Cropping and Crops

The DAM Center has a series of predefined crop settings with different aspect ratios (that can be changed in the DAM Center). Typically named like "Square", "Banner", "Widescreen" and so on. These are the crops that you can create for the various images. In Episerver, just as in the DAM Center they will appear as seperate assets/content objects, but if a content object is a crop of another it will hold a reference to the 'parent' in it's properties, and if will have the name of the crop listed in it's "CropName" property.

Media Formats

Media formats on the other hands are more to be considered different views or blobs of the same asset/content object. The DAM Center will also have a defined list of media formats to be made available for the different asset types (video, images and so on). Upon upload it will automatically create the different predefined media formats. The media formats are in Episerver either exposed as Blob properties on the asset content model (and can be accessed directly in the url) or they can be fetched using one of the extension helper methods.

Learn more:

Dealing with Media Formats

Rendering Assets

Modelling Asset Content Types

Caching

The DAM For Episerver integration heavily relies on cache to improve performance. The caching mechanism has multiple working parts:

  1. Binary Disk Cache. Most media assets (except for videos) are lazily cached on disk. By default in the "Digizuite" folder (but that can be customized in configuration) below "App_Data". The folder contains a subfolder for each cached asset (named with the AssetID) and in each folder there are copies of the cached media formats of that asset (named by the media format ID from Digizuite). When an asset is being requested from the blob provider it will always use the cached asset from the local disk before attempting to fetch it from Digizuite. A cache timeout can be set in configuration, but the binary disk cache will also reflect Digizuite changes reflected by the Sync job (Scheduled Job).
  2. Binary Disk Temporary Upload Cache. When an editor uploads files to Episerver that should be sent on to Digizuite, they will temporarily be stored in the "DigizuiteTemp" (configurable) folder below App_Data. Then, a background process will send them to Digizuite without slowing down the editorial experience. After successful upload to Digizuite the temporary files will be automatically removed.
  3. In-memory Asset Metadata Cache. The Digizuite Client that retrieves all metadata automatically caches asset metadata in memory, using the Episerver Synchronized Caching mechanism. Just like with the binary data, this cache is both subject to a time-out and adheres to the Sync job.
  4. Rebuild Cache Job. This scheduled job will try to load all assets in the various Digizuite folders and cache them on Episerver for fast access. This can be useful for example if you have switched DAM Centers and need to clear the cache and fill it with new assets.
  5. Sync Job. This scheduled job will call Digizuite and get changed/updated/removed assets since it was last called. Then, it will remove those assets from cache - as well as alert other Episerver front-end servers to clear their cache for those items. This is a fairly lightweight job and can easily be configured to run very frequently (every few minutes).
  6. WarmUp Job. This final scheduled job typically runs automatically on site initialization (configurable) and will 'warm up the cache' meaning that it will fetch and cache the first batch of assets in every folder to ensure the editorial experience is fast.


Resilience

Besides caching - which has the main purpose of improving performance - there is Resilience, which is there to improve stability in case of fault scenarios. The general philosophy behind the Digizuite/Episerver integration is that should the connection fail between the 2 systems for whatever reason, it should never result in a site crashing. Furthermore, the site should also still show main assets and it should be possible to work in edit-mode - albeit without full Digizuite functionality.

If Media resilience is enabled (default, but can be switched off for performance in configuration), then that means that every time an editor published a content item in Episerver that is referencing any asset on Digizuite, a copy of the asset is stored in the Episerver blob storage, and the metadata of the asset is stored in the Episerver Dynamic Data Store. Then - if a fault/disaster scenario is detected - it will fallback to the previously stored asset. It is worth noting though, that this functionality does not apply to videos, nor does it work with dynamically fetched assets (fetched by custom developer code) - only assets that have been referenced in the regular way will be detected and made resilient.

On top of that, all components of the integration has been made robust, so that even if they are in a fault scenario, they will not crash - but rather report an error (both in error logs and editor notifications) - and offer very limited functionality.

Still - it is of course important to keep in mind that this is purely for disaster scenarios - and if you happen to end up in such a scenario, the main priority should be to reestablish connection! When connection is reestablished, the integration will detect it and re-initialize itself.