Step by Step, Little by Little (1)

Posted on May 3, 2015 in umbraco

(Move a little closer to me)

As of today, this site is running the latest 7.3.0 branch + the latest dev-v7-contentcache branch. This means that (a) the highly work-in-progress 7.3.0 branch can be upgraded to, which is a good thing in itself and (b) the content cache branch is stable enough.

What's up?

Much of the recent work has not been focused on the content cache per se. After the Xml cache was properly isolated, it appeared that the rest of Core would not trigger the proper events that were required in order to maintain the cache consistent with the content, especially in distributed (load-balanced) environments. In fact, it appeared that some of Core services were not consistent enough at all.

So, during the past weeks, I have essentially worked on the content, content type, data type, media and member services and refactored their events, their locking policies, and most of how they work internally, in order to improve global consistency.

Among other things:

• These services now rely on database-level transactions and locks that guarantee that you cannot modify a content type and some content at the very same time and end up with a corrupt database or Xml cache—and because this is database-level, it works accross all distributed nodes.

• The umbraco.config file is now protected by a system-wide semaphore-based lock, that ensures that when the web application restarts, the application that is starting cannot read from the file before the application that is shutting down is done writing to the file—something that regularly corrupted the Xml cache file.

With a few extra improvements to the Distributed Cache refactoring initiated by Shannon, we have been able to test-load a two-nodes distributed system, where 16 concurrent users hit both nodes and continuously create content on both nodes with a 40% chance for each request to also cause an application restart... eventually ending up with more than 2000 created documents, and a clean, non-corrupt umbraco.config file on both nodes.

This is an important milestone for several reasons. It means that, from content creation to content rendering, there practically isn't any legacy code running anymore. Which means that understanding, debugging, improving, etc. becomes much easier. It also means that Core is becoming more and more robust and trustworthy.

What's next?

Now that the environment is stable enough, work is resuming on the "new cache", officially code-named NuCache. It is fully object-oriented, ie it directly caches IPublishedContent objects instead of caching an Xml document—while still supporting XPath and Xslt. My next objective is to run this blog on NuCache, ASAP.

Of course much of this is long-term effort; and because 100% backward compatibility cannot always be guaranteed, it probably will only get merged into v8.

However, an important part of the Xml file locking has been back-ported to 7.2.5-RC and is being tested right now, because it should prevent some file-locking issues that some sites have been experiencing during application restart.