IIS Configuration change? Not!

Posted on May 19, 2015 in dotnet , umbraco and edited on October 29, 2015

(this post has been updated—see at the end)

This blog is (at the moment) running on a Windows 2012 R2 Standard server with the builtin IIS 8.5 server, and ASP.NET 4.5.2. For a while, the simple fact of publishing a document would cause the application to restart, with the dreaded message:

Domain Unload
Message:
  IIS configuration change
  HostingEnvironment initiated shutdown
  HostingEnvironment caused shutdown
Stack:
  at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
  at System.Environment.get_StackTrace()
  at System.Web.Hosting.HostingEnvironment.InitiateShutdownInternal()
  at System.Web.Hosting.PipelineRuntime.StopProcessing()

So I went through what this post describes... and reached the same conclusion: Process Monitor says that strictly nothing changes, all the memory limits and other reasons for IIS to restart are OK, and ETW tracing reports:

Event Name: IISGeneral
Event Type: CONFIG_CHANGE_NOTIFICATION
UserData: MACHINE/WEBROOT/APPHOST/OURSITENAME

And the application restarts.

Process Monitor highlights something: right before restarting, there is an important file activity in App_Data—more precisely: Examine's Lucene indexes are updated, optimized, etc. So I tried to disable Examine, editing the ExamineSettings.config file and emptying its /Examine/ExamineIndexProviders/providers element. Guess what? No more restarts.

So the restarts definitively seem to be linked to file activity in the website root.

Some suggest to tweak the fcnMode setting, either in the registry or in web.config. Unfortunately, that does not change anything. At that point I am only guessing: the stack trace begins with the call to PipelineRuntime.StopProcessing which is a COM entry point. It is IIS telling ASP.NET to please shut down, and it is too late for ASP.NET to do anything about it. And fcnMode is an ASP.NET setting with not effect on IIS. And so IIS itself would have some file watchers, that do not quite like seeing a lot of file activity in the website root, and eventually decide to kill the application.

Is this correct? What is the solution? Nobody seems to know. Some have mentionned KB950573 but... that is a 2009 fix for IIS 7 and I don't even want to try and install it. Google leads to more reports, but no solution.

Hello Microsoft?

Update 05/27 2015

Gijs van Dam in the comments below has pointed me to this StackOverflow post and this Umbraco issue which I seem to have overlooked. The server did have KB3000850 installed, so I have installed the pre-released KB3052480, and reverted the ExamineSettings.config changes.

Still need to monitor the server, but as far as I can tell, the issue is fixed.

Oh, and yes, it seems it was an internal IIS error. Read this post (in CZ) for details.

Update 06/18 2015

Microsoft have released hotfix 3052480 for the issue. It is not part of Windows regular updates, so you need to request, download and install the hotfix manually. It looks identical to the pre-released hotfix (and will not install on top of it anyway).

Update 09/02 2015

The update is released as KB3052480 and can be received in Windows Update. Case closed.

Update 10/29 2015

Or, almost closed. There still are some weird issues with site restarts and filesystem watchers, as analyzed by Shannon in this post.