The strange case of infinite recursion in an add-in severely crashing Visual Studio 2005 and 2008

Don’t you love when Visual Studio severely crashes? I really hate those moments because I am always afraid that if it is a problem of VS and not a problem in my code, I am in trouble…

Some time ago I explained The strange case of Visual Studio 2008 severely crashing loading an add-in that was caused by Visual Studio commandbars. Fortunately the cause of this problem and a workaround was known.

I also opened a bug report at Microsoft Connect due to Application of custom attribute crashes VS 2008 (VB) still with SP1 applied. This causes a crash in the VB 2008 compiler. While Microsoft closed the report as “not reproducible” I am glad that they have reopened it recently and they sent me some “dumping” tools to install on my computer and send them back more information to get a clue about the problem since it is not reproducible with a few lines of code. Hopefully this one can be fixed because it is preventing me to migrate to VS 2008.

Yesterday I got a new severe crash while debugging my MZ-Tools add-in. The crash was something like:

Problem signature
Problem Event Name: APPCRASH
Application Name: devenv.exe

Fault Module Name: kernel32.dll
Fault Module Version: 6.0.6001.18000
Fault Module Timestamp: 4791a76d
Exception Code: e053534f
Exception Offset: 000442eb

Searching the web I only found a similar issue but it didn’t explain the cause or fix. The line of code crashing in my code was just setting the value of a class field, which seemed quite innocent. On Windows Vista the dialog didn’t offer the chance of sending it to MS but today on a Windows XP machine the dialog allowed me to send the bug report. But more importantly, after more careful debugging I realized that my code was calling the method of a class that called another method, and then other, and then other, and 10 calls later it was calling again the original method, causing an circular dependency that eventually crashed Visual Studio. For example this code reproduces the problem:

public void OnConnection(object application, ext_ConnectMode connectMode, object addInInst, ref Array custom)
{
   switch (connectMode)
   {
      case ext_ConnectMode.ext_cm_Startup:
      case ext_ConnectMode.ext_cm_AfterStartup:
         f1();
         break;
   }
}
void f1()
{
   f2();
}
void f2()
{
   f1();
}

While the fault was in my code (I have already fixed it), I wondered if Visual Studio could show better information, such as an StackOverflowException or OutOfMemoryException or similar  (that would give me a hint about a circular dependency) rather than a crash in the kernel32.dll module. And guess what? I did a test and VS.NET 2003 shows a more friendly message with the StackOverflow exception indicating a too deep recursion. Somehow something was changed in VS 2005 and now Visual Studio behaves much worse crashing completely. So I have opened a bug report at Microsoft Connect (you are invited to vote to get it fixed):

Add-in causing stack overflow causes “Microsoft Visual Studio has encountered a problem and needs to close”
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=414301

It happens that I had seen this very same problem in my code a couple of years ago but I had totally forgotten that such crashes in VS are sometimes produced by circular dependencies of methods in the code. I hope this helps other people (and me) when searching on the web a crash in a VS add-in in
Exception Offset 000442eb of Kernel32.dll.

More on the The strange case of sporadic COMException 0x800A01A8 calling CommandBarButton.Delete from a Visual Studio add-in (solved)

Tired of the strange case of sporadic COMException 0x800A01A8 calling CommandBarButton.Delete from a Visual Studio add-in, but very annoying, I decided to take a closer look and solve it. First, I noticed that my MZ-Tools add-in only calls CommandBarButton.Delete for buttons that places on VS built-in toolbars. For buttons that places on its own toolbar, I don’t call CommandBarButton.Delete since I will destroy the toolbar anyway and that will remove its buttons (which is also faster). So, if your add-in doesn’t add buttons to built-in toolbars of VS chances are that you’ll never experience this COMException. Alas, my add-in adds buttons to VS built-in toolbars that must be removed. For these, I took the following approach:

  • Rather than calling CommandBarButton.Delete when the add-in is unloaded, and I remove the command calling Command.Delete (which also automatically deletes all CommandBarButtons created from that command) and I create it again.
  • As you may know, in general add-ins should not delete commands when unloaded because:
    • Command keyboard bindings are lost. To solve this, before deleting the command I get the command keyboard bindings (Command.Bindings) and I restore them when re-creating the command.
    • If the user has created a button from that command on some other toolbar of her choice (see “Addition of a button of the add-in to a toolbar not belonging to it” case in my article HOWTO: Handling buttons and toolbars user customizations from a Visual Studio add-in), that button will be lost when the add-in is unloaded. There is no solution to this case, but I can live with that. Hopefully my users don’t need to create buttons from my add-in on other toolbars (I strive for providing good button/toolbar customization inside the add-in).