All posts by Carlos Quintero

Announcing the revised MZ-Tools Articles Series with C# samples and more

One of the initiatives that I want to accomplish this year 2009 is to revise and update all the more than 125 articles that I have written all these years since 2004 in the MZ-Tools Articles Series to include the following:

  • Samples in C#.  I have provided all the samples in the articles in VB.NET since it is my preferred development language and I assumed that every C# developer could understand the code. But I have noticed that:
    • To write add-ins in C# is somewhat more difficult due to the lack of optional parameters in C# (version 4.0 will support them, BTW) and other things.
    • Not every C# developer reading the articles bothers to try the VB.NET code to run the sample and then translates it to C#. Instead, they keep inventing code that it is not in the sample.

    So I have decided to make it easy for everybody and provide the samples of add-ins in VB.NET and C#. Macros will remain in VB.NET since C# is not supported as a macro language.

  • Revision of the Visual Studio version(s) that the articles apply to. Many articles were written when VS 2005 and VS 2008 were not released yet. So I am revising if each article still applies to new versions, such as if a bug has been fixed, etc.
  • “Copy” button to copy the code sample to the clipboard. Currently it works only with Internet Explorer. For FireFox it is more complicated and I haven’t found yet a satisfactory solution using the FireFox interfaces or the Flash approach. Security is a major concern in all browsers and operating systems such as Windows Vista and getting access to the clipboard of the client machine is increasingly more difficult. Nonetheless, FireFox is my preferred browser, so I will keep trying in the future.
  • Revision of the nomenclature prefixes used in the sample code. I am now trying to use the Microsoft guidelines, avoiding prefixed for types.
  • Revision of the “Related Articles” section and the end of each article. Some articles will reference now other articles that were not written yet when they were written.
  • Hopefully less spelling and grammar mistakes :-). I am from Spain so I don’t speak English natively, and the editor that I use lacks a spell checker.

I have created a spreadsheet with all the articles that need modification and this happens to be a huge task that will take me months keeping a pace of one article per workable day, but I think it will be worth. This also means that this year will be less new content, but I have still some topics to write about.

Here you have a sample of a revised article:

HOWTO: Add a popup command bar to the context menu of a code window of Visual Studio .NET

The strange case of infinite recursion in an add-in severely crashing Visual Studio 2005 and 2008

Don’t you love when Visual Studio severely crashes? I really hate those moments because I am always afraid that if it is a problem of VS and not a problem in my code, I am in trouble…

Some time ago I explained The strange case of Visual Studio 2008 severely crashing loading an add-in that was caused by Visual Studio commandbars. Fortunately the cause of this problem and a workaround was known.

I also opened a bug report at Microsoft Connect due to Application of custom attribute crashes VS 2008 (VB) still with SP1 applied. This causes a crash in the VB 2008 compiler. While Microsoft closed the report as “not reproducible” I am glad that they have reopened it recently and they sent me some “dumping” tools to install on my computer and send them back more information to get a clue about the problem since it is not reproducible with a few lines of code. Hopefully this one can be fixed because it is preventing me to migrate to VS 2008.

Yesterday I got a new severe crash while debugging my MZ-Tools add-in. The crash was something like:

Problem signature
Problem Event Name: APPCRASH
Application Name: devenv.exe

Fault Module Name: kernel32.dll
Fault Module Version: 6.0.6001.18000
Fault Module Timestamp: 4791a76d
Exception Code: e053534f
Exception Offset: 000442eb

Searching the web I only found a similar issue but it didn’t explain the cause or fix. The line of code crashing in my code was just setting the value of a class field, which seemed quite innocent. On Windows Vista the dialog didn’t offer the chance of sending it to MS but today on a Windows XP machine the dialog allowed me to send the bug report. But more importantly, after more careful debugging I realized that my code was calling the method of a class that called another method, and then other, and then other, and 10 calls later it was calling again the original method, causing an circular dependency that eventually crashed Visual Studio. For example this code reproduces the problem:

public void OnConnection(object application, ext_ConnectMode connectMode, object addInInst, ref Array custom)
{
   switch (connectMode)
   {
      case ext_ConnectMode.ext_cm_Startup:
      case ext_ConnectMode.ext_cm_AfterStartup:
         f1();
         break;
   }
}
void f1()
{
   f2();
}
void f2()
{
   f1();
}

While the fault was in my code (I have already fixed it), I wondered if Visual Studio could show better information, such as an StackOverflowException or OutOfMemoryException or similar  (that would give me a hint about a circular dependency) rather than a crash in the kernel32.dll module. And guess what? I did a test and VS.NET 2003 shows a more friendly message with the StackOverflow exception indicating a too deep recursion. Somehow something was changed in VS 2005 and now Visual Studio behaves much worse crashing completely. So I have opened a bug report at Microsoft Connect (you are invited to vote to get it fixed):

Add-in causing stack overflow causes “Microsoft Visual Studio has encountered a problem and needs to close”
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=414301

It happens that I had seen this very same problem in my code a couple of years ago but I had totally forgotten that such crashes in VS are sometimes produced by circular dependencies of methods in the code. I hope this helps other people (and me) when searching on the web a crash in a VS add-in in
Exception Offset 000442eb of Kernel32.dll.

More on the The strange case of sporadic COMException 0x800A01A8 calling CommandBarButton.Delete from a Visual Studio add-in (solved)

Tired of the strange case of sporadic COMException 0x800A01A8 calling CommandBarButton.Delete from a Visual Studio add-in, but very annoying, I decided to take a closer look and solve it. First, I noticed that my MZ-Tools add-in only calls CommandBarButton.Delete for buttons that places on VS built-in toolbars. For buttons that places on its own toolbar, I don’t call CommandBarButton.Delete since I will destroy the toolbar anyway and that will remove its buttons (which is also faster). So, if your add-in doesn’t add buttons to built-in toolbars of VS chances are that you’ll never experience this COMException. Alas, my add-in adds buttons to VS built-in toolbars that must be removed. For these, I took the following approach:

  • Rather than calling CommandBarButton.Delete when the add-in is unloaded, and I remove the command calling Command.Delete (which also automatically deletes all CommandBarButtons created from that command) and I create it again.
  • As you may know, in general add-ins should not delete commands when unloaded because:
    • Command keyboard bindings are lost. To solve this, before deleting the command I get the command keyboard bindings (Command.Bindings) and I restore them when re-creating the command.
    • If the user has created a button from that command on some other toolbar of her choice (see “Addition of a button of the add-in to a toolbar not belonging to it” case in my article HOWTO: Handling buttons and toolbars user customizations from a Visual Studio add-in), that button will be lost when the add-in is unloaded. There is no solution to this case, but I can live with that. Hopefully my users don’t need to create buttons from my add-in on other toolbars (I strive for providing good button/toolbar customization inside the add-in).

The lack of code model support inside methods

As you probably know, the code model (EnvDTE.ProjectItem.FileCodeModel) of the automation model of Visual Studio doesn’t provide information of variables, constants, statements, etc. inside a method. At most, you get the parameters of the method.

Eric Lippert explains in this post how the C# (or other compilers) work. It happens that they need two steps: the first one parses and resolves everything but the code inside the methods. Once the compiler knows about every possible type, the second step parses method bodies.

I am not sure if the file code model of EnvDTE relies on the internal .NET parsers or parses on its own but in any case it seems to follow the same two-step approach, but unfortunately without the second step (maybe for performance reasons, maybe because the file code model should parse on its own and MS never implemented it). Needless to say, to parse a method body can be quite complex and error prone (MS has whole teams for the development and testing of parsers). Hopefully we’ll have some day access to that information when the compiler is a service, as MS announced in the PDC 2008.

HTML/ASP.NET extensibility for VS 2010: it’s now or never

If you have written add-ins for Visual Studio that deal with Windows Forms, and have tried to make it work with HTML/ASP.NET pages and controls, you already know that, well, it is quite difficult if not impossible because:

1) HTML pages contain HTML controls, while ASP.NET pages can contain HTML controls or ASP.NET controls

2) VS 2008 changed the internal designer arquitecture for HTML/ASP.NET

3) VS doesn’t expose good extensibility points for add-ins (and maybe not even for packages)

There were lots of questions in the forums about all this and I wrote a post a year ago:

The (lack of) IDesignerHost in Visual Studio 2008 (Orcas) ASP.NET Web pages
https://www.visualstudioextensibility.com/2007/12/18/the-lack-of-idesignerhost-in-visual-studio-2008-orcas-asp-net-web-pages

OK, now the good news: Mikhail Arkhipov, the person within the Visual Studio team that can change this (I exhanged e-mails about this in the past) has just posted this in his blog:

What kind of extensibility would you like to see in HTML/ASPX editor?
http://blogs.msdn.com/mikhailarkhipov/archive/2009/01/19/what-kind-of-extensibility-would-you-like-to-see-in-html-aspx-editor.aspx

So, Microsoft is listening. It is our chance to speak 🙂

I will write Mikhail in a couple of days with a detailed explanation and sample of what I would like to do from within an add-in.

The strange case of sporadic COMException 0x800A01A8 calling CommandBarButton.Delete from a Visual Studio add-in

I don’t have an answer to this strange case, but after 4+ years getting reports from users, I will write about it just in case someone can post a solution.

When using a temporary user interface (not a permanent one), add-ins typically add CommandBarButtons to commandbars using Command.AddControl when loaded. When the add-in is unloaded, it must remove its buttons calling CommandBarButton.Delete. You can see a sample in the following article of mine: HOWTO: Adding buttons, commandbars and toolbars to Visual Studio .NET from an add-in

The problem is that sometimes you get COMException 0x800A01A8 when calling CommandBarButton.Delete. There are some facts about this problem:

  • It happens very sporadically (I have received less than 20 user reports in all these years).
  • When I ask the users if the problem is reproducible, they always answer that no, that the add-in unloads fine next time.

So the occurrence is really really tiny and it is not a bug in the code of the add-in since it works fine almost always, but nonetheless I would like to erradicate it.

Searching the Web, it seems that this problem appears more commonly with add-ins for Office, but no forum seems to deal with Visual Studio add-ins or provides an answer.

A couple of years ago I posted in the MSDN Forum:

To Microsoft: Sporadic COM exceptions adding/removing commandbars and buttons from Visual Studio add-ins
http://social.microsoft.com/Forums/en-US/vsx/thread/8fff7d90-7867-4608-a8b6-ba9362d91411/

In that post I exposed three reproducible cases of COM Exceptions, but I mentioned that there were more COM exceptions. Reviewing the user reports in my e-mail system, I think that in fact there is only a non-reproducible COMException 0x800A01A8 and it happens always calling CommandBarButton.Delete.

Craig Skibo (former member of the Visual Studio extensibility team) answered to that post but since I posted 3 reproducible cases and 1 non-reproducible case the discussion was a bit messed and in the part that dealed with CommandBarButton.Delete he argued that buttons created with Command.AddControl shouldn’t be destroyed calling CommandBarButton.Delete, but IMHO he is wrong or his workarounds are not really workarounds.

So, at this point I don’t know it I should ignore that exception because I am not sure if the COM exception is caused because the button is not already there (it would be OK to ignore the exception) or if I would get a duplicated button the next time that the add-in is loaded (in this case I would like to notify the previous exception to know why I am getting a duplicated button).

Anyone is experiencing this problem? You can either post a comment or contact me by e-mail.

Finally, wouldn’t be nice to be able to debug the code of Microsoft.VisualStudio.CommandBars.dll when the exception is thrown? Or better yet, a managed IDE where the menus and toolbars are no longer based on Office COM-based commandbars?

Update (February 3, 2009). See More on the The strange case of sporadic COMException 0x800A01A8 calling CommandBarButton.Delete from a Visual Studio add-in (solved)

The strange case of “%ALLUSERSPROFILE%\Datos de programa\Microsoft\MSEnvShared\AddIns” add-in file path

(Or why it is a bad idea for an add-in to add add-in file paths to Visual Studio)

When Visual Studio 2005 introduced XML-based add-ins, which used an .AddIn file rather than COM registration), rather than providing just two fixed folders to place such files (one for all users, other for the current user), Microsoft provided five (5!!) folders, listed in the Tools, Options menu, Environment, Add-in/Macros Security section.

Not only that, Visual Studio allows to add more folders using the Add… button or programmatically writing to the “HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\8.0\AutomationOptions\LookInFolders” registry key (VS 2005).

Not only that, you will have a difficult time expanding VS-proprietary placeholders like %VSAPPDATA%, %VSCOMMONAPPDATA% or %VSMYDOCUMENTS%, if your add-in setup wants to use one of those folders.

Not only that, Visual Studio 2008 added a sixth folder, %ALLUSERSDOCUMENTS%\Microsoft\MSEnvShared\Addins.

Not only that, Visual Studio 2008 actually fails to look for XML files in that new folder (see HOWTO: Using the Process Monitor (ProcMon) tool to diagnose Visual Studio add-ins problems)

And not only that (finally), Visual Studio hardcoded “Application Data” in one of the paths (“%ALLUSERSPROFILE%\Application Data\Microsoft\MSEnvShared\Addins”) without taking into account that that folder is localized and in other languages it doesn’t exist with that name on the hard disk. For example, in Spanish, it is should be “%ALLUSERSPROFILE%\Datos de programa\Microsoft\MSEnvShared\AddIns”.

I documented all that in the article INFO: Default .AddIn file locations for Visual Studio add-ins.

Guess what? Yesterday, while writing an utility that scans the add-in file paths, I noticed that on my Spanish computer Visual Studio 2005/2008 listed “%ALLUSERSPROFILE%\Datos de programa\Microsoft\MSEnvShared\AddIns” as one of the add-in file paths in the Tools, Options menu, Environment, Add-in/Macros Security section. Indeed VS 2005 listed 6 folders instead 5, and VS 2008 listed 7 folders instead of 6.

How could that be? At first I thought that the service pack 1 of both IDEs could have “fixed” the problem adding the localized version of that path maintaining the wrong one (with “Application Data” hardcoded). But I quickly discarded that explanation because I had installed SP1 from long time and those folders were not there. Then I remembered that I had installed recently an add-in (I won’t cite its name), and looking into the contents of the “%ALLUSERSPROFILE%\Datos de programa\Microsoft\MSEnvShared\AddIns” folder, there was the .AddIn files of this add-in. So, the setup of this add-in is placing the .AddIn file in that folder, AND it is adding that folder to the “HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\8.0\AutomationOptions\LookInFolders”. In other words, it is trying to “fix” what Microsoft did wrong with that folder. But I think this is a bad idea for this reason: if this add-in is installed on a non-English computer and the user happens to be a developer of add-ins who is unaware of the problem of the hardcoded “Application Data” path, he will see two paths listed:

%ALLUSERSPROFILE%\Application Data\Microsoft\MSEnvShared\Addins
%ALLUSERSPROFILE%\Datos de programa\Microsoft\MSEnvShared\AddIns

And certainly he will use the second one because any non-English developer knows that it is the correct one (using an API call to get the actual localized folder), but when his add-in is installed on another machine, the second path is not in the list of paths that VS scans, and the add-in won’t appear in the Add-In Manager.

Placing the .AddIn file in the first path (%ALLUSERSPROFILE%\Application Data\Microsoft\MSEnvShared\Addins) is also a bad idea because the setup will create that English folder on non-English computers, and then the computers will have those two paths physically on the hard disk.

So, what to do? I’d recommend to use the “%VSCOMMONAPPDATA%\AddIns” folder instead, which is also for all users but it neither mess the list of paths of Visual Studio nor creates English folders on non-English computers. The only drawback is that the folder is actually different each VS version:

For Windows XP:

  • VS 2005: “C:\Documents and Settings\All Users\Application Data\Microsoft\VisualStudio\8.0\Addins”
  • VS 2008: “C:\Documents and Settings\All Users\Application Data\Microsoft\VisualStudio\9.0\Addins”

For Windows Vista:

  • VS 2005: “C:\ProgramData\Microsoft\VisualStudio\8.0\Addins”
  • VS 2008: “C:\ProgramData\Microsoft\VisualStudio\9.0\Addins”

To get it programmatically the custom action of your setup that copies the .AddIn file must call the API that returns special folders with the parameter CSIDL_COMMON_APPDATA and then concatenate
“Microsoft\VisualStudio\8.0\Addins” for VS 2005 or “Microsoft\VisualStudio\9.0\Addins” for VS 2008. For example:

  • Using the .NET Framework the call is System.Environment.GetFolderPath(System.Environment.SpecialFolder.CommonApplicationData)

Being the path different for each VS version you can’t use a single .AddIn file with two hosts inside (HostApplication tags) for VS 2005 and VS 2008, but that’s a minor extra work for the setup of an add-in, and we all know that software is all about our users, not about us the developers…

Three things from Eclipse that would be nice to have in Visual Studio to develop extensions

While I have been working with Visual Studio .NET since the first version (2002), some years ago I started a start-up with a friend to develop crossword games for J2ME-enabled mobile phones. So I learned Java, downloaded the Eclipse IDE and the EclipseME plug-in (now Mobile Tools for Java) to develop J2ME MIDLets, and I had a lot of fun 🙂

There are three things from Eclipse that impressed me from the start that all these years developing add-ins for Visual Studio I miss a lot:

1) There is no setup.

You download a zip file, unzip it to some folder and that is all. Compare this with the installation of Visual Studio that even for service packs is increasingly longer. This is also true for other Microsoft products like Office, but at least Office will “fix” the problem when the Web-enabled version appears, so you will be able to create documents or spreadsheets like you can do it today with Google Docs. If you develop extensions for Visual Studio you need to have several versions and even service packs installed on your machine, and installing all them is painful, and you can’t have a version without service pack and other with service pack on separate folders, and you can’t install CTPs or betas because nobody could cleanup and restore everything when uninstalled, and the “solution” to all this is to use… virtual machines, which have their own dosis of problems (at least Virtual PC), they are increasingly heavier, require more memory on your computer and you need some more steps to deactivate CTP timebombs. Imagine that you could unzip Visual Studio versions, service packs or CTPs/betas on separate folders of your machine without touching the Windows registry…

2) You can have the source code for the IDE.

I can not emphasize enough how important is this when you develop some non-trivial extension for Visual Studio. But until recently even the source code of the .NET Framework was not available, so tools like .NET Reflector to decompile assemblies appeared, a tool that would have made rich its author if he would have charged just five dollars to each .NET developer that uses it rather than making it freeware (thanks, Lutz). Microsoft provided the source code of the .NET Framework with the release of Visual Studio 2008, and even the mechanism to debug it if you need to do so. But the source code of Visual Studio remains closed so you can’t understand how some internals of VS work, not to mention debug it, why your add-in or package doesn’t load or give you a cryptic COM error, etc. While it is understandable that some portions should remain closed (such as the .NET compilers or parsers), others could be perfectly provided with source code. All new additions to VS are now managed (not native), so you can decompile them to try to understand how they work. It would be nice to have the source code without decompilation and be able to debug your extension along with Visual Studio code.

3) It was thought with extensibility in mind from scratch

I have not used extensibility in Eclipse but from what I have read, it was fully extensible from scratch and everything was made with a minimal shell with lots of plug-ins for the features. Visual Studio, on the other hand, only provided extensibility through add-ins, macros and templates in the initial VS.NET 2002. The VSIP program to develop 3rd party packages didn’t appear until the VS.NET 2003 release. You didn’t have something named Visual Studio SDK until the VS 2005 release. Something like the Visual Studio Shell didn’t appear until the VS 2008 release. All that complicated by an IDE that was born as a native, COM (not .NET) application to develop mainly .NET applications but that required extensions to be COM. We are still suffering that…

The good news is that Microsoft is working a lot and really hard on point #3 to catch up. I wish they can think about points #1 and #2 too.

VB.NET 10.0 and C# 4.0 future directions: co-evolution and “compiler as service”

In the last days I have watched (among others) two videos of the PDC 2008 about subjects that are always of interest for developers of extensions for Visual Studio, specially if the extension deals with source code, which will be the usual thing in most cases:

Future Directions for Microsoft Visual Basic
http://channel9.msdn.com/pdc2008/TL12/

The Future of C#
http://channel9.msdn.com/pdc2008/TL16/

The matter of new syntax and language constructs is of most interest for many 3rd party tools. To put two examples, C# 4.0 will allow optional parameters and VB 10.0 will not require the line continuation character “_” to break lines in many cases.

There are two concerns regarding all this:

  1. Whether the automation model (EnvDTE*) will be updated to recognize it. It is not a secret that EnvDTE was not much enhanced since VS 2005 when EnvDTE80 brought new classes like EnvDTE.CodeClass2, EnvDTE80.CodeFunction2, etc.
  2. Four releases later, the automation model doesn’t support yet code inside methods, so you can’t get the method variables, constants and statements unless you parse the code on your own. While this is somewhat easier for VB.NET due to its verbose syntax (Dim, Const, End If, Next, etc.) it is painfull for C#.

In C# 3.0 and VB 9.0 parsing, type resolution and expression resolution got complicated with lambda expressions, extension methods, infering types from initializers, etc. but in the next versions of the languages with its “dynamic” approach this is going to be crazy. For example, VB 10.0 will support this syntax:

Sub Main()
...
   Dim thread As New Thread(Sub()
         For Each o In scores
            Console.WriteLine(o.ToString)
         Next
      End Sub
...
End Sub

That is, a Sub inside a Sub!. So, when you parse looking for variables, you will need also to take into account nested methods…  will the automation model provide something like EnvDTE100.CodeFunction3.CodeElements or EnvDTE100.CodeFunction3.CodeFunctions?

Not all are bad news; in fact, there are a couple of very good news:

  1. “Co-Evolution – there are no differences between the users of VB and C#. We are going to co-evolve the languages going forward. This means that if we introduce the features in one – then we will work hard to make that feature available in the other.”I wonder why MS took 6 years and four releases to realize this. When I heard some years ago that VB.NET developers where more concerned about productivity and C# developers were more concerned about “the code”, I thought it was a nonsense, as if C# developers wouldn’t like to be productive or VB.NET developers wouldn’t care about the elegance of code. The fact is that it is only a question of syntax: VB.NET is the natural option for people coming from a VB6/VBA background while C# is the natural choice for people coming from Java/C++ (let me say that I think that any good developer should know several languages). And from that point, both groups want the same features in the IDE and in the language(s). I have heard some people arguing that forcing both VB.NET / C# teams inside MS to provide the same features would hinder innovation, it takes time to catch on the other language, and that competition is good. For me, the question is why there are two whole huge separate teams to create two project subtypes that for the most part differ only in the syntax of the language (this is also true for the deceased Visual J#). I mean, they use similar project subtypes, with files or linked files, with imports/using, with compiled or source code references, with project property pages with similar properties, with configurations (Debug/Release) with similar properties, that compile to the same IL, etc. I guess this is for historical reasons, and maybe I am too naive, but it seems to me more natural to evolve towards a whole huge team that takes care of common things and allows extensions for VB.NET and C# (for example, the parsers), which would be developed by two small teams. My MZ-Tools add-in for Visual Studio supports VB.NET, C# and VJ #, it has tons of files and code, but it has a single file called MultiLanguageLibrary that handles the language differences. For example, imagine that Microsoft builds parsers that convert:C#
    int i = 5;
    f(i);
    

    or

    VB.NET

    Dim i As Integer = 5
    Call f(i)
    

    to something like this:

    <codeelement name = "i" kind="methodvariable" type="Int32" initialValue="5">
    <codeelement kind="methodcall" name="f">
       <arguments>
          <argument kind="methodvariable" name="i"></argument>
       </arguments>
    </codeelement>
    

    (just an idea)

    and then the remaining path is common for all languages. BTW, it would be nice to provide that parsing output to developers of extensions, and this is the second good news:

  2. Parsers/Compilers/Code Generators as services: Microsoft is going to “open up the box” so it is no longer a “black box” and will expose services to interact with its internal parsers, compilers, etc. so hopefully you don’t need to build your own parser and you don’t need to use the buggy and incomplete EnvDTE.Project.CodeModel / EnvDTE.ProjectItem.FileCodeModel. Don’t get too excited yet, since this will take years, though, and several Visual Studio releases.