Another critical VS 2010 Beta 2 bug in CommandBars: this one causes VS to evaporate without warning, nothing less

After 3 hours isolating this to the minimal expression (which is tedious if your add-in is has lots of user interface items and debugging a VS 2010 add-in on a virtual machine is painfully slow), here it is:

VS 2010 Beta 2 Bug: deleting a CommandbarPopup unloading add-in causes VS to disappear
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=503093

(the problem happens if the add-in creates two (not just one) CommandBarPopups on a parent CommandBarPopup and tries to delete them when unloaded)

Frustrating new bugs in commandbars of VS 2010 Beta 2

I can’t express how disappointed, frustrated (and more things!) I am with VS 2010 Beta 2 regarding correct behavior of commandbars for add-ins. It seems that once a bug is fixed in a VS 2010 build and an add-in can pass that point, new bugs surface that prevent its correct loading (and I have tested several builds since Beta 1).

Suffice to make you aware of these bugs:

VS 2010 Beta 2 Bug: CommandBarButton of add-in duplicated in new commandbar when already created in another commandbar
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=502289

(For my MZ-Tools add-in with tons of buttons, menus and popups, only that bug of VS 2010 Beta 2 has completely ruined it, not to mention the time it took me to isolate it until I realized what was happening because initially the add-in seemed to hang, due to the number of duplications that were occurring at the glacial speed of VS 2010 debugging an add-in on another VS 2010 instance on a virtual machine)

VS 2010 Beta 2 Bug: CommandBarButton.Style not honored, always Icon + Caption
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=500403

VS 2010 Beta 2 Bug: System.AccessViolationException attempting to create a temporary popup commandbar
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=500408

VS 2010 Beta2: InvalidCastException casting
DirectCast(CommandBarControl, CommandBarPopup).CommandBar.Parent back to CommandBarControl
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=499483

I realize that changing the COM-based commandbars of VS 2008 by new WPF-based commandbars in VS 2010 is a daunting and difficult task, but if the next milestone after Beta 2 is Release Candidate (where only critical bugs are fixed) then Beta 2 should be already free of such nasty bugs. Or there should be a known roadmap of betas / CTPs until RTM so that developers of add-ins can know how many builds they will have to test and rest assured that they will have enough weeks/months to test their add-ins (considering that in Beta 2 they can’t get them even loaded in Visual Studio).

I think that since Microsoft builds and extends Visual Studio with packages (SDK) and not add-ins, commandbars/commands/buttons of packages get better behavior earlier in the development cycle of Visual Studio. But for add-ins (automation) every time that commandbars are changed in some VS release, you have to wait until the RTM to check is some bug that could ruin your add-in is fixed. I experienced the same with VS 2005, which introduced 24-bit bitmaps (rather than the 16-color bitmaps of VS.NET 2003) and a different assembly for Microsoft.VisualStudio.Commandbars.

VS 2010 RC/RTM fixing ” ‘Object must be the same type as the enum’ exception calling CommandBars.Add from an add-in” problem

Microsoft has reported that this problem will be fixed:

 ” ‘Object must be the same type as the enum’ exception calling CommandBars.Add from an add-in”
http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=462766

Notice that the “Embed Interop Types” property seems to have been renamed to “Isolated” in VS 2010 Beta 2, in one of those renaming changes that make things more obscure…

VS 2010 Beta 2 certainly supporting RGB=0,254,0 as background color for command pictures as in previous versions of Visual Studio

FWIW, I have verified that VS 2010 Beta 2 certainly fixes the issue of supporting RGB=0,254,0 for transparent color in 24-bit command pictures that I mentioned here.

So, if your add-in targets VS 2005, 2008 and 2010 you can use that color for the transparent background color of command pictures and they won’t appear with lime green in Beta 2 as it happened in Beta 1 or previous CTPs builds.

If your add-in only targets VS 2010 it is preferable to use the new support for 32-bit command bitmaps. I guess that in a few years Microsoft will only support 32-bit bitmaps in Visual Studio.

VS 2010 and the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface

This is going to be messy, so step by step:

Imagine that you are developing an add-in that uses a temporary user interface (not permanent one), and creates a couple of commandbars of kind “toolbar”, and many commandbars of kind “popup” (with submenus).

Since it uses a temporary user interface, it must delete those commandbars when unloaded.

For that purpose, it must keep a reference to each created commandbar at class level (all this is explained in the article of the previous link).

Since it creates lots of commandbars, rather than keeping each one individually, they are stored in a class-level collection. Of which type?

CommandBars of kind “Toolbar” have the “Microsoft.VisualStudio.CommandBars.CommandBar” type but commandbars of kind “popup” have the Microsoft.VisualStudio.CommandBars.CommandBarPopup” type, so both can’t be stored in the same typed collection, but we would like to use a single collection, not two, or not a collection of System.Object.

Fortunately the Microsoft.VisualStudio.CommandBars.CommandBarPopup type has a CommandBar property, and since the Microsoft.VisualStudio.CommandBars.CommandBar type has a Parent property (typed as System.Object, that should return a Microsoft.VisualStudio.CommandBars.CommandBarPopup actually), you can use a generic typed collection List<Microsoft.VisualStudio.CommandBars.CommandBar>, which stores the CommandBar of toolbars, and the CommandBarPopup.CommandBar of popup commandbars.

When the add-in is unloaded, it iterates the commandbars of the collection to delete them:

  • If the CommandBar.Type is MsoBarType.msoBarTypeNormal, the commandbar is a toolbar and can be deleted calling the Delete method.
  • If the CommandBar.Type is MsoBarType.msoBarTypePopup, the commandbar is a popup so we get its Parent property (which is a System.Object) and we cast it to Microsoft.VisualStudio.CommandBars.CommandBarPopup. Then, we call the Delete method of CommandBarPopup.

(As I mentioned, my MZ-Tools add-ins seems to use every conceivable technique provided by the Microsoft.VisualStudio.CommandBars API)

However, this approach that worked perfectly in VS 2005/2008, doesn’t work in VS 2010. Why?

It happens that VS 2010 uses WPF-based commandbars, but add-ins for VS 2010 still use the Microsoft.VisualStudio.CommandBars Interop (ActiveX) assembly for backwards compatibility.

That interop assembly provides the Microsoft.VisualStudio.CommandBars.CommandBarPopup type, which is actually an interface (although it doesn’t follow the convention for interface names), not a class. The same happens with the CommandBar and CommandBarControl types, which are interfaces, not classes. The CommandBarButton type is a class, though. To complicate things, you have the CommandBars class and the _CommandBars interface… anyway:

In VS 2005/2008, which use native (COM) commandbars, the classes that actually implement those interfaces are native, and from a managed add-in you see them through a System.__ComObject which is the Runtime Callable Wrapper (RCW) that you get when you don’t have an interop assembly. But since they implement the interfaces provided by the Microsoft.VisualStudio.CommandBars assembly, the add-in works fine.

In VS 2010, which uses managed (.NET) WPF commandbars, the classes that actually implement those interfaces are managed and are provided by the Microsoft.VisualStudio.PlatformUI.Automation namespace in the managed (.NET) Microsoft.VisualStudio.Shell.UI.Internal.dll assembly (in one of the VS 2010 folders).

When you create a commandbar popup calling:

Microsoft.VisualStudio.CommandBars.CommandBarControl commandBarControl;
commandBarControl = parentCommandBar.Controls.Add(MsoControlType.msoControlPopup);

you get some class instance that implements the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface. So you can do this cast:

Microsoft.VisualStudio.CommandBars.CommandBarPopup commandBarPopup;
commandBarPopup = (Microsoft.VisualStudio.CommandBars.CommandBarPopup) commandBarControl;

In VS 2010, the actual class that implements that Microsoft.VisualStudio.CommandBars.CommandBarPopup interface is the Microsoft.VisualStudio.PlatformUI.Automation.CommandBarPopup._Marshaler class.

However, when you do this:

commandBarPopup.CommandBar.Parent

in VS 2005/2008 you get an object that implements the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface (as expected), so the approach described in the beginning of this post worked.

In VS 2010 you get an instance of the Microsoft.VisualStudio.PlatformUI.Automation.CommandBarPopup class, not of the Microsoft.VisualStudio.PlatformUI.Automation.CommandBarPopup._Marshaler class that you got when creating it.

It happens that the Microsoft.VisualStudio.PlatformUI.Automation.CommandBarPopup class doesn’t implement the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface. It implements IMarshaledObject<CommandBarPopup> instead. It also has a Marshaller property that would return the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface implementation.

So, the approach that worked with VS 2005/2008 doesn’t work with VS 2010.

Since you can’t cast or get the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface and an add-in targeting VS 2005/2008/2010 with the same dll can’t include a reference to the Microsoft.VisualStudio.Shell.UI.Internal.dll, keeping this approach the only way would be to call its Delete method would be through Reflection.

VS 2010 Beta 2 commandbar-related bugs with fixes on the way

If you are testing your add-in with the recently released VS 2010 Beta 2 chances are that you have noticed that the new WPF-based commandbars, buttons and icons are not yet ready for prime time. It seems that my MZ-Tools add-in uses every possible method, property and technique offered by the Microsoft.VisualStudio.CommandBars API so I am discovering lots of bugs (I have reported 15 so far) since I started testing with Beta 1. I reported these two ones before Beta 2 but the fixed will come in builds after Beta 2:

VS 2010 Beta 2 Bug: System.AccessViolationException attempting to create a temporary popup commandbar
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=500408

VS 2010 Beta 2 Bug: CommandBarButton.Style not honored, always Icon + Caption
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=500403

Debugging add-ins for VS 2010 (Beta 2) created with VS 2008: The debugger’s protocol is incompatible with the debuggee

If you are developing add-ins for in-house use, chances are that you can drive, encourage or even force the whole organization to use the latest version of Visual Studio and therefore your add-in needs to target only that version. But if you are developing add-ins for commercial or open-source purposes, to be used by lots of people, your add-in needs to target several IDE versions (nowadays, at the very least VS 2005, 2008 and 2010). To simplify your life as a developer, you try to keep a single code base and a single binary dll. For example:

or:

  • You create the add-in with VS 2008 (targeting .NET Framewor 2.0 / CLR 2.0) with a single project, with the same references than above, taking care of not referencing EnvDTE90 or VSLangProj90 (which are specific to VS 2008), generate a single DLL, but you create the proper .AddIn file and put in in the proper folder to target VS 2005 too.

Both approaches work. Some people use the first one to ensure 100% that the add-in will work with VS 2005 (since it was created with that IDE and .NET Framework 2.0) although I personally use the second one (I like VS 2008 much more than VS 2005) with no problems so far.

If for some reason the add-in needs to know the IDE version where is hosted or behave differently dynamically depending on the IDE version, it can guess it using DTE.Version or DTE.RegistryRoot, but there is no need to have separate DLLs if both VS 2005 and VS 2008 can use the same CLR 2.0, .NET Framework 2.0 and the same set of references.

Now, VS 2010 comes. So, your natural approach is to keep your existing code, binary dll and make it target VS 2010 adjusting the .AddIn file. Although VS 2010 uses a new CLR (4.0) instead of CLR 2.0, a CLR can load assemblies compiled against the previous version, in the same way that VS.NET 2003 (using CLR 1.1) could use add-ins created with VS.NET 2002 (using CLR 1.0) if the add-in was properly registered.

When you create your add-in with VS 2005 and you want to debug it on VS 2008, you just simply go to the Debug tab of the project properties and set the Start Action to start the devenv.exe of VS 2008, and this works perfectly.

Similarly, when you create your add-in with VS 2008 and you want to debug it on VS 2010, this worked perfectly until VS 2010 Beta 2. With the Beta 2 released two days ago you get the error:

The debugger’s protocol is incompatible with the debuggee

I have confirmed with Microsoft that this is not a bug, it is by design: VS 2008 (using CLR 2.0) or VS 2005 won’t be able to debug CLR 2.0 add-ins hosted in VS 2010 (which uses CLR 4.0). Which is a pity, but this means that your workarounds in the scenarios above are:

  • Use VS 2010 to create a separate binary add-in DLL (CLR 4.0) for VS 2010 only. For me, this is the worst workaround, only considered as a last resort.
  • Use VS 2005/2008 to create a single binary add-in DLL (CLR 2.0) to target VS 2005, 2008 and 2010, giving up debugging on VS 2010, resorting to MessageBoxes in your code to guess what’s going on if you need to debug a problem that can’t be reproduced when the add-in is hosted on VS 2005 or VS 2008.
  • Use VS 2010 to create a single binary add-in DLL (targeting CLR 2.0 /  .NET Framework 2.0, not CLR 4.0 / .NET Framework 4.0) to target VS 2005 / 2008 / 2010. I haven’t test this yet but I hope it works. Even if VS 2010 can’t debug a CLR 2.0 add-in hosted on another VS 2010 instance, you can temporarily switch it to target CLR 4.0 and this should work. Once the problem is debugged and solved, you can switch it back to CLR 2.0. I hope that VS 2010 can also debug a CLR 2.0 add-in hosted on VS 2005 or VS 2008.

Let me know if you have more insights about all this.

The strange case of VS 2010 (Beta 2 /CTPs) closing at startup or failing to create add-ins

As you know if you develop add-ins, Visual Studio 2005 introduced XML-based add-ins that use an .AddIn file that you must place in the proper folder for Visual Studio to show the add-in in the Add-In Manager. Which XML parser Visual Studio uses to parse .AddIn files wouldn’t be of interest if it wasn’t because for some people their Add-In Manager didn’t show XML-based add-ins… and the problem was solved installing MSXML. While I didn’t experience that problem personally, I have experienced a related problem with VS 2010 Beta2 and older CTPs.

The problem was that when I tried to create an add-in with the add-in wizard I got this misleading error message (misleading because I do have the proper language installed).

“An error occurred, and the wizard could not generate the project. Verify that the programming language is properly installed”

And more interestingly, if a place an .AddIn file (even an empty .AddIn file!) in one of the folders that VS scans for .AddIn files, VS 2010 fails to load with the following error message:

“The application cannot start”

and then it closes!

I remembered the problem of the empty Add-In Manager, I thought it could be related, so I verified with the Process Monitor tool that VS was certainly failing to get the MSXML2.Document.6.0 ProgId from the registry just before scanning folders for .AddIn files. I installed MSXML 6.0 and lo and behold, the problem was solved. 

So, I discussed this with several people inside the VS team and they confirmed that VS 2010 will use MSXML 6.0 to parse .AddIn files, etc. (I am not sure if VS 2005/2008 uses that version or older ones). It happens that I was using a virtual machine with Windows XP SP2, and it doesn’t include MSXML 6.0 (it includes MSXML 3.0 SP5), but VS 2010 Beta 2 / CTPs didn’t install it either. When I asked why VS 2010 didn’t install it as a prerequisite, the answer was that VS 2010 will require SP3 of Windows XP (which includes MSXML6), and the final release of VS 2010 will refuse to install on Windows XP SP2, something that the current betas / CTPs don’t enforce yet.

Now I hope that the Visual Studio Team modifies VS 2010 to show better behavior and diagnostics if, for whatever reason, Visual Studio doesn’t find the MSXML parser that it requires. We as developers tend to focus on code paths where things go as expected, and pay less attention to edge cases, but when things go bad, it causes a lot of pain to other people trying to diagnose and solve the problem. Something better than silent errors (empty Add-In Manager), crashes without further information or misleading error messages can be done…

CommandBarButton.Mask property deprecated in VS 2010

As you already know from my posts in the last months, VS 2010 will use WPF-based commandbars and will introduce support for 32-bit bitmaps with built-in transparency in the alpha channel for pictures of commands of add-ins, apart from the old way of using 24-bit bitmaps with RGB=0,254,0 for background color. While VS 2010 Beta 1 only supports the new way, I’ve been told that Beta 2 will support the old way for backwards compatibility.

Toolwindows in VS 2005 and higher have always supported either 32-bit bitmaps with built-in transparency in the alpha channel, or 24-bit bitmaps with RGB=255,0,255 as background color, and VS 2010 will support both too.

The third and last area problematic with custom pictures is when you want to set the picture of a CommandBarButton directly (not through a command). While some people use this approach to avoid satellite dlls, the only scenario where I personally use and would recommend this approach is to provide a context menu in some listview or treeview with menu entries that don’t have a command behind (because they are operations for the listview or treeview such as “add”, “edit”, “remove” items, not global operations). To make such context menu with a look & feel 100% consistent with Visual Studio I create a CommandBarPopup. I haven’t written yet an article to show this but the basic idea is to create a commandbar popup calling:

myCommandBarPopup = dte.CommandBars.Add(Name:="MyCommandBar", Position:=MsoBarPosition.msoBarPopup, Temporary:=True)

and then add CommandbarButtons calling:

myCommandBarButton = DirectCast(myCommandBarPopup.Controls.Add(...), CommandBarButton)

and then set the Caption, Picture and Mask properties:

myCommandBarButton.Caption = "..."

myCommandBarButton.Picture = ...

myCommandBarButton.Mask = ...

and then set-up an event handler for the Click event.

Notice that you have to provide a “picture” bitmap, which in VS 2005 and 2008 should be a 24-bit bitmap with any background color, and a “mask” bitmap which should use White=255,255,255 for the pixels that should be transparent color and Black=0,0,0 for the pixels that are actually colored in the “picture” bitmap. Since this has the inconvenience of being forced to provide two bitmaps per button, most people use a custom method to generate the mask image from the picture image.

This is not going to work in VS 2010, though. Since VS 2010 supports 32-bit bitmaps with built-in transparency in the alpha channel, Microsoft is making the CommandBarButton.Mask property deprecated (you get an exception if used). So, if your add-in for VS 2005 / 2008 uses the CommandBarButton.Mask property in one of the two scenarios that I have mentioned, you will need to revisit it and adjust it accordingly to make it work with VS 2010. Assuming that you want the same binary dll to work with VS 2005, 2008 and 2010, one approach is to design 32-bit bitmaps images with built-in transparency (for the VS 2010 version) and generate programmatically the 24-bit “picture” bitmap and the “mask” bitmap for the 2005 and 2008 versions. If your add-in only targets VS 2010, you no longer have to use the CommandBarButton.Picture approach to avoid the satellite dll since VS 2010 will get rid of satellite dlls for add-ins.

Microsoft fixing problem of grayscale disabled picture of add-in commands in VS 2010

The problem that always existed in Visual Studio with the generated grayscale picture for a disabled command of an add-in (or CommandBarButton without a command) is going to be fixed in Visual Studio 2010, according to the tests that I have done. That is, an add-in with a command that provides a 24-bit bitmap in a satellite dll with RGB=0,254,0 as background color (the old way compatible with VS 2005 / 2008) now shows a crisp and defined disabled picture if the command is disabled. Notice that providing a 32-bit bitmap with built-in transparency (the new way introduced in VS 2010) is not required to get this “fix”, it is something that Microsoft has fixed internally even for 24-bit bitmaps in the new WPF-based commandbars of VS 2010.

I have been unable to test if the same applies to CommandBarButtons whose picture is provided through the Picture property (without a command) but I think it will be fixed too.

VS SDK, packages, add-ins, macros and more…