New Visual Studio “15” Preview 3 Feature: Roaming Extension Manager

Just in case you missed it, the Preview 3 of Visual Studio “15” introduces the Roaming Extension Manager, that allows you “to keep track of all your favorite extensions across all of your development environments. Roaming your extensions keeps track of the extensions you have installed by creating a synchronized list in the cloud. “:

RoamingExtensionManager

You have some details and explanation of behavior/icons in the Release Notes.

dotnetConf 2016 Session: The Power of Roslyn: Improving Your Productivity with Live Code Analyzers

Dustin Campbell gave yet another session about the .NET Compiler Platform (“Roslyn”), this time at the dotnetConf 2016 and about live code analyzers:

“Two years ago we open sourced the C# and Visual Basic compilers and exposed their functionality through APIs as the Roslyn project. The Roslyn analyzer API empowers the developer community to be more productive by lighting up their own features in the editor in real-time—meaning anyone can write a refactoring, code fix, code style rule, or code diagnostic. Come learn more about the Roslyn project and what these live analyzers mean for you and your productivity.”

I have added this  session to the Videos section of this website, but here you have the link:

Component Diagnostics extension for VSX developers

There are tons of extensions on the Visual Studio Gallery, but most of them are for general developers using Visual Studio, or for some specific development technology (Web, etc.). However, there are some extensions that are specific to Visual Studio Extensibility (VSX) developers. The most known is the Extensibility Tools extension by Mads Kristensen, but there are also little gems from other Microsoft members.

I found a reference to this one by Paul Harrington that I didn’t know in a thread of the MSDN VSX forum:

Component Diagnostics
“Provides real-time diagnostics output of various sub-systems within Visual Studio. Access the toolwindow from the Help menu.”

ComponentDiagnostics

It supports VS 2012, 2013 and 2015. Despite the name, it’s an extension to diagnose Visual Studio subsystems such as:

  • Document Tab Well
  • File Change Service
  • Main Frame Data Model
  • Navigation History
  • OLE Component Manager
  • Package Manager
  • Running Document Table
  • Scrollbar Theming
  • Selection and UIContext
  • Solution
  • Task Scheduler Service

If your extension interacts with one of those subsystems, you can find it very useful. Furthermore, it is extensible so your own extension can provide diagnostics on that tool using the IVsDiagnosticsProvider interface.

Category “Addins” still present on Add Command dialog of VS 2015

Although VS 2015 removed support for add-ins and the Tools > Add-In Manager of past versions was removed, there is still at least one place where its past existence surfaces:

Click the Tools > Customize… menu to show the Customize dialog and click the Commands tab:

CustomizeDialog

Then click the Add Command… button and you will see the Addins item in the Categories list:

AddCommandDialog

I have reported this minor bug on Microsoft Connect.

Achieving .NET CLR independence of .NET-based add-ins for the VBA editor of Office

Being quite a perfectionist, there was one issue in older versions of my MZ-Tools 8.0 add-in for VBA that was not right for me: it required .NET Framework 2.0, which is installed by default on Windows 7, but not on Windows 8, Windows 8.1 or Windows 10, it’s an optional feature:

NetFrameworkOptionalFeature

So, installing MZ-Tools 8.0 for VBA on Windows 8.0 or higher likely you would get this dialog:

NetFramework20Required

And then the user should download and install .NET Framework 2.0. Needless to say, this is mostly inconvenient and can hurt sales, so I thought it would be nice if MZ-Tools 8.0 for VBA could use whatever higher .NET Framework was installed on Windows (such as .NET Framework 4.0 on Windows 8 or higher).

Before digging on how this can be done, some basics: a .NET Framework is composed of a Common Language Runtime (CLR), which contains the virtual machine for .NET, with the JIT compiler, garbage collector, etc. and a set of libraries. So:

.NET Framework = CLR + libraries

There are three kinds of .NET Frameworks:

  1. .NET Frameworks that provide their own, new, CLR. This has been the case for:
    • .NET Framework 1.0 (provides CLR 1.0)
    • .NET Framework 1.1 (provides CLR 1.1)
    • .NET Framework 2.0 (provides CLR 2.0)
    • .NET Framework 4.0 (provides CLR 4.0)
  2. .NET Frameworks that don’t provide a new CLR, they just provide new libraries. This has been the case for:
    • .NET Framework 3.0 (uses CLR 2.0 of .NET Framework 2.0)
    • .NET Framework 3.5 (uses CLR 2.0 of .NET Framework 2.0)
  3. .NET Frameworks that provide an update of an existing CLR (backwards compatible). This has been the case for:
    • .NET Framework 4.5, 4.5.1, 4.5.2, 4.6, etc. (all of them update the CLR 4.0).

A native process (such as an Office app) must load a CLR in order to load a managed (.NET) add-in. CLRs 1.0, 1.1 and 2.0 cannot coexist in the same process, that is, once of those CLRs is loaded, no other CLR version can be loaded in the same process. However, CLR 4.0 can coexist with CLR 2.0 in the same process.

Note: from this point, I will talk only about CLR 2.0 / CLR 4.0 (.NET Framework 2.0 or higher).

So, there are two aspects that an add-in for VBA must consider:

  • The lowest .NET Framework that it requires, which depends on the libraries used by the add-in. MZ-Tools 8.0 uses WinForms (not WPF), so it can work on .NET Framework 2.0 (it doesn’t require libraries from .NET Framework 3.x or 4.x).
  • The lowest CLR that it requires, which depends on the lowest .NET Framework that it requires. Since MZ-Tools 8.0 requires .NET Framework 2.0, the lowest CLR is 2.0.

Now the question is: given a native process that must load a .NET-based add-in, which CLR version does it use?.

An .exe process can have an .exe.config file that specifies the supported/required runtimes of the .NET Framework. Visual Studio uses such .config files but Office apps don’t, so they don’t care about CLR versions (they can use CLR 2.0, CLR 4.0, or even both at the same time). By no means an add-in should install an .exe.config file to modify the behavior of an Office application regarding supported/required CLRs, because it can break other add-ins.

However, there are still two approaches that an add-in written against .NET 2.0 (or .NET 3.x, which also uses CLR 2.0) can use to work with CLR 2.0 on systems that have it installed (such as Windows 7) and with CLR 4.0 on systems that don’t have CLR 2.0 installed but have CLR 4.0 installed (such as Windows 8, Windows 8.1 or Windows 10):

The first approach is to make the setup to detect which CLR versions are installed on the system:

  • If CLR 2.0 is detected, register the .NET assembly for COM Interop setting the Runtimeversion value of the InprocServer32 key to v2.0.50727, as it would be the usual since it’s an assembly compiled against .NET 2.0.
  • If CLR 2.0 is not detected and CLR 4.0 is detected, set the Runtimeversion value to  v4.0.30319. This forces the assembly to use .NET 4.0 even if it was compiled against .NET 2.0:

ComInteropRegistration

Note: to register the assembly for COM Interop, the setup cannot use regasm.exe, not even the version of .NET 4.0 (C:\Windows\Microsoft.NET\Framework\v4.0.30319\regasm.exe) because regasm.exe sets always the Runtimeversion value to the value of the CLR that the assembly was compiled against. Instead, the setup must create the registry entries for COM Interop directly (see the required entries here).

The second approach is to use a COM Shim, which I explained in Isolating .NET-based add-ins for the VBA editor with COM Shims. Since a COM Shim is a native C++ dll that loads the CLR to create a custom AppDomain and to load the assembly, it can decide which CLR version to load, depending on which CLRs are installed on the machine: if CLR 2.0 is installed, it loads it, and if CLR 2.0 is not installed but CLR 4.0 is installed, it loads CLR 4.0.

The strange case of MZ-Tools 8.0 crashing the VBA editor of Autodesk Inventor

Recently I got a bug report from a customer of my MZ-Tools 8.0 add-in for VBA stating that it crashed the VBA editor of Autodesk Inventor 2016. If he uninstalled MZ-Tools, the problem disappeared. This user has been using MZ-Tools 8.0 for months but now, suddenly, this problem appeared.

I downloaded the trial version of Autodesk Inventor 2017 and I was able to reproduce the problem. This was the first window of the crash:

AutodeskCrash

Clicking the View Report Details more info was shown:

AutodeskCrashDetails

So, it was an unhandled exception. The dmpuserinfo.xml contained the stack trace:

AutodeskCrashStackTrace

So, when MZ-Tools shows a modal window, Autodesk Inventor tries to set a “quick access toolbar gradient color” using a function that crashes the app. Initially I thought (and so it was reported) that the crash happened only when MZ-Tools was set to load on startup, but not when loaded manually using the Add-In Manager. However, I realized that it happened at any time that MZ-Tools tried to show a modal dialog.

What could be causing this after months of normal usage? My main suspect was the COM Shim that I introduced in build 8.0.0.77 (June 1) to provide proper isolation from other .NET-based add-ins for the VBA editor. And certainly the hypothesis was correct. When I ran MZ-Tools without the COM Shim, the problem didn’t happen. So, it seems that WinForms dialogs don’t play well when loaded from a different AppDomain. The several Office apps and versions don’t suffer this problem, but Autodesk Inventor does. Although I reported the problem to Autodesk, I guess that likely they won’t fix it, or anyway the existing released Autodesk Inventor 2016 and 2017 builds will persist with the problem. So, as a workaround, the new setup of MZ-Tools 8.0 for VBA will detect if Autodesk products are installed. In that case, the setup won’t register MZ-Tools using the COM Shim.

Isolating .NET-based add-ins for the VBA editor with COM Shims

Office 2010 introduced the 64-bit version, which supports only 64-bit COM add-ins. This meant that all add-ins created with VB6 for its VBA editor stopped to work, because they are 32-bit add-ins. The only possibilities to create 64-bit add-ins are using C++ 64-bit, or using .NET x64 and COM Interop. So, it is not a surprise that quite a few add-ins for the VBA editor are created now using .NET. I wrote the article  HOWTO: Create an add-in for the VBA editor (32-bit or 64-bit) of Office with Visual Studio .NET back in 2012 along with a few others referenced at the end of that article. One thing that many developers of add-ins for the VBA editor may not know is that by default all .NET-based add-ins are loaded in the same AppDomain (a .NET concept for isolation of assemblies), and therefore are not properly isolated. A couple of bad things that can happen are the following:

  1. “System.InvalidCastException: Unable to cast object of type ‘<Namespace of add-in 1>.VBProjectClass’ to type ‘<Namespace of add-in 2>._VBProject'”. This exception happens because two add-ins use different Interop Assemblies (IAs) for the “Microsoft Visual Basic for Applications Extensibility 5.3” COM reference. For example, my MZ-Tools add-in uses its own local Interop Assemblies, while others use their own Interop Assemblies or Primary Interop Assemblies (PIAs) provided by Microsoft. I have good reasons to use my own IAs, but I have tested that even two add-ins using PIAs can suffer the problem depending on the version of the PIAs or their locations. This problem happens because both add-ins are loaded in the same AppDomain, otherwise it wouldn’t happen.
  2. Since Office is a COM application, .NET-based add-ins use Runtime Callable Wrappers (RCWs). It is important that an add-in releases its count on RCWs when no longer used, calling Marshal.ReleaseComObject. For example, if a .NET add-in calls VBE.SelectedVBComponent.CodeModule.CodePane, it is leaking RCWs for VBComponent and CodeModule. How bad is this depends on the host, but suffice to say that for VB6 you can freeze the code editor. The article Office application does not quit after automation from Visual Studio .NET client explains the issue when automating Office but it applies also to using the extensibility model for VBA. The problem is that RCWs are created by AppDomain, so when an add-in needs a RCW, if another add-in in the same AppDomain already created one for the same COM object, the same RCW is reused by the second add-in. Apart from an add-in failing to decrease its usage count on a RCW, it can also happen than an add-in decreases usage counts not belonging to it, for example calling Marshal.FinalReleaseComObject.

So, how are both problems solved? The solution is well-known for developers of add-ins for Office applications using .NET since 2002 or 2003, through the use of a COM Shim. A COM Shim is an ActiveX dll, written in C++, that acts as a native add-in, and loads the Common Language Runtime (CLR) of .NET, creates an AppDomain, and loads the .NET-based add-in. Calls to the native C++ add-in (OnConnection, OnDisconnection, etc.) are passed to the managed (.NET) add-in.

If you are a developer of .NET-based add-ins for the VBA editor, the following are the MSDN articles that you need to read, understand and apply for your add-in to be a good citizen, running isolated from other add-ins in its own AppDomain. When reading the articles, replace “add-in for Office” or “Office extension” by “add-in for VBA editor”:

Isolating Office Extensions with the COM Shim Wizard
This article is relevant because it explains all about isolation and COM Shims.

Isolating Microsoft Office Extensions with the COM Shim Wizard Version 2.0 (VS 2005)
This article is relevant because it doesn’t use the Managed Aggregator introduced in next version 2.3. Add-ins don’t require “aggregation”, just “containment” is required.

Isolating Microsoft Office Extensions with the COM Shim Wizard Version 2.3 (VS 2008)
This article is relevant because it uses “aggregation” instead of “containment”, and you need to realize that add-ins don’t require “aggregation” and you can remove with some adjustments the ManagedAggregator project and its output assembly.

COM Shim Wizards for VS 2010 (2.3.2.0)
This article is relevant because it’s the latest available COM Shim wizard with bug fixes, yet using “aggregation”. I used this and then removed “aggregation” to use “containment” of version 2.0.

Taking COM Shim Wizards to 64-bit
This article is relevant because it explains how to port the COM Shim to 64-bit.

There are two ways of knowing which AppDomain an add-in is loaded into:

Using Process Explorer, you can use the .NET Assemblies tab of the properties window of a process. For example, this image shows that MZ-Tools is loaded in its own MZToolsAppDomain using CLR 4.0:

AppDomainCLR40

For some reason that I don’t know, Process Explorer doesn’t show AppDomains for CLR 2.0:

AppDomainCLR20

The other way is calling from the own add-in System.AppDomain.CurrentDomain and showing its FriendlyName property. This image shows the MZ-Tools uses its own MZToolsAppDomain for CLR 2.0 (that Process Explorer didn’t show):

IsolatedAppDomain

If not using the COM Shim, the AppDomain is the DefaultDomain:

DefaultAppDomain

The four versions of a Visual Studio package

One of the confusing aspects when you create a new Visual Studio package is that there seems to be many places where a version number appears. They are the following:

  1. The version on the Tools > Extensions and Updates dialog:

VersionExtensionsAndUpdates

2. The version on the Help > About dialog:

VersionHelpAbout

3. The version on the deployment folder:

VersionFolder

4. The version of the file:

VersionFile

5. The version of the assembly:

VersionAssembly

All those five versions are set in four places, that are the following:

1. The version on the Tools > Extensions and Updates dialog and the version on the deployment folder are the same version and it is set in the source.extension.vsixmanifest file, Version field. It is recommended (in my opinion) to set it to the file version (4 numbers):

VersionManifest

2. The version on the Help > About dialog is set in the package class, InstalledProductRegistration attribute. It is recommended to set the same value than in the previous point, so that the Help > About and the Tools > Extensions and Updates dialogs show the same value:

VersionPackageClass

3. The version of the file is set in the AssemblyInfo.cs file, AssemblyFileVersion attribute. This file version is a Win32 concept (not a .NET concept) and it is best practice to change it on each build:

FileVersionAssemblyInfo

4. The version of the assembly is set in the AssemblyInfo.cs file, AssemblyVersion attribute. This assembly version is a .NET concept and it is not required to change it on each build:

AssemblyVersionAssemblyInfo

Note: to learn the difference between Assembly File Version and Assembly version read the Knowledge Base Community article How to use Assembly Version and Assembly File Version.

The strange case of MZ-Tools 8.0 for VBA could not be loaded

Yesterday I got this error when loading my add-in MZ-Tools 8.0 for VBA:

MZTools8VBACouldNotBeLoaded

I hate that error. The VBA and VB6 IDEs hide valuable information about why an add-in couldn’t be loaded. In Visual Studio you get at least a COM error number (the error message is almost always <Unknown error> for COM errors):

MZTools8VSCouldNotBeLoaded

And you have this article of mine explaining most causes given an error number: HOWTO: Troubleshooting Visual Studio and Office add-ins.

But for VB6/VBA, you are clueless. I have got that error many times because my MZ-Tools add-in is supported in several IDEs using different approaches for registration for the end user (using a setup) and for myself during development/debugging (using PowerShell scripts for registration/unregistration).

For the end user (using a setup):

  • MZ-Tools 8.0 for VBA uses per-user COM-registration, not requiring admin rights to install, which is important for many Office users in corporate environments. Per-user COM registration works because Office doesn’t run with admin rights by default (an executable running elevated can not see per-user COM-registration, for security reasons).
  • MZ-Tools 8.0 for VB6/VB5 uses machine-wide COM-registration, requiring admin rights to install, because per-user COM-registration wouldn’t work since VB6 runs elevated by default (to be able to register ActiveX Dlls when compiling).

For development/debugging (using PowerShell scripts):

  • MZ-Tools 8.0 for VB6/VB5 uses machine-wide COM-registration, because of the reason explained above. That means that I must run Visual Studio always elevated, so it can launch vb6.exe for debugging without annoying prompts for elevation of Visual Studio with a restart.
  • MZ-Tools 8.0 for VBA uses also machine-wide COM-registration, because since I run Visual Studio always elevated, Office runs always elevated during debugging, and therefore per-user COM-registration wouldn’t work (remember, a process running elevated cannot see per-user COM-registration).

All this means that any error using the wrong registration or running Visual Studio, VB6 or Office with the wrong elevation (or lack of), and I get the dreaded “MZ-Tools 8.0 could not be loaded”.

In those cases, I need to use Process Monitor (see HOWTO: Using the Process Monitor (ProcMon) tool to diagnose Visual Studio add-ins problems) to try to guess which registry entry or file is causing a problem. In this case, I got this:

PathNotFound

So, the dll of the add-in was not found. But, wait a minute: it is looking for it in the path “C:\Users\<user>\AppData\Local\MZTools Software\MZTools8VBA\”, which is used for the end user (setup), not in the path used during development. I went to the Control Panel, Program and Features, and MZ-Tools 8.0 was not installed!. So, somehow the last uninstallation of the setup failed silently, removing the file but leaving the COM-registration in the Windows registry. How could that be? I test the setups and PowerShell scripts exhaustively when created/modified and they were working correctly for weeks since the last changes.

Then, suddenly, I remembered something. I am using Windows 10, and I always use the “Control Panel”, “Program and Features” item to uninstall the software, because I like colors and I am not a fan of the new bland “Settings” > “System” > “Apps and features” user interface of Windows 10. But the day before, I used the “Apps and features” to uninstall MZ-Tools 8.0 for VBA, maybe because I guess that the Control Panel will be removed at some point since it seems a remaining duplicated thing. So, after some manual cleaning of registry entries, I tried again to install and uninstall using the setup (using “Settings” > “System” > “Apps and features”) and then to register for development using the PowerShell script and the problem was reproduced. One try again and I realized the cause:

  • If you have a setup that doesn’t require admin rights to install, the  “Control Panel”, “Program and Features” doesn’t require admin rights to uninstall either, which is correct.
  • However, the “Settings” > “System” > “Apps and features” prompts for admin rights to uninstall even if the setup was installed without admin rights, which seems incorrect to me.

It happens that some users of MZ-Tools 8.0 for VBA run Office with admin rights (I don’t know really why they want that), and therefore if MZ-Tools 8.0 for VBA runs with per-user COM-registration, it doesn’t work. So I changed the setup of MZ-Tools 8.0 for VBA some months ago to behave as follows:

  • By default, the setup runs without admin rights. In that case, the setup registers the add-in using per-user COM-registration.
  • If you run Office with admin rights, you can force the setup to use machine-wide COM-registration (so it works) running it with admin rights (using the “Run as administrator” context menu). The setup detects that it is running with admin rights and uses machine-wide COM-registration instead of the default per-user COM-registration. Conversely, the uninstaller detects if it is running with admin rights and in that case removes machine-wide COM-registration; otherwise, it removes per-user COM-registration.

And then it happens that on Windows 10, if you use the “Settings” > “System” > “Apps and features” to uninstall, the uninstaller is elevated to use admin rights (even if it was installed without admin rights) and it removes the non-existent machine-wide COM-registration, leaving intact the existing per-user COM-registration that was created when installed without admin rights. When the PowerShell script registers the add-in using machine-wide COM-registration, two COM-registrations are in place. If I run Office with admin rights, the machine-wide COM-registration is used and it works. But if I run Office without admin rights, the per-user COM-registration that shouldn’t be there takes precedence, and points to a file that was uninstalled, so the error.

Now I have to modify the uninstaller to remove also the per-user COM-registration even if running with admin rights.