VS Extensibility

Visual Studio extensions and DPI-awareness

2 Comments

This first thing that I noticed (very horrified) when I installed Visual Studio and my MZ-Tools extension in the new MacBook Pro 13″ Retina that I purchased last month was that at 100% scaling (96 Dpi) everything looked very tiny and at 150% (144 Dpi) or 200% (192 Dpi) scaling MZ-Tools looked horrible with very wrong layouts. In some Visual Studio versions (such as VS 2008) it looked better. And since MZ-Tools also works with the VBA editor and VB6/VB5 I also tested with different results. At that point I became DPI-aware and read everything I found about Windows and DPI, which is a lot because it is quite complex and tricky. So, if you are not DPI-aware yet, I urge you to buy a retina external monitor or a retina laptop  so that you can set at least 200% scaling (better than 150% scaling) to do some tests and fixes and avoid a painful experience. A “large” monitor such as the one of the iMac 27″, that I also own, doesn’t qualify if it is not the new retina model. In the future, especially with Visual Studio, two screens would be needed for better tests with “per-monitor DPI” awareness introduced by Windows 8.1 once Visual Studio supports it.

First, I will introduce some resources that explain better than me DPI-awareness and Windows OS support:

To summarize:

  • Windows XP used a pseudo-DPI scaling (fonts and some other UI items)
  • Windows Vista (and Windows 7 and Windows 8) introduced true DPI-scaling with the following notes:
    • The same DPI is used in all the monitors of a computer (this is called “system DPI”).
    • An application must declare through a manifest (embedded in the executable or in an .exe.manifest file, or even by code) its DPI-awareness.
    • If an application lacks the manifest, or states that it is not DPI-aware, then the OS scales the application as a bitmap (something called “DPI virtualization”) while the application still thinking that is running at 100% scaling (96 dpi).
  • Windows 8.1 introduced “per-monitor DPI”, where each monitor can have a different DPI and an application is notified of a change in the DPI when moved to another monitor, so that it can accommodate the new display.

Given that your extension is hosted in an application (Visual Studio), you don’t have the chance of declaring anything about the DPI awareness of your extension, you are conditioned by the DPI-awareness of the Visual Studio version that is hosting your extension.

To know the DPI-awareness of an executable (such as devenv.exe of Visual Studio) you can use these approaches:

  • At design-time, you can see if the .exe file is accompanied by an .exe.manifest file (this is the approach used by Office applications):

DPIAwareInArchiveManifest

ArchiveManifest

Or you can open the .exe file with Visual Studio (“File” > “Open” > “File…” menu) and search for a “RT_MANIFEST” resource (this is the approach used by Visual Studio):

DPIAwareInEmbeddedManifest

  • At run-time, you can use Process Explorer and configure it to show the “DPI Awareness” column:

DPIAwarenessColumn

The results are as follows on Windows 8.1:

DevenvAwareness

  • Visual Studio 2005 and 2008 don’t state DPI-awareness in any manifest (embedded or otherwise) but sometimes are shown in Process Explorer as “Per-monitor aware”, for some reason that I haven’t found yet. Certainly they are not “Per-monitor DPI-aware” (not even “System DPI-aware”).
  • Visual Studio 2010 and higher state “System DPI-awareness” through an embedded manifest (but not per-monitor DPI awareness on Windows 8.1).

The bottom line is that since Visual Studio 2010 your extension must be at least “System DPI-aware” and even earlier for that issue with VS 2005/VS2008. Microsoft enhanced the DPI handling in VS 2013, and even more with high-resolution icons in VS 2015.

When ensuring DPI-awareness, you need to address three aspects:

  1. Once the form or toolwindow is scaled, the layout of controls must be correct, that is, without overlapping controls or clipping out of boundaries.
  2. The size of controls that show images must be scaled according to the actual DPI. Visual Studio does this with the toolbars and menus, but you must do it for the rest of your user interface. This includes graphic buttons on your own toolbars, pictureboxes, images on treeviews, listviews, etc. For example, what at 100% scaling measures 16×16 pixels must measure 32×32 pixels at 200% scaling. At this point we are talking only about resizing the container stretching the contained image. This point and the previous one guarantee that your extension doesn’t have usability issues with clickable controls at high scaling (such as tiny toolbar buttons or checkboxes on treeviews/listviews)
  3. Ideally, you should provide high-resolution images for scaled containers of images of the previous step. That is, if a button holds a 32×32 image at 200% scaling, you should provide a 32×32 image and not a stretched 16×16 image. If you are a perfectionist and have a lot of time you should provide the following dimensions for the images:
    • 16×16 pixels at 100% scaling (96 Dpi)
    • 20×20 pixels at 125% scaling (120 Dpi)
    • 24×24 pixels at 150% scaling (144 Dpi)
    • 32×32 pixels at 200% scaling (192 Dpi)
    • 40×40 pixels at 250% scaling (240 Dpi)
    • etc.
      In practice, you can compromise providing just 16×16 and 32×32 resolutions and stretch images for resolutions that you don’t provide.

Now, how to provide DPI-awareness in your extension depends on the technology that you are using: it is said that it is easier with Windows Presentation Foundation (WPF) but if you created your extension back in VS 2005 chances are that you are still using Windows Forms. The good news is that you can achieve excellent DPI-awareness with Windows Forms, even with .NET Framework 2.0. I say this because in newer versions of .NET Framework (since 4.5) a Windows Forms app can opt-in for better DPI handling.

I have found several issues in the last weeks that I have been working on this and I have resolved all of them to my complete satisfaction:

  • First of all, ensure that all the designers of your forms and usercontrols include the following two lines. This is especially important if your extension has evolved from an add-in created back in VS.NET 2002/2003 because .NET Framework 1.0 / 1.1 used an inferior scaling mechanism with other deprecated properties:
this.AutoScaleDimensions = new System.Drawing.SizeF(6F, 13F);
this.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font;
  • You may need to scale some controls at run-time, depending on how they are added to the user interface.
  • You may need to adjust at run-time some widths and heights, especially if you are using the SplitContainer control.
  • You need to scale hard-coded dimensions in pixels that you may be using in your code for performing layouts at run time.
  • You need to scale pictureboxes according to the actual DPI.
  • You need to provide 32×32 size imagelists (either with 16×16 scaled images or, preferably, 32×32 native images)
  • You need to adjust the ItemHeight property of treeviews.
  • You need to adjust the dimensions of toolstrip buttons with the ToolStrip.ImageScalingSize property.
  • Avoid the use of the StateImageList property of the TreeView control, because in .NET Framework 4.0 or lower it’s always 16×16, and in .NET Framework 4.5 or higher the application can opt-in for a DPI-aware StateImageList, but your extension cannot. Instead you can use the Win32 API for TreeView controls to set the state imagelist and image index for nodes, as it was the case in .NET Framework 1.0 / 1.1, whose TreeView control lacked the StateImageList property.
  • You need to adjust the width of the columns of the listviews because they are not adjusted automatically in high DPI scaling. I had already coded this (because I hate columns that don’t adjust automatically to the contents).
  • The glyph of Radiobutton and Checkbox controls is scaled automatically if visual styles are enabled, as it happens with Visual Studio. For some reason, in the VBA editor of Office they are not enabled so a call to the System.Windows.Forms.Application.EnableVisualStyles method is required.
MZ-Tools Articles Series, VS Packages

MZ-Tools Articles Series: HOWTO: Force files to open with a specific editor from a Visual Studio package

This other article also comes from one of my answers in the MSDN VSX forum:

HOWTO: Force files to open with a specific editor from a Visual Studio package

It explains how to overcome a limitation of the automation model (EnvDTE) forcing a ProjectItem to open in a specific editor even when the user has set the default to other editor (especially problematic if the editor is external, such as Notepad). It is also useful if you want to start getting rid of “EnvDTE” code in your package.

MZ-Tools Articles Series

MZ-Tools Articles Series: HOWTO: Create two Visual Studio packages in a single assembly

The first MZ-Tools Articles Series that I have written this new year is this:

HOWTO: Create two Visual Studio packages in a single assembly

I did it as a pure exercise after a question in the MSDN VSX forum and I don’t think it has any practical use even for the problem of a package targeting multiple versions of Visual Studio with the maximum reuse of code at source and binary levels.

Currently the most notorious problem that I have found (and that I reported at Uservoice) is that you can’t provide a single .vsix package for VS 2010/2012/2013 with a single binary assembly and two .vsct tables with the following conditions:

  • The package uses one .vsct command table for VS 2010 (with colorful icons)
  • The package uses another .vsct command table for VS 2012/2013 (with gray icons)

So, you can create a single package in a single assembly with two .vsct tables, or two packages in a single assembly with a different .vsct file each one (like in the example provided in the article), but neither you can specify in the .vsct table or package attributes the target VS version, nor you can specify in the manifest of the .vsix file the target VS version of each .pkgdef file.

VS 2015, VS Extensibility

Code model of EnvDTE using Roslyn internally in VS 2015. Bugs ahead.

3 Comments

One of the things that I have done in the last weeks has been to run the integration tests that I have for my MZ-Tools extension (now converted to a package) in Visual Studio 2015 Preview. I have hundreds of them and pass successfully in VS 2005 / 2008 / 2010 / 2012 and 2013 (with some sporadic timing-related failures that I try to reproduce and fix forever, but that’s another subject). When I ran them on VS 2015 Preview, dozens and dozens of them failed, to my desolation, because I immediately knew that there was something very wrong in VS 2015. One after another, I clasified them, identified patterns and all of them were related to the code model (EnvDTE.ProjectItem.FileCodeModel, EnvDTE.CodeElement, etc.) which my add-in uses intensively in several features. Eventually I isolated the root causes and I identified several bugs in the implementation of the automation code model in VS 2015, which soon I learned was changed internally to use the .NET compiler platform (“Roslyn”). I even discovered and disassembled the new assemblies that provide the implementation because one of the bugs is so severe that it crashes Visual Studio:

FatalExecutionEngineError / InvalidCastException after sorting body-less properties in VB.NET file

Although I can reproduce it 100%, I have not isolated it to the minimal package because I think that with the information provided to Microsoft should be enough. I am still awaiting the acknowledgement of the problem.

Other ones, that are far less severe, don’t crash Visual Studio but cause exceptions, and would need some fix:

ComException getting start/end points of EnvDTE80.CodeEvent Adder/Remover/Thrower

EnvDTE.CodeParameter.StartPoint and EndPoint causes exception for “ByVal ParamArray” parameters

EnvDTE.CodeFunction.FunctionKind causes exception for methods of classes with explicit interface implementation

And yet another ones don’t cause exceptions, but return an incorrect value when in VS 2013 returned the correct values:

EnvDTE.CodeElement.FullName doesn’t return full name for constants, fields, methods, properties and events in VB.NET

EnvDTE.CodeClass.Members collection doesn’t include code elements for “Declare” methods of VB.NET class

EnvDTE.CodeFunction.FunctionKind doesn’t return vsCMFunction.vsCMFunctionDestructor for Protected Overrides Sub Finalize()

EnvDTE.CodeFunction.FunctionKind doesn’t return correct value for EnvDTE80.CodeEvent.Adder, Remover or Thrower

EnvDTE.CodeDelegate.Attributes and EnvDTE80.CodeEvent.Attributes returns 0 elements always

This change in the implementation of the automation code model to use “Roslyn” is quite surprising to me. I reported lots of bugs in the automation code model in the last years until I got tired of getting the same answer: no fix would be provided because a new API was being developed. Later I learned that the new API was Roslyn but I didn’t expect the automation code model bugs to be fixed by leveraging Roslyn. AFAIK, Microsoft hasn’t provide any information about this change, but the truth is that bugs have been introduced. I have implemented workarounds in my code for some of them until Microsoft fixes them (in VS 2015 CTP5 they are not fixed). There are also a couple of them (very minor) that I haven’t reported yet. So, if your extension uses the automation code model, I encourage you to test that area carefully and report the bugs that you find.

VS 2015, VS Extensibility

Past, present and future challenges for developers of Visual Studio extensions

It has been a while since my last post. In this time I have spent four weeks of vacations (although working a lot on my MZ-Tools extension), I have got a new laptop (MacBook Pro 13″ Retina), a new year has come and I have been awarded Microsoft MVP again (my 12th award). Most of the time working on my extension has been in two painful tasks, which reminds me how challenging is to develop extensions for Visual Studio, specially when Microsoft releases new major versions. This is the history of difficulties that I recall since Visual Studio (.NET) was born:

  • In Visual Studio .NET 2002/2003 you had to learn a new extensibility model (EnvDTE), different from the one used by Visual Basic 6.0. Also, creating toolwindows was extremely tricky because you needed to create a shim ActiveX control in C++. A native satellite dll for custom bitmaps was required.
  • In Visual Studio 2005 you had additional automation assemblies (EnvDTE80), you had to convert all the icons from 16 colors to 256 colors and you could create COM-free add-ins (with the .AddIn file). A satellite dll for custom bitmaps was still required, but no longer native.
  • In Visual Studio 2008 there was no much change, apart from the tricky magical colors to get transparency in bitmaps of buttons and toolwindow icons.
  • Visual Studio 2010 was the most stressful version ever released. The introduction of WPF-based commandbars (instead of the previous Office-based commandbars) caused tons of bugs in add-ins that I reported to Microsoft. A satellite dll for custom bitmaps was no longer required. You could use WPF for your dialogs, but chances were that you kept using Windows Forms. In the packages area, new APIs and deployment mechanisms were introduced.
  • In Visual Studio 2012 macros were removed, a new icon style was introduced (which forced your extension to follow suit) and a new dark theme was introduced, which also forced you to theme your toolwindows and dialogs if you wanted to follow suit. Applying a dark theme to Windows Forms forms is hard.
  • In Visual Studio 2013 there was no much change.

At the present time, Visual Studio 2015 introduces these new challenges:

  • Add-ins are removed (the Add-In Manager is gone), which means that a migration to a package is required.
  • The code model (EnvDTE.Project.CodeModel and EnvDTE.ProjectItem.FileCodeModel) are reimplemented internally to use the .NET compiler platform (“Roslyn”). So, expect bugs, small bugs with possible workarounds and serious ones as to crash Visual Studio. I have reported several of them to Microsoft Connect in the last weeks (one of the two painful tasks that I mentioned). This will be the subject of my next post.

Also, I have mentioned that in the past weeks I have adquired a new laptop. I had a MacBook Air 11″ which has served me well in the last 2.5 years but it had its limits (128 GB SSD, 4 GB memory and 1280 pixels of resolution), and I wanted something more powerful. So, I got a MacBook Retina 13″ with 16 GB of memory and 256 GB SSD. The first thing that I noticed is that at native resolution and 100% DPI icons and texts are tiny, so you have to increase the DPI level to 150%. And at that level my MZ-Tools extension behaves badly. So I have spend a lot of time in the last weeks (the other painful task) making it fully DPI-aware, which means 1) correcting layouts, 2) scaling low resolution images to the correct size, and 3) providing high resolution images (for VS 2010 icon style and for VS 2012/2013/2015 icon style in light/dark themes). Applying high-DPI awareness to Windows Forms is tricky. This will be the subject of another post, but high-DPI awareness is another area that developers of Visual Studio extensions will need to learn and implement because retina displays will become mainstream for laptops this year, and 4K monitors for computers are also starting to appear. And you don’t want to contribute to the painful lifestyle of high-DPI desktops.

In the future at the very least I still think that the automation model (EnvDTE) used even by packages is at risk, despite Microsoft updating it in VS 2015 using Roslyn internally to provide more accurate results in the code model with the current API. The EnvDTE automation model was created in an era (Visual Studio .NET 2002) when add-ins and macros were the only form of Visual Studio extensibility (you couldn’t create packages until Visual Studio .NET 2003 and the VS SDK was not introduced until Visual Studio 2005). Ten years later, macros were removed in Visual Studio 2012 and add-ins will be removed in Visual Studio 2015. The VS SDK provides interfaces for most EnvDTE stuff, except the code model (that will be provided by Roslyn from now on) and another small areas (such as to create and edit solution configurations). I think that at some point Microsoft will consider another step in simplifying APIs removing EnvDTE. And getting rid of awful APIs and interfaces in the VS SDK when creating packages is long overdue, so expect new challenges.

VS 2013, VS 2015, VS Extensibility

Great news for developers of Visual Studio extensions

Unless you have been out of the (programming) world the last couple of days, you already now that Microsoft has made the biggest announcements in years regarding the .NET Framework and Visual Studio. Concerning developers of Visual Studio extensions, four of the announcements will have a big impact:

These changes respond to the strategy of Microsoft for the next years (mobile-first, cloud-first), which pivots around three pillars:

  • Windows: on desktop (specially enterprise desktops, less likely to move to non-desktop or to non-Windows desktops such as Linux or Mac OS X), on tablets (Surface or otherwise) and on smartphones.
  • Office 365: on every possible device, Windows-based or not (web-based, iOS, Android, Mac OS X, etc.).
  • Windows Azure: cloud computing for client apps that run everywhere.

And for that strategy to succeed, Microsoft needs, you know, developers, developers, developers, like Windows on desktop many years ago.

So, let’s examine how each of the announcements above will impact you as a developer of Visual Studio extensions today and in the distant future:

  • The new Visual Studio 2013 named “Community Edition” with full extensibility is already available. Microsoft could have waited until Visual Studio 2015 next year to provide it, but it has preferred to provide it now. The Community Edition is basically the Professional Edition and as such it supports any kind of extension: packages, MEF, templates, and still add-ins without any need to update extensions. It replaces the former Express editions, which lacked extensibility at least for non-Microsoft extensions, despite being very requested. This yields to a bit bizarre VS 2013 offering that previously included paid “Professional”, “Premium”, “Test Professional” and “Ultimate” editions, free “Express” editions for Web, Windows (Phone and Store apps) and Windows Desktop and now the “Community” edition. I guess this offering will be cleared with Visual Studio 2015, hopefully with just two editions “Community” and “Enterprise”, given that the new VS 2013 Community Edition is free for up to 5 users for commercial/non-commercial development but it is not valid for enterprise use. So, regarding this announcement, you just need to test that your extension works with the VS 2013 Community Edition (as expected).
  • The new “Preview”, rather than “CTP” (Community Technology Preview”), of Visual Studio “14” has now the official and expected name “Visual Studio 2015” (likely you will need to refer to this name in your documentation, web site and maybe setup and even in the extension itself). If you are a developer of packages, templates or MEF extensions this new version will not have as much impact (except some details such as high-resolution icons) as if you are still a developer of add-ins, which in Visual Studio 2013 were deprecated but present, and in Visual Studio 2015 they are gone (no “Tools” > “Add-In Manager” menu). So, you have to migrate your add-in to a package, which can be quite a daunting task depending on the size and complexity of your add-in (only alleviated by the fact that you can still use the automation model (EnvDTE) from a package.
  • The .NET Framework being now open source will benefit any .NET developer, not just developers of Visual Studio extensions. The most important thing here is the hope that some day Visual Studio may be open source too. To have access to the source code (and even with debugging) of the host where an extension is loaded is a huge time-saver when problems arise (and they do).
  • The .NET Framework being now cross-platform could have a huge impact for Visual Studio in the future, because there will be an increasing demand for Visual Studio to be cross-platform too (Linux, Mac OS X). There is already an increasing demand for Visual Studio to be a portable app (runnable from a thumb-drive without installation). Both things are impossible in the current incarnation of Visual Studio, which is COM-based, requires admin rights to install (although not to run) and it’s basically a monster of technologies (COM/.Net, Windows Forms/WPF, etc.), and languages (C++, C#) with a mess of awful APIs for extensibility and so on. All this has an impact on performance, starting with the installation of the product (which currently takes longer than installing the Windows OS). My guess is that at some point Microsoft is going to need to start from scratch with a new IDE which is “performant”, modular, portable, cross-platform, designed to be beautifully extensible, etc.  Such endeavor may take years, but that’s not a deterrent for Microsoft: creating the .NET platform took years, as well as creating the new .NET-based compilers for C# and VB.NET, and Office is today multi-platform (Windows, Mac OS X, iOS) although not with the same binaries. Of course, it won’t be backwards compatible and extensions will need to be rewritten, so maybe this new IDE will have a different name and could coexist with Visual Studio for some time. Here you have an strategy that you can start adopting to mitigate this scenario.
VS Extensibility

Cancelling the wizard of project template creation

Another question about cancelling the IWizard when creating a project, this time in the StackOverflow forum that I am visiting now quite often, has prompted me to create a project template with a wizard using the IWizard interface, as explained in the MSDN documentation How to: Use Wizards with Project Templates. The sample certainly doesn’t show how to cancel the wizard and the IWizard interface methods don’t seem to offer that possibility so apparently it is not possible. However, it is possible in a non-intuitive way: there are two exceptions, WizardCancelledException and WizardBackoutException that the wizard can throw to return to before showing the Add Project dialog, or to that dialog, respectively. There seems to be some cleanup to do, as Rory Primrose explains in his post Pitfalls of cancelling a VSIX project template in an IWizard. You may also find useful a long explanation by Ron Petrusha about Developing Visual Studio Project Wizards, which I will add to the Articles section about templates of this site. And remember that if you want full control of the wizard you can always use the old COM-based IDTWizard interface instead.

VS Extensibility

Added StackOverflow reference to the Forums section

While I was aware of the StackOverflow Q&A site all these years, my preference has been always the specific MSDN Visual Studio Integrate forum (formerly MSDN Visual Studio Extensibility forum), and I have only answered a couple of questions in StackOverflow, mainly pointed by someone by e-mail.

However, today I have registered in StackOverflow, added my favorited tags (vsx, vs-extensibility, vsix, vspackage, visual-studio-addins, envdte, visual-studio-sdk, etc.) and I hope to get used to that site and visit it as often as I do with the MSDN VSX forum.

I have also added it to the Forums section of this VSX site since I find it a valuable resource to get answers: https://www.visualstudioextensibility.com/forums/

 

VS Templates

Project templates wizards (IWizard vs IDTWizard)

A question in the MSDN VSX forum yesterday about fully customizing the solution creation (not the project creation) with a template made me to investigate to provide a long answer and I discovered something that I had totally forgotten: the old COM-based EnvDTE.IDTWizard that appeared in the first versions of Visual Studio .NET (2002/2003) and that can be used still today. That interface provides a single Execute method that allows you full control over the solution/project creation, on the contrary to the new wizard Microsoft.VisualStudio.TemplateWizard.IWizard that provides many methods to server as hooks in the project creation. Unfortunately I have had these two days very busy at the office and personal life (and the forecast for the next days doesn’t seem much better) so I haven’t had time to write articles with code about these wizards. Interestingly I have found that I wrote a long article with code for a Spanish magazine 8 years ago about the IDTWizard. Anyway, what I have done is to update the Visual Studio Extensibility section and Templates section to include both kinds of wizards.

Add-Ins, MZ-Tools Articles Series

MZ-Tools Articles Series: PRB: Window.Close doesn’t save changes if requested for .sql files in non-database projects

My new article is to document a bug that a customer of my MZ-Tools add-in reported some days ago, and that is caused by a bug (or problem) in the automation model (EnvDTE). It happens in the following scenario:

  • Using the latest versions of Visual Studio (it doesn’t happen with old versions such as VS 2005, 2008)
  • You add a .sql file to a project that doesn’t support .sql files, for example to a C# project. The problem doesn’t happen with projects that support .sql files such as database projects.
  • If you open the project item to get its (invisible) window, get its text document, modify it and close the window saving the changes, the changes are not persisted to disk.

You have the details and code to reproduce the problem here:

PRB: Window.Close doesn’t save changes if requested for .sql files in non-database projects
http://www.mztools.com/articles/2014/MZ2014027.aspx

I am not sure if this is a bug or not, because the project doesn’t support .sql files. There is an easy workaround, though. You can call:

window.Document.Save(“”);
window.Close(EnvDTE.vsSaveChanges.vsSaveChangesNo);

instead of:

window.Close(EnvDTE.vsSaveChanges.vsSaveChangesYes);

VS SDK, packages, add-ins, macros and more…