Hosting Visual Studio Extensions on a private NuGet Gallery

NuGet is a great way of distributing software packages you use during development of your .NET applications. Because NuGet itself is open-source, the team have given tools (such as NuGet.Server and NuGetGallery) to help you jump-start deploying your own package feeds which can become integral to your company's development workflow.

Another aspect of development with Visual Studio is extensions, which can come in a variety of forms, from IDE updates, language services, and even project and item templates. Sadly at this point, the mechanism by which updates are made available does not currently align with how software packages are provide.

For Visual Studio Extensions, these are provided through an Atom feed. Because this is feed-driven, it does open up scenarios whereby we could publish our own Atom feed for our own internal extensions, much like running a private NuGet feed.

This got me thinking, that perhaps we could utilise the NuGet Gallery backend to provide a feed, but adapted as an Atom feed to be consumed by Visual Studio.

Turns out, it's relatively easy to do, so first things first, I forked the NuGetGallery repo and got to work.

Adding support for an Atom feed

My first port of call was to understand how the Atom feed was structured, and broadly it boils down to adding a custom <Vsix></Vsix> for handling versioning and Id, and update the <content> element to point to the location of the .vsix package.

I've added a controller, the ExtensionsController to generate the feed. VS can't consume a standard V1/V2/V3 NuGet feed, but we can shape our own feed based on the same data. Here's the controller:

public class ExtensionsController : Controller
{
  private readonly ISearchService _searchService;
  
  public ExtensionsController(ISearchService searchService)
  {
    _searchService = searchService;
  }
  
  public async Task<XDocumentResult> Feed()
  {
      var doc = await CreateFeedDocument();
      return new XDocumentResult(doc);
  }
}

We'll also need some routes:

routes.MapRoute(
    "ExtensionV1Feed",
    "api/extensions/v1/feed",
    new { controller = "Extensions", action = "Feed" }
);

routes.MapRoute(
    "ExtensionV1Content",
    "api/extensions/v1/{id}/{version}",
    new { controller = "Extensions", action = "VsixContent" }
);

I've added a custom XDocumentResult type which renders an XDocument to the response stream, but I've omitted it here for brevity. My CreateFeedDocument result constructs an XDocument by naively reading from the ISearchService with no arguments. I wanted to short-cut this part for getting results.

var filter = SearchAdapter.GetSearchFilter(string.Empty, 1, null, SearchFilter.UISearchContext);
var results = await _searchService.Search(filter);

Next up, we start constructing our document:

var doc = new XDocument(new XDeclaration("1.0", "utf-8", "false"));

XNamespace atom = "http://www.w3.org/2005/Atom";
XNamespace vsix = "http://schemas.microsoft.com/developer/vsx-syndication-schema/2010";

var root = new XElement(atom + "feed");
doc.Add(root);
root.Add(new XElement(atom + "title", new XAttribute("type", "text"), "My Extension Gallery"));
root.Add(new XElement(atom + "id", "My Extension Gallery"));

If we have results, we can mark an <updated> entry in the feed using the max Published date of the available packages:

if (results.Data.Any())
{
    var updated = results.Data.Max(p => p.Published);
    root.Add(new XElement(atom + "updated", updated.ToString("yyyy-MM-ddTHH:mm:ssZ")));
}

Now, for each package we are creating an <entry> item to be added to the <feed> element. Most of the attributes we can grab from our package model:

var entry = new XElement(atom + "entry");
entry.Add(new XElement(atom + "id", package.PackageRegistration.Id));
entry.Add(new XElement(atom + "title", new XAttribute("type", "text"), package.Title));
entry.Add(new XElement(atom + "summary", new XAttribute("type", "text"), package.Description));
entry.Add(new XElement(atom + "published", package.Published.ToString("yyyy-MM-ddTHH:mm:ssZ")));
entry.Add(new XElement(atom + "updated", package.Created.ToString("yyyy-MM-ddTHH:mm:ssZ")));
entry.Add(new XElement(atom + "author", new XElement(atom + "name", package.FlattenedAuthors)));

if (!string.IsNullOrWhiteSpace(package.IconUrl))
{
    entry.Add(new XElement(atom + "link", new XAttribute("rel", "icon"), new XAttribute("type", "text"), new XAttribute("href", package.IconUrl)));
}

if (!string.IsNullOrWhiteSpace(package.ProjectUrl))
{
    entry.Add(new XElement(atom + "link", new XAttribute("rel", "alternate"), new XAttribute("type", "text/html"), new XAttribute("href", package.ProjectUrl)));
}

In terms of tags, there is nothing similar with Atom feeds, but we can map NuGet tags to Atom categories:

var tags = (package.Tags ?? "").Split(new[] { ',', ' ' }, StringSplitOptions.RemoveEmptyEntries).Select(t => t.Trim());
foreach (string tag in tags)
{
    entry.Add(new XElement(atom + "category", new XAttribute("term", tag)));
}

Now, the last two import parts, content and the Vsix extension element. The <content> element needs to point to a URL that Visual Studio can download the .vsix package. As NuGet packages are zip archives, we need a way of accessing the .vsix package from within that. So we'll add this soon, but for now, let's generate a URL:

entry.Add(new XElement(atom + "content",
    new XAttribute("type", "application/octet-stream"),
    new XAttribute("src", Url.RouteUrl("ExtensionV1Content", new { id = package.PackageRegistration.Id, version = package.NormalizedVersion }))));

This will generate a URL similar to /api/extensions/v1/<package-name>/<version> which will implement soon.

Lastly, implement the custom <Vsix> element:

var ext = new XElement(vsix + "Vsix",
    new XAttribute(XNamespace.Xmlns + "xsd", "http://www.w3.org/2001/XMLSchema"),
    new XAttribute(XNamespace.Xmlns + "xsi", "http://www.w3.org/2001/XMLSchema-instance"),
    new XElement(vsix + "Id", package.PackageRegistration.Id),
    new XElement(vsix + "Version", package.NormalizedVersion));

And that's really it, this will allow us to generate the feed, so visiting /api/extensions/v1/feed will give us something like:

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title type="text">My Extension Gallery</title>
  <id>My Extension Gallery</id>
  <updated>2015-12-07T12:12:48Z</updated>
  <entry>
    <id>6165843C-6289-4B47-B61B-18C969AD9547</id>
    <title type="text">Sample Extension</title>
    <summary type="text">Sample Extension</summary>
    <published>2015-12-07T12:12:48Z</published>
    <updated>2015-12-07T12:12:48Z</updated>
    <author>
      <name>Matthew Abbott</name>
    </author>
    <content type="application/octet-stream" src="/api/extensions/v1/6165843C-6289-4B47-B61B-18C969AD9547/1.8.0" />
    <Vsix xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/developer/vsx-syndication-schema/2010">
      <Id>6165843C-6289-4B47-B61B-18C969AD9547</Id>
      <Version>1.8.0</Version>
    </Vsix>
  </entry>
</feed>

The <Id> element is quite important, as this unique ID (which is the same ID set in the .vsixmanifest file) is used by VS to determine what is installed and at what versions.

We can now test this by adding the extension feed to Visual Studio:

And to test, let's browse for the extension:

Flowing `.vsix` packages from NuGet packages

As mentioned previously, NuGet packages are really just zip containers, and the standard NuGetGallery project doesn't do much in terms of reading the content, but with a few small modifications to the INupkg and NuPkg types, we can expose a method to read from the package on server.

We add a GetFileStream method which allows us to read a file, buffered in a memory stream:

public Stream GetFileStream(string filePath)
{
    if (filePath.StartsWith("\\"))
    {
        filePath = filePath.Substring(1);
    }
    filePath = filePath.Replace("\\", "/");

    using (var za = new ZipArchive(GetStream(), ZipArchiveMode.Read, true))
    {
        var entry = za.GetEntry(filePath);
        if (entry == null)
        {
            return null;
        }

        var memoryStream = new MemoryStream();

        using (var entryStream = entry.Open())
        {
            entryStream.CopyTo(memoryStream);
        }

        memoryStream.Seek(0, SeekOrigin.Begin);

        return memoryStream;
    }
}

Next, let's now add the method to our ExtensionsController:

public async Task<ActionResult> VsixContent(string id, string version)
{
    string fileName = string.Format(Constants.PackageFileSavePathTemplate, id.ToLower(), version.ToLower(), Constants.NuGetPackageFileExtension);
    var file = await _fileStorageService.GetFileReferenceAsync(Constants.PackagesFolderName, fileName);
    if (file == null)
    {
        return HttpNotFound();
    }

    using (var stream = file.OpenRead())
    {
        using (var pkg = new Nupkg(stream, true))
        {
            string embeddedFileName = pkg.GetFiles().FirstOrDefault(f => Path.GetExtension(f).Equals(".vsix", StringComparison.OrdinalIgnoreCase));

            var vsix = pkg.GetFileStream(embeddedFileName);
            if (vsix != null)
            {
                return new FileStreamResult(vsix, "application/octet-stream")
                {
                    FileDownloadName = Path.GetFileName(embeddedFileName)
                };
            }
        }
    }

    return HttpNotFound();
}

Here we are actually first finding the .vsix package within the NuGet package, and then return that to the response using the standard FileStreamResult.

We should now be in a position to install the package from our custom feed.

Updating extensions from NuGet feeds.

Because extensions are provided through the same NuGet core as normally packages, you can use nuget.exe the same way as part of your workflow, either manually, or through CI/CD. This allows a great deal of automation when you are authoring updates to your internal extensions. Alternatively, because this is based on NuGet Gallery, you can simply upload a new version of your extension, e.g, I currently have this listed in my personal extensions gallery:

I can use the UI to update this. If I deploy 1.8.0 over 1.7.0, this should be reflected in the Extensions dialog:

If Automatically update this extension is enabled, then you may not even have to update the extension, as VS will check daily (and every time you open the Extensions dialog) for new extensions.

So, imagine now how you can utilise the power of NuGet to deploy your internal extensions, like project templates and items, using the same tool chain as deploying your own NuGet packages.

The code is available as a fork on GitHub.

Adding support for an Atom feed

Flowing .vsix packages from NuGet packages

Updating extensions from NuGet feeds.

Flowing `.vsix` packages from NuGet packages