by ·1 Comment
abbreviations

We all use libraries from NuGet tremendously often. But how many times you’ve created your own, even the simplest library that is being published there? I expect, not so often… So do I. Thus, I decided to make the super simple example of doing that, while utilizing the new and shiny GitHub Actions for all automations.

This serves many purposes, as often when I do something:

  • Minimum Viable Product (or example) of creating your own library that is automatically tested/versioned and published as a NuGet package – it requires to combine various building blocks and it turns out to be not-so-trivial. My goal would be to provide it as a kind of template for creating a new NuGet-based libraries.
  • playground for fresh open source contributors that are willing to do some contributions that will be then propagated to a regular NuGet package – without a risk and fear of destroying something
  • funny, joke-like package that, still, may be extended/improved
  • and obviously, a great place to learn all this stuff (including me)

So this is how Strings.Abbreviations has been coined. A very simple C#-based library that makes the simplest thing I could imagine – provide sets of methods that resolve popular abbreviations like BRB or IMHO. For example Strings.BRB() with return “Be right back” string. To add some “logic”, there is an additional option to use “title casing”. So, Strings.BRB(titleCase: true) will return “Be Right Back”.

Obviously, this is trivial but it is by design! Thanks to that there is a very low entrance level for any contributor. Additionally, automation workflow is not hidden inside the complex logic and dependencies.

Initial design

So, this repository is based on a regular solution containing .NET Core library and test project. After the recent PR, there is also a trivial F#-based library (because we can!). As I said, the core part here is that Strings.Abbreviations is build, tested and published as a NuGet package with the use of Github Actions. All workflows we define are living inside the repository under .github/workflows folder. Currently there are two working workflows defined.

Building and testing

Every push and PR to master will trigger building and unit testing the whole solution. What’s super nice – thanks to that the result of the tests are available as an additional PR check. And we have a “badge” available on the top of this file (we will return to that).

Setting this was trivial, as we can use the default .NET Core workflow suggested when using Actions for the very first time. Underneath this workflow uses dotnet restore/build/test commands.

Building and publishing a NuGet package

Every push to master that bumps up Version in the Strings.Abbreviations.csproj file will trigger building and publishing a new NuGet package. Additionally, a new git tag will be pushed with the given version. It uses Publish NuGet custom Github Action which was also really easy to set up thanks to the documentation. All package data are taken from csproj file (no need for a separate .nuspec file). The only additional thing required here is to add NUGET_API_KEY secret key from the repository Settings tab.

Badges

To make our repository more professional and fancy, it is nice to add some “badges”. Readme.md file contains some. One is provided by the build-and-test workflow, and others (like current NuGet version or number of downloads) are provided by super-popular https://shields.io/ site:

What’s next?

It is not a full MVP still. There is a field of many improvements to make it a true template for potential library authors, including:

  • the next big thing to add is producing GitHub releases every time a new version is being created. I wanted to depend on version tags being already pushed by the publish-nuget workflow. Unfortunately I have an issue with that.
  • GitHub Actions for creating prerelease versions
  • some simple documentation auto-generation example, like the list of currently supported abbreviations
  • more automated tests, like code coverage or even benchmarks
  • the core “logic” improvement, like auto-generating title case versions, more abbreviations, more languages etc.
  • …? Any other good, not-to-complex automation ideas

So, in the end, please feel invited to look around and help me building it with your contributions. Any contributions to the “logic” itself is also welcomed!

by ·12 Comments
Mobius Overview

.NET application is “just” a piece of CIL bytecode to be executed by the .NET runtime. And .NET runtime is “just” a program that is able to perform this task. It happens that currently .NET Framework/.NET Core runtimes are written in C++. I am also fully aware of CoreRT that was .NET runtime with many parts rewritten to C# (like type system) but still, crucial parts (including JIT compiler and the GC) were left written in C++.

But what if we write .NET runtime as… .NET application? Is is possible at all? I mean, literally no native/C++ code, everything running as .NET Core application written in C#? Does this sound like kind of inception and infinite recursion? It would require running one .NET runtime on the top of another .NET runtime, right?

I decided to check it out and that’s how Mobius runtime idea has been coined! Yeah, I know it sound strange and I do not expect it will be anything close to production ready thingy in the nearest century. I am fully aware of the amount of code needed to be written to make full .NET runtime. However, I found it interesting to validate such idea and I find it small usages as well. Imagine a NuGet package with the separate runtime that you can add to your application 😉

Continue reading

by ·16 Comments
zerogclead

A few months ago I wrote an article about Zero GC in .NET Core 2.0. This proof of concept was based on a preview version of .NET Core 2.0 in which a possibility to plug in custom garbage collector has been added. Such “standalone GC”, as it was named, required custom CoreCLR compilation because it was not enabled by default. Quite a lot of other tweaks were necessary to make this working – especially including required headers from CoreCLR code was very cumbersome.

However upcoming .NET Core 2.1 contains many improvements in that field so I’ve decided to write follow up post. I’ve also answered one of the questions bothering me for a long time (well, at least started answering…) – how would real usage of Zero GC like in the context of ASP.NET Core application?

.NET Core 2.1 changes

Here is a short summary of most important changes. I’ve updated CoreCLR.Zero repository to reflect them.

  • first of all, as previously mentioned, now standalone GC is pluggable by default so no custom CoreCLR is required. We will be able to plug our custom GC just by setting a single environment variable:
  • as standalone GC matured, documentation in CoreCLR appeared
  • a great improvement is that code between library implementing standalone GC and CoreCLR has been greatly decoupled. Now it is possible to include only a few files directly from CoreCLR code to have things compiled:

    Previously I had to create my own headers with some of the declarations from CoreCLR copy-pasted which was obviously not maintanable and cumbersome.
  • loading path has been refactored slightly. InitializeGarbageCollector inside CoreCLR calls GCHeapUtilities::LoadAndInitialize() with the following code inside:

    Inside LoadAndInitializeGC there is a brand new functionality – verification of GC/EE interface version match. It checks whether version used by standalone GC library (returned by GC_VersionInfo function) matches the runtime version – major version must match and minor version must be equal or higher. Additionaly, GC initialization function has been renamed to GC_Initialize.
  • core logic of my the poor man’s allocator remained the same so please refer to the original article for details

ASP.NET Core 2.1 integration

As this CoreCLR feature has matured, I’ve decided do use standard .NET CLI instead of CoreRun.exe. This allowed me to easily test the question bothering me for a long time – how even the simplest ASP.NET Core application will consume memory without garbage collection? .NET Core 2.1 is still in preview so I’ve just used Latest Daily Build of .NET CLI to create WebApi project:

I’ve modified Controller a little to do something more dynamic that just returning two string literals:

Additionally, I’ve disabled Server GC which is enabled by default. Obviously setting GC mode does not make sense as there is no GC at all, right? However, Server GC crashes runtime because GC JIT_WriteBarrier_SVR64 is being used which requires valid card table address – and there are no card tables either 🙂

Then we simply compile and run, remembering about the environment variable:

Everything should be running fine so… congratulations! We’ve just run ASP.NET Core application on .NET Core with standalone GC plugged in which is doing nothing but allocating.

Benchmarks

I’ve created the same WebApi via regular .NET Core 2.0 CLI for reference. Then via SuperBenchmarker I’ve started simple load test: 10 concurrent users making 100 000 requests in total with 10 ms delay between each request.

.NET Core 2.1 with Zero GC:

zerogcaspnetcore02

.NET Core 2.0:

zerogcaspnetcore03

As we can see classic GC from .NET Core was able to process slightly more requests (357.8 requests/second) comparing to version with Zero GC plugged in. It does not surprise me at all because my version uses the most primitive allocation based on calloc. I’m quite surprised that Zero GC is doing so well after all. However, this is not so interesting because I assume that replacing calloc with a simple bump a pointer allocation would improve performance noticeably.

What is interesting is the memory usage over time. As you can see in the chart below, after a minute of such test, the process using Zero GC takes around 1 GB of memory. This is… quite a lot. Not sure yet how to interpret this. Version with regular GC ended with a stable 120 MB size. Both started from fresh run.

zerogcaspnetcore

This would mean that each REST WebApi requests triggers around 55 kB of allocations. Any comments will be appreciated here…

Update 30.01.2018: After debugging allocations during single ASP.NET requests, most of them comes from RouterMiddleware. This is no surprise as currently this application does almost nothing but routing… I’ve uploaded sample log of such single request which seems to be minimal (others are allocating some buffers from time to time). It consumes around 7 kB of memory.

by ·6 Comments

We can often hear that allocation of objects is “cheap” in .NET. I fully support this sentence because the most important part is its continuation – allocation is cheap but allocating a lot of objects will hit you back as sooner or later garbage collector will kick in and start messing around. Thus, the fewer allocations, the better.

However, I would like to add a few words about “allocation is cheap” itself. This is true to some extent because the typical path of objects allocation is indeed really fast. So-called bump a pointer technique is most often used. It consists of the following simple steps:

  • it uses so-called allocation pointer as an address of a newly created object
  • it increases allocation pointer by the requested size (so next object will be created there

Continue reading

by ·0 Comments
Zero Garbage Collector on .NET Core

Starting from .NET Core 2.0 coupling between Garbage Collector and the Execution Engine itself have been loosened. Prior to this version, the Garbage Collector code was pretty much tangled with the rest of the CoreCLR code. However, Local GC initiative in version 2.0 is already mature enough to start using it. The purpose of the exercise we are going to do is to prepare Zero Garbage Collector that replaces the default one.

Zero Garbage Collector is the simplest possible implementation that in fact does almost nothing. It only allows you to allocate objects, because this is obviously required by the Execution Engine. Created objects are never automatically deleted and theoretically, no longer needed memory is never reclaimed. Why one would be interested in such a simple GC implementation? There are at least two reasons:Continue reading