What’s with .NET nowadays?

I was a bit confused about the current state of .NET. So I put together this very simplified overview.

.NET Standard Library
The formal specification for .NET. Specifies a minimal set of APIs for a .NET implementation.

.NET Framework
The old .NET implementation from Microsoft. Proprietary license. For Windows only.

Mono (Xamarin)
A third party implementation of .NET. Open source. Cross-platform.

.NET Core
The new .NET implementation from Microsoft. Open source. Cross-platform.

ASP.NET
The old set of web frameworks for .NET. Open source. Runs on .NET Framework and Mono.

ASP.NET Core
The new set of web frameworks for .NET. Open source. Runs on .NET Framework and .NET Core.

Windows Forms
The old technology for desktop applications for Windows. Runs on .NET Framework.

Windows Presentation Foundation
The newer technology for desktop applications for Windows. Runs on .NET Framework.

Universal Windows Platform
The newest technology for applications for Windows and Windows Phone. Runs on .NET Framework and .NET Core.

Visual Studio
The IDE for .NET development. Proprietary license. For Windows only.

Visual Studio Code
An editor for .NET development. Open source. Cross-platform.

Advertisements

Unit testing: replacing dependencies

The point of unit testing is to test classes in isolation. This presents a problem when the class has dependencies. These dependencies must be replaced somehow.

namespace SpaceGame
{
    public class Level
    {
        private IEnumerable<IEnemy> enemies;

        public Level(params IEnemy[] enemies)
        {
            this.enemies = enemies;
        }

        public bool Cleared
        {
            get { return !enemies.Any(e => e.Alive); }
        }

        public void Shoot(int damage)
        {
            damage = damage / enemies.Count();
            foreach (var enemy in enemies)
            {
                enemy.Hit(damage);
            }
        }
    }
}

Dependency injection gives an opportunity to make those replacements. The dependencies are passed to the class instead of the class creating them itself. And by using interfaces the dependencies can be replaced with something else.

namespace SpaceGame.UnitTests
{
    [TestFixture]
    public class LevelTests
    {
        [Test]
        public void OneEnemyNotAlive_IsCleared()
        {
            var enemy = Substitute.For<IEnemy>();
            enemy.Alive.Returns(false);

            Level level = new Level(enemy);

            Assert.IsTrue(level.Cleared);
        }

        [Test]
        public void Shoot_TwoEnemies_DealsHalfDamageToEach()
        {
            var enemy1 = Substitute.For<IEnemy>();
            var enemy2 = Substitute.For<IEnemy>();

            Level level = new Level(enemy1, enemy2);
            level.Shoot(100);

            enemy1.Received().Hit(50);
            enemy2.Received().Hit(50);
        }
    }
}

Isolation frameworks provides a convenient way to create replacements for dependencies. The replacements can be configured to behave as needed for the test. They can also be used to check that they are called appropriately during the test. There are many isolation frameworks available for .NET. This example uses NSubstitute.

Unit testing: getting started

Unit testing is hailed by many as the best way to write and maintain code. Unit tests are written with the help of a unit testing framework. NUnit is the dominating framework in .NET.

namespace SpaceGame
{
    public class Enemy
    {
        private int hitpoints;

        public Enemy(int hitpoints)
        {
            this.hitpoints = hitpoints;
        }

        public bool Alive
        {
            get
            {
                return hitpoints > 0;
            }
        }

        public void Hit(int damage)
        {
            if (damage < 0)
            {
                throw new ArgumentOutOfRangeException("damage");
            }
            hitpoints -= damage;
        }
    }
}

In NUnit tests are contained in its own class. Normally with a test class for each class under test. It is also practical to put all tests in a separate project and assembly.

To run the tests we need a test runner. NUnit has its own runner in the form of a console or GUI program. However, it is much more convenient to use a runner that integrates with Visual Studio. If you already use ReSharper or CodeRush they might be a good choice.

namespace SpaceGame.UnitTests
{
    [TestFixture]
    public class EnemyTests
    {
        private Enemy enemy;

        [SetUp]
        public void Setup()
        {
            enemy = new Enemy(100);
        }

        [Test]
        public void NewEnemy_IsAlive()
        {
            Assert.IsTrue(enemy.Alive);
        }

        [TestCase(100)]
        [TestCase(101)]
        [TestCase(Int32.MaxValue)]
        public void Hit_OnceForMoreThanHitpoints_Kills(int damage)
        {
            enemy.Hit(damage);
            Assert.IsFalse(enemy.Alive);
        }

        [TestCase(0)]
        [TestCase(1)]
        [TestCase(50)]
        [TestCase(99)]
        public void Hit_OnceForLessThanHitpoints_RemainsAlive(int damage)
        {
            enemy.Hit(damage);
            Assert.IsTrue(enemy.Alive);
        }

        [TestCase(-1)]
        [TestCase(Int32.MinValue)]
        public void Hit_OnceForNegativeDamage_ThrowsException(int damage)
        {
            Assert.Catch<ArgumentOutOfRangeException>(() =>
            {
                enemy.Hit(damage);
            });
        }

        [Test]
        [Ignore("Currently fails due to bug 1")]
        public void Hit_TwiceForMaxInteger_Kills()
        {
            enemy.Hit(Int32.MaxValue);
            enemy.Hit(Int32.MaxValue);
            Assert.IsFalse(enemy.Alive);
        }
    }
}

An important goal when writing unit tests is to make them readable. Each test should test for one thing only and tests should not contain any logic of their own. Contrary to regular programming hardcoded values are a good thing when writing tests.

The last part shows how a test can be ignored which may be useful when a bug is uncovered but there isn’t time to fix it yet. This should be used sparingly of course.

Regular expressions in \.NET

The String class in .NET provides plenty of methods for dealing with text. But when they fall short we can turn to the Regex class.

bool valid = Regex.IsMatch(identifier, @"^[a-z][a-z0-9]*$");

string episode = Regex.Match(filename, @"S\d{2}E\d{2}").Groups[0].Value;

Match match = Regex.Match(url, @"^http://(?<domain>[^:]+):(?<port>[0-9]+)");
if (match.Success)
{
    string domain = match.Groups["domain"].Value;
    string port = match.Groups["port"].Value;
}

MatchCollection matches = Regex.Matches(users, @" (\w+),?");
foreach (Match match in matches)
{
    string user = match.Groups[1].Value;
}

Match match = Regex.Match(users, @"Users: ((\w+)(, )?)*");
foreach (Capture capture in match.Groups[2].Captures)
{
    string user = capture.Value;
}

The syntax for regular expressions in .NET is fairly standard. Character classes uses Unicode unless you pass the ECMAScript flag. Remember to escape backslashes or use verbatim string literals. The last example uses captures, which is a way to get history when multiple matches are made.

Regex regex = new Regex(@"^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} (.+)$",
    RegexOptions.ECMAScript | RegexOptions.Compiled);

foreach (string line in log)
{
    Match match = regex.Match(line);
    if (!match.Success) { continue; }
    string message = match.Groups[1].Value;
}

The Regex methods can be called statically, like in the earlier examples, or the class can be instantiated as in the latter example. Since a regular expression have to be processed before it can be used it makes sense to reuse it. However, when used statically the Regex class uses an internal cache, so the static usage isn’t as bad as it may seem.

Regex objects can also be processed further by passing the Compiled flag. The regular expression is then turned into an assembly. This makes string matching faster but comes at a significant startup and memory cost. You can even go all out and turn a regular expression into a permanent assembly.

Regular expressions in Visual Studio

image

As useful as regular expressions are in code it can be even more useful as a tool when writing code. But for reasons unknown the syntax used in Visual Studio is very different from that in .NET. For example, tags are enclosed with { } instead of ( ), character classes are prefixed with : instead of \ and repetitions are expressed as ^n instead of {n}.

F# for real: a troposphere model

F# is a functional language for .NET with some imperative elements. It has full support in Visual Studio 10 and later (except for the Express editions). F# can be a useful complement to C# but you have to keep them in separate projects.

Instead of making a tutorial we are going to look at a real world example where F# can help us. This is mainly thanks to a feature called Units of Measure which lets us introduce measurement units into the type system.

The example I have written is an implementation of a tropospheric error model for GPS as described in the system specifications for SBAS (an augmentation system for GPS). Section 6.3 on page 32 in this PDF has more information.

namespace GpsErrorModels
open System

// Units and equivalences
[<Measure>] type degree
[<Measure>] type kelvin
[<Measure>] type kilogram
[<Measure>] type meter
[<Measure>] type second
[<Measure>] type joule = kilogram meter^2 / second^2
[<Measure>] type millibar = kilogram / meter / second^2

// Tropospheric model (defined in RTCA/DO-229C) for a specific receiver
// position (latitude and height) and time (date)
type TroposphereModel(latitude:float<degree>, height:float<meter>,
                      time:DateTime) =

    let Interpolate_ζ(latitude:float<degree>, values:float<'u> list):float<'u> =
        let interpolate(k, k0, k1, v0, v1):float<'u> =
            // Interpolate (equations A-4 and A-5)
            v0 + (v1 - v0) * (k - k0) / (k1 - k0);
        let latitudes = [15.0<degree>; 30.0<degree>; 45.0<degree>;
                         60.0<degree>; 75.0<degree>;]
        // Interpolation according to table A-2
        if latitude < latitudes.[0] then values.[0]
        elif latitude < latitudes.[1] then
            interpolate(latitude, latitudes.[0], latitudes.[1],
                        values.[0], values.[1])
        elif latitude < latitudes.[2] then
            interpolate(latitude, latitudes.[1], latitudes.[2],
                        values.[1], values.[2])
        elif latitude < latitudes.[3] then
            interpolate(latitude, latitudes.[2], latitudes.[3],
                        values.[2], values.[3])
        else values.[4];

    let Get_ζ(latitude:float<degree>, time:DateTime, averages:float<'u> list,
              variations:float<'u> list):float<'u> =
        let ζ0 = Interpolate_ζ(latitude, averages)
        let Δζ = Interpolate_ζ(latitude, variations)
        let D = (float)time.DayOfYear
        let Dmin = if latitude > 0.0<degree> then 28.0 else 211.0
        // Compute meterological parameter (equation A-3)
        ζ0 - Δζ * Math.Cos(2.0 * Math.PI * (D - Dmin) / 365.25);

    // Pressure values from table A-2
    let Get_P(latitude:float<degree>, time:DateTime) =
        let averages = [1013.25<millibar>; 1017.25<millibar>; 1015.25<millibar>;
                        1011.25<millibar>; 1013.00<millibar>;]
        let variations = [ 0.00<millibar>; -3.75<millibar>; -2.25<millibar>;
                          -1.75<millibar>; -0.50<millibar>;]
        Get_ζ(latitude, time, averages, variations);

    // Temperature values from table A-2
    let Get_T(latitude:float<degree>, time:DateTime) =
        let averages = [299.65<kelvin>; 294.15<kelvin>; 283.25<kelvin>;
                        272.15<kelvin>; 263.65<kelvin>;]
        let variations = [ 0.00<kelvin>; 7.00<kelvin>; 11.00<kelvin>;
                          15.00<kelvin>; 14.50<kelvin>;]
        Get_ζ(latitude, time, averages, variations);

    // Water vapor pressure values from table A-2
    let Get_e(latitude:float<degree>, time:DateTime) =
        let averages = [26.31<millibar>; 21.79<millibar>; 11.66<millibar>;
                         6.78<millibar>; 4.11<millibar>;]
        let variations = [0.00<millibar>; 8.85<millibar>; 7.24<millibar>;
                          5.36<millibar>; 3.39<millibar>;]
        Get_ζ(latitude, time, averages, variations);

    // Temperature lapse rates from table A-2
    let Get_β(latitude:float<degree>, time:DateTime) =
        let averages = [6.30E-3<kelvin/meter>; 6.05E-3<kelvin/meter>;
                        5.58E-3<kelvin/meter>; 5.39E-3<kelvin/meter>;
                        4.53E-3<kelvin/meter>;]
        let variations = [0.00E-3<kelvin/meter>; 0.25E-3<kelvin/meter>;
                          0.32E-3<kelvin/meter>; 0.81E-3<kelvin/meter>;
                          0.62E-3<kelvin/meter>;]
        Get_ζ(latitude, time, averages, variations);

    // Water vapor "lapse rates" from table A-2
    let Get_λ(latitude:float<degree>, time:DateTime) =
        let averages = [2.77<1>; 3.15<1>; 2.57<1>; 1.81<1>; 1.55<1>;]
        let variations = [0.00<1>; 0.33<1>; 0.46<1>; 0.74<1>; 0.30<1>;]
        Get_ζ(latitude, time, averages, variations);

    let ZenithDelay:float<meter> =
        // Meteorological values from table A-2
        let P:float<millibar> = Get_P(latitude, time)
        let T:float<kelvin> = Get_T(latitude, time)
        let e:float<millibar> = Get_e(latitude, time)
        let β:float<kelvin/meter> = Get_β(latitude, time)
        let λ:float<1> = Get_λ(latitude, time)
        // Constants defined on page A-9 and A-10
        let k1 = 77.604<kelvin/millibar>
        let k2 = 382000.0<kelvin^2/millibar>
        let Rd = 287.054<joule/kilogram/kelvin>
        let gm = 9.784<meter/second^2>
        let g = 9.80665<meter/second^2>
        // Zero-altitude zenith dry delay (equation A-6)
        let z_hyd:float<meter> = (1E-6 * k1 * Rd * P)
                                / gm
        // Zero-altitude zenith wet delay (equation A-7)
        let z_wet:float<meter> = (1E-6 * k2 * Rd)
                               / (gm * (λ + 1.0) - β * Rd)
                               * e / T
        // Height-mapped zenith dry delay (equation A-8)
        let d_hyd:float<meter> = (1.0 - β * height / T)
                              ** (g / (Rd * β))
                               * z_hyd
        // Height-mapped zenith wet delay (equation A-9)
        let d_wet:float<meter> = (1.0 - β * height / T)
                              ** ((λ + 1.0) * g / (Rd * β) - 1.0)
                               * z_wet
        // Combined zenith dry and wet delay (part of equation A-2)
        d_hyd + d_wet;

    // Calculate the tropospheric delay for a satellite given its elevation
    member this.GetCorrection(elevation:float<degree>):float<meter> =
        let Sin(angle:float<degree>):float =
            Math.Sin(((float)angle * Math.PI / 180.0))
        // Pre-calculated zenith delay
        let d = ZenithDelay
        // Elevation mapping value (equation A-10)
        let m = 1.001 / sqrt (0.002001 + Sin(elevation) ** 2.0)
        // Tropospheric delay correction (equation A-2)
        -d * m;

To show how the unit system is used I have highlighted the relevant lines leading up to the equation on lines 103 to 105. Because the code compiles I can be reasonably sure the equation is correct. If I had made a mistake and the units didn’t match up it would have shown as an error in exactly the same way as if the types didn’t match.

This system isn’t perfect. You can still make mistakes that aren’t caught. But it gives an extra level of confidence that may well be worth it in some applications.

TroposphereModel troposphere = new TroposphereModel(
    59.35264, 39.773, DateTime.Parse("2004-07-08"));

Debug.Assert(Math.Abs(troposphere.GetCorrection(60.4) + 2.791) < 0.05);
Debug.Assert(Math.Abs(troposphere.GetCorrection(37.0) + 4.024) < 0.05);
Debug.Assert(Math.Abs(troposphere.GetCorrection(12.3) + 11.144) < 0.05);

Here is a piece of C# code where we call the F# code and check the results against a set of known values taken from a Nordnav-R30 GPS receiver.

LINQ: I only speak in SQL

LINQ to SQL (Language Integrated Query to Structured Query Language) is probably the best known use of LINQ. It gives us an easy to use way of communicating with SQL databases.

CREATE TABLE People (
  Id INT IDENTITY,
  Age INT NULL,
  Name VARCHAR(50) NULL
);

First we create a simple table in our database. Out of the box LINQ to SQL only supports SQL Server but you can use LINQ to DataSet or other LINQ providers to get around this.

[Table(Name = "People")]
public class Person
{
    [Column(IsPrimaryKey = true, IsDbGenerated = true)]
    public int Id;
    [Column]
    public int Age;
    [Column]
    public string Name;
}

In the code we then create a class that the table can be mapped against. We use attributes to tell LINQ that the class represents a table. In this case we also tell it the table name since it’s different from the class name. It makes sense to name tables in plural and classes in singular. In order for LINQ to work properly the table must have a primary key, which in this case is generated by the database.

Visual Studio can generate this class for us, or it can generate the database from the class or even both of them from a schema. In this case I have created both the class and the table manually.

DataContext database = new DataContext(
    @"Data Source     = 127.0.0.1,15096;
      Network Library = DBMSSOCN;
      Initial Catalog = linqtest;
      User Id         = sa;
      Password        = hunter2;");
Table<Person> people = database.GetTable<Person>();

Before we can use the database we must connect to it. We do this by creating a new DataContext and giving it a connection string. The DataContext takes care of connecting to the database as needed. It also keeps track of changes which we will come to later.

We must also create a table object in order to have something to query against. In practice you should create a strongly typed DataContext instead. That is, a class which inherits from DataContext and declares all tables as public fields.

from p in people select p.Age + " " + p.Name
"28 Edgar", "34 Marit", "14 Trulf"
from p in people orderby p.Age select p.Age + " " + p.Name
"14 Trulf", "28 Edgar", "34 Marit"
people.Where(p => p.Age > 20).Select(p => p.Age + " " + p.Name)
"28 Edgar", "34 Marit"

We can now use LINQ to run queries against the table. These queries are translated to expression trees which are in turn translated to SQL and run against the database.

foreach (Person p in people)
    p.Age++;
database.SubmitChanges();

from p in people orderby p.Age select p.Age + " " + p.Name
"15 Trulf", "29 Edgar", "35 Marit"

Part of the DataContext’s responsibilities is to keep track of changes to objects that correspond to table rows. Because of this we can make our changes permanent by submitting them to the database.

Person alan = new Person() { Age = 41, Name = "Alan" };
people.InsertOnSubmit(alan);
Person marit = people.Single(p => p.Name == "Marit");
people.DeleteOnSubmit(marit);
database.SubmitChanges();

people.Select(p => p.Name)
"Edgar", "Trulf", "Alan"

Naturally, we can also add or remove rows from the table. This also requires that we submit our changes. If you create associations to other tables additions and removals will be tracked automatically.

LINQ: from one IEnumerable to another

LINQ (Language Integrated Query) is a .NET framework which adds three things. A set of classes for use with IEnumerable, a new query syntax and a new IQueryable interface. The last one extends IEnumerable with the ability to run data queries. This is used by LINQ to SQL and LINQ to XML. But in this blog post I will focus on LINQ to Objects.

List<int> integers = new List<int> { 1, 2, 5, 7, 2, 9, 2 };
IEnumerable sorted = integers.OrderBy(i => i);
sorted;
1, 2, 2, 2, 5, 7, 9
integers.Add(4);
sorted;
1, 2, 2, 2, 4, 5, 7, 9
integers.Add(5);
sorted;
1, 2, 2, 2, 4, 5, 5, 7, 9

The first thing to learn is that LINQ methods does not return a list. What it returns is an object that knows what the data source is and knows what to do with it. This means that LINQ can, and will, run the query against the data source every time the result is requested. The example above shows what this means in practice. If you do not want this behavior you can convert the result to something permanent with ToList or ToArray.

from i in integers select i;
1, 2, 5, 7, 2, 9, 2, 4, 5
from i in integers select i + 2;
3, 4, 7, 9, 4, 11, 4, 6, 7
integers.Select(i => i + 3);
4, 5, 8, 10, 5, 12, 5, 7, 8
integers.Select((i, index) => index + ":" + i);
"0:1", "1:2", "2:5", "3:7", "4:2", "5:9", "6:2", "7:4", "8:5"

The LINQ query syntax as shown in the two first examples above is probably the most noticeable part of LINQ. However, it’s only syntactic sugar and hence it’s not a .NET extension but a language extension. Every query written in this new way can be written with methods as shown in the third example. There are LINQ methods that cannot be used via the query syntax.

The query syntax looks like SQL but notice that the select clause comes last. From and select are the only required clauses. You can use select to map a set of objects to a new set of objects.

from i in integers where i < 5 select i;
1, 2, 2, 2, 4
integers.Where(delegate(int i) { return i >= 5; });
5, 7, 9, 5
integers.Where((i, index) => i > index);
1, 2, 5, 7, 9

The where clause is used to filter the result set. You may have noticed the => syntax by now. This is the new lambda expression syntax which can be used wherever you used to use delegates before. I showed that you can still use delegates if you want to in the second example. There is however an important difference. Lambda expressions can be turned into expression trees which LINQ can translate into something else, for example SQL. This is how you can write LINQ queries in C# against a database. Hence you can’t use delegates with LINQ to SQL (which is hardly a loss).

from i in integers orderby i select i;
1, 2, 2, 2, 4, 5, 5, 7, 9
from i in integers orderby i / 3 descending, i select i;
9, 7, 4, 5, 5, 1, 2, 2, 2
integers.OrderByDescending(i => i / 3).ThenBy(i => i);
9, 7, 4, 5, 5, 1, 2, 2, 2

Orderby is used to order the set. You can order by multiple conditions as seen in the last two examples. Notice the use of ThenBy which is required, if OrderBy is used the whole set will be reordered and the first condition forgotten. The last example also shows chaining which is very useful in LINQ when you’re not using the query syntax.

from i in integers group i by i / 3 into g let m = g.Key * 3 select
  String.Format("{0}-{1}:{2}", m, m + 2, g.Count());
"0-2:4", "3-5:3", "6-8:1", "9-11:1"

You can use group to group elements like in SQL. This example also shows let which you can use to introduce a new variable inside the query. Here I use it so I don’t have to multiply by three twice.

List<KeyValuePair<int, char>> characters = new List<KeyValuePair<int, char>> {
  new KeyValuePair<int, char>(1, 'a'),
  new KeyValuePair<int, char>(2, 'b'),
  new KeyValuePair<int, char>(3, 'c'),
  new KeyValuePair<int, char>(4, 'd'),
  new KeyValuePair<int, char>(5, 'e')};
from c in characters select c.Key + "=" + c.Value;
"1=a", "2=b", "3=c", "4=d", "5=e"

from i in integers from c in characters where i == c.Key select c.Value;
'a', 'b', 'e', 'b', 'b', 'd', 'e'
from i in integers join c in characters on i equals c.Key select c.Value;
'a', 'b', 'e', 'b', 'b', 'd', 'e'
from i in integers
  join c in characters on i equals c.Key into n
  from k in n.DefaultIfEmpty(new KeyValuePair<int, char>(0, '_'))
  select i + "=" + k.Value;
"1=a", "2=b", "5=e", "7=_", "2=b", "9=_", "2=b", "4=d", "5=e"

Of course you can also use joins. If you use multiple from you get a cross join. A join on equals gives an inner join. In my example, where I use a where clause for the cross join, these give the same result. In SQL the cross join is highly discouraged so I would suggest not using it here either. The last example shows how to do an left outer join.

There are several other methods which I haven’t shown here, such as Single, Union, Count, Distinct etc. Or you can create your own if you feel something is missing.