Efficiency driven developer. Ex-Microsoft (ADX, AATP, ATA). Ex-Aorato (Acquired by Microsoft).

profile for i3arnon on Stack Overflow
© Bar Arnon
  • October 3, 2015 | Protobuf-net, Bugs | comments | Edit
  • Protobuf-net Is Broken Around DateTime
  • Protocol Buffers by Google are a great mechanism for serializing (and deserializing) structured data in a very fast and efficient way. Protobuf-net is Marc Gravell’s port of Protocol Buffers for the .NET ecosystem.

    While being very efficient, protobuf-net has a big issue when deserializing .NET’s DateTimes. Behind the scenes DateTimes are converted into Unix-Time which is a count (of ticks in this case) starting from the Unix Epoch (1970/01/01 UTC). When deserializing back to .NET protobuf-net adds that count to a DateTime representing the Epoch-Time resulting in the correct DateTime value. The issue with this process is that it loses the DateTime’s original DateTimeKind.

    DateTimeKind is an enum telling whether the DateTime’s value represents a local time, UTC time or unspecified. That value isn’t serialized by protobuf-net so all DateTimes, be they local time or UTC, are deserialized as DateTimeKind.Unspecified.

    DateTimeKind.Unspecified values have a behavior that I initially found surprising but later realized is the best possible option. Let’s assume you’re in Hawaii (because where else would you want to be?) and your time zone is UTC-10:00. If you have a DateTime value with DateTimeKind.Unspecified and you call ToLocalTime the method assumes the value is in UTC and “corrects” it, so 11:00 becomes 01:00. If however you call ToUniversalTime on that value the method now assumes it’s in local time and “corrects” it so 11:00 becomes 21:00. So the same value is treated as local while adjusting to universal and universal when adjusting to local. Let’s see that in code:

    static void Main()
    {
        var dolly = new Sheep { DateOfBirth = new DateTime(1966, 07, 05, 11, 0, 0, DateTimeKind.Utc) };
        Console.WriteLine(dolly.DateOfBirth.ToString("HH:mm:ss K")); // "11:00:00 Z" (Z means UTC)
    
        dolly = Serializer.DeepClone(dolly); // Serialize and deserialize using protobuf-net
        Console.WriteLine(dolly.DateOfBirth.ToString("HH:mm:ss K")); // "11:00:00" (no Z means unspecified)
        Console.WriteLine(dolly.DateOfBirth.ToLocalTime().ToString("HH:mm:ss K")); // "01:00:00 -10:00" (Hawaii timezone)
        Console.WriteLine(dolly.DateOfBirth.ToUniversalTime().ToString("HH:mm:ss K")); // "21:00:00 Z"
    }
    
    [ProtoContract(ImplicitFields = ImplicitFields.AllPublic)]
    class Sheep
    {
        public DateTime DateOfBirth { get; set; }
    }
    

    This can get extremely problematic especially if you, like me, depend upon some library that uses ToUniversalTime or ToLocalTime. For me that library was the .NET’s MongoDB Driver that stores all DateTimes in MongoDB in UTC. Using these 2 libraries together is impossible as DateTime values would keep changing value infinitely.

    I have posted on the protobuf-net repository on github explaining this and managed to convince him to fix this problem (which he did with this commit). However, this fix was made 5 months prior to me writing this post and there’s still isn’t a new release including the fix (the latest stable release is from 2013/09/30).

    But don’t fear… I do have a workaround you can use for the meantime. Protobuf-net uses a DateTime value representing the Unix Epoch (1970/01/01) to create all the DateTimes by adding the relevant delta to the epoch value. Since creating a new DateTime from an existing DateTime preserves the DateTimeKind, replacing the single epoch value with a UTC one will result with all protobuf-net DateTime values having DateTimeKind.UTC. We can do that by using reflection and replacing the epoch value with a UTC one:

    typeof (BclHelpers).
        GetField("EpochOrigin", BindingFlags.NonPublic | BindingFlags.Static).
        SetValue(null, new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc));
    
    var dolly = new Sheep { DateOfBirth = new DateTime(1966, 07, 05, 11, 0, 0, DateTimeKind.Utc) };
    Console.WriteLine(dolly.DateOfBirth.ToString("HH:mm:ss K")); // "11:00:00 Z" (Z means UTC)
    
    dolly = Serializer.DeepClone(dolly); // Serialize and deserialize using protobuf-net
    Console.WriteLine(dolly.DateOfBirth.ToString("HH:mm:ss K")); // "11:00:00 Z"
    

    I admit it’s not pretty, but until Marc releases a new version, it’s preferable to building your own protobuf-net.