Bug with unsafe (fixed buffer) structs

ApocDev

For performance reasons, we have a few structures that use fixed-buffer arrays to align padding, and provide quicker array access (via pointer usage) instead of the typical [MarshalAs] approach.

Example struct:

[StructLayout(LayoutKind.Sequential, Pack=1)]
unsafe struct Example
{
        public IntPtr InternalEntries;
        public uint MaxEntries;
        private fixed byte padding8[8];
        public uint NumEntries;
        private fixed byte padding14[4];
        public IntPtr InternalEntriesSecond;
        private fixed byte padding1c [4];
}

public static class MarshalCache<T>
    {
        public static int Size;

        static MarshalCache()
        {
            if (typeof(T).IsEnum)
            {
                var underlying = typeof(T).GetEnumUnderlyingType();
                Size = Marshal.SizeOf(underlying);
            }
            else Size = Marshal.SizeOf(typeof(T));
        }
    }

The above class is simply a cache for Marshal.SizeOf (its fairly slow on complex types, so we just pseudo-cache it with the static-generic trick)

In a console application:

Console.WriteLine(MarshalCache<Example>.Size)

The above will print "32" when built from VS. (Assuming x86 mode due to IntPtr changing between 4/8 bytes)

However, if you run SA over the same console application, with literally zero options enabled, it will print "19". This is causing a ton of bugs in our application where we've done optimizations to reduce CPU usage.

Brian Donahue

Thanks - I will turn this over to the development team as I really don't understand this very well -- you have an enum, which would normally be a long or int, but instead you are making an enum out of struct, and the struct is packed. Then you ask marshal about the size of the struct and it does not report the correct length. Is that the gist of it?

It would probably be important to let us know which protection options you are using so we can try to isolate which one may be interfering with this.

ApocDev

Brian Donahue wrote:

Thanks - I will turn this over to the development team as I really don't understand this very well -- you have an enum, which would normally be a long or int, but instead you are making an enum out of struct, and the struct is packed. Then you ask marshal about the size of the struct and it does not report the correct length. Is that the gist of it?

It would probably be important to let us know which protection options you are using so we can try to isolate which one may be interfering with this.

Not quite. The issue is with structure sizes. (The enum sanity check is due to Marshal.SizeOf not being able to determine the size of an enum, so we grab the underlying type (enum Foo: uint, will cause the underlying type to be uint, [4 bytes]))

The bug is with structs with fixed buffers. (public fixed byte Padding[50];)

This happens with zero protection options turned on. (Not even strong-name signing, or error reporting. No obfuscation, no pruning, no string encryption, etc)

Brian Donahue

The problem happens when the object is not an enum, then.

Marshal.SizeOf(typeof(T));

Brian Donahue

I tried to get a reproduction, but both before and after SmartAssembly, my example prints out "32".

using System;
using System.Collections.Generic;
using System.Text;
using System.Runtime.InteropServices;

namespace ConsoleApplication1
{
 class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine(MarshalCache<UnsafeStuff.Example>.Size);
            Console.ReadLine();
        }
    }
 public class UnsafeStuff
    {
        [StructLayout(LayoutKind.Sequential, Pack = 1)]
        public unsafe struct Example
        {
            public IntPtr InternalEntries;
            public uint MaxEntries;
            private fixed byte padding8[8];
            public uint NumEntries;
            private fixed byte padding14[4];
            public IntPtr InternalEntriesSecond;
            private fixed byte padding1c[4];
        }
    }

class MarshalCache<T>
    {
        public static int Size;
        static MarshalCache()
        {
            if (typeof(T).IsEnum)
            {
                 Size = 9999; // cut this bit as it's not relevant
            }
            else Size = Marshal.SizeOf(typeof(T)); 
        }
    }

}