4

It seems easy but I can't see it easy at all. My question is I have a struct and I need to convert it to byte stream without any additional bytes for types nor padding nor metadata. Assume I have a struct

[StructLayout(LayoutKind.Sequential)]
public struct MyStruct
{
    public ushort a;                 
    public uint b;  
    public uint c;
    public ushort d;
}

Remark: I can't change the pack here to 1(Project constrains), so using the following solution won't work as there's a padding added

int size = Marshal.SizeOf(typeof(T));
byte[] arr = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(str, ptr, true);
Marshal.Copy(ptr, arr, 0, size);
Marshal.FreeHGlobal(ptr);

Also Remark: I can't use Binary format serialization as there's a metadata added. All what I want simply if a=1 b=2 c=3 d=4 to get binary format in a GENERIC WAY as

    arr {byte[12]}  byte[]
    [0]  1  byte
    [1]  0  byte
    [2]  2  byte
    [3]  0  byte
    [4]  0  byte
    [5]  0  byte
    [6]  3  byte
    [7]  0  byte
    [8]  0  byte
    [9]  0  byte
    [10] 4  byte
    [11] 0  byte

Any help?

10
  • Have you tried manually creating that array with BitConverter? Commented Dec 5, 2017 at 9:13
  • Manually of course will work I need something generic Commented Dec 5, 2017 at 9:14
  • 1
    If you sprinkle a bit of reflection magic over it (GetFields) it would be very generic. Commented Dec 5, 2017 at 9:16
  • @ManfredRadlwimmer I tried that but I was not able find a complete solution if you have a snip of code to do so you can post it Commented Dec 5, 2017 at 9:18
  • 1
    @tulipe if your data includes an array: you can forget it - now you're talking about multiple objects, which means you're going to need to define what it should look like when you pack the data; there's no de-facto way of storing multiple objects or arrays (how is the range of the array demarked, for example). The only thing that has an automatic native representation is the layout of a single struct (or a vector of such structs). Fixed buffers inside structs work; arrays don't. Commented Dec 5, 2017 at 9:37

4 Answers 4

1

If you don't mind a bit of manual work, you can write your own converter and handle each datatype differently, for example:

public static class StructSerializer
{
    public static byte[] Serialize<T>(T data) where T : struct
    {
        List<byte> result = new List<byte>();
        Type type = data.GetType();
        IEnumerable<FieldInfo> orderedFields = type.GetFields().OrderBy(f => Marshal.OffsetOf(type, f.Name).ToInt32());

        foreach (FieldInfo fieldInfo in orderedFields)
        {
            object value = fieldInfo.GetValue(data);
            MethodInfo conversion = typeof(BitConverter).GetMethod(nameof(BitConverter.GetBytes), new[]{fieldInfo.FieldType});
            if (conversion == null) continue;
            byte[] converted = (byte[])conversion.Invoke(null, new []{value});
            result.AddRange(converted);
        }

        return result.ToArray();
    }
}

You didn't specifically mention it, but keep the ByteOrder in mind. You can check the byte order with BitConverter.IsLittleEndian.

Sign up to request clarification or add additional context in comments.

3 Comments

I'm not aware of any guarantees from the CLI that would make MetadataToken a sensible thing to order by, especially if you want it to keep working between builds
@MarcGravell Unfortunatly, yes. I just took this info from this answer, maybe I can find something in the reference source that would work a bit better.
ooh, that's gnarly; completely undocumented (or at least: informally documented via github), and only known to work in some cases (specifically: LayoutKind.Sequential), but indeed it looks like it might work, but... shudder!
0

Edit: I may have misunderstood the question; if your point is that you want the specific layout - then see the FieldOffset hints that Tim Rutter notes in their answer.


unsafe might be the easiest option:

using System;
using System.Runtime.InteropServices;

static class P
{
    private static unsafe void Main()
    {
        MyStruct orig = new MyStruct { a = 1, b = 2, c = 3, d = 4 };

        byte[] raw = new byte[sizeof(MyStruct)];
        // write the empty array to prove it is empty
        Console.WriteLine(BitConverter.ToString(raw));


        // serialize
        fixed (byte* p = raw)
        {
            var typed = (MyStruct*)p;
            *typed = orig;
        }

        // write the serialized data
        Console.WriteLine(BitConverter.ToString(raw));

        // deserialize
        MyStruct clone;
        fixed (byte* p = raw)
        {
            var typed = (MyStruct*)p;
            clone = *typed;
        }

        Console.WriteLine($"a = {clone.a}, b = {clone.b}, c = {clone.c}, d = {clone.d}");
    }
}

with:

[StructLayout(LayoutKind.Sequential)] // see Tim's answer for explicit layout
public struct MyStruct
{
    public ushort a;
    public uint b;
    public uint c;
    public ushort d;
}

However, it isn't very generic. If you need more generic code: the Unsafe type has many utility methods for this kind of thing.

5 Comments

That still has padding OP is trying to avoid
@Evk yes, that's why I added the edit to the top pointing to Tim's answer; however, I'm going to leave this here to illustrate how to avoid the Marshal usage
Why not just set Pack to 1?
@TimRutter because I addressed a tangential point - which I have acknowledged in the first line; you're already doing a great job of discussing the explicit/pack options in your answer, so there's no benefit in my duplicating that
Thats fine I thought I might have been missing something.
0

You can use Explicit layout to define exactly how you want the array to be formed:

    [StructLayout(LayoutKind.Explicit)]
    public struct MyStruct
    {
        [FieldOffset(0)]
        public ushort a;
        [FieldOffset(2)]
        public uint b;
        [FieldOffset(6)]
        public uint c;
        [FieldOffset(10)]
        public ushort d;
    }

But of course its not at all generic

5 Comments

not sure why this is downvoted - this very concisely solves the exact problem and gives the expected output
As for Pack, OP explicitly states in question that he cannot use Pack=1 (because of some project requirements). I guess he cannot use FieldOffset either then (though of course I didn't downvote).
@Evk 1 He didn't explicitly state that when I wrote my answer. Its a more recent edit. may still be a useful answer to someone in the future though.
@Tim I did not make any update to the question, I have not devoted it :)
@tulipe Ah sorry I missed that bit when I first read it then.
0

Thanks all for your valuable answers/comments! I have found a workaround I will remove the padding manually

        static byte[] getBytes<T>(T str) where T : struct
    {

        int size = Marshal.SizeOf(typeof(T));
        int Pack = str.GetType().StructLayoutAttribute.Pack;
        byte[] arr = new byte[size];
        IntPtr ptr = Marshal.AllocHGlobal(size);
        Marshal.StructureToPtr(str, ptr, true);
        Marshal.Copy(ptr, arr, 0, size);
        Marshal.FreeHGlobal(ptr);
        arr = RemovePadding<T>(arr.ToList(), str, str.GetType().StructLayoutAttribute.Pack);
        return arr;
    }
    static byte[] RemovePadding<T>(List<byte> buffer, T str, int Pack)
    {
        int largestsize = 0;
        int index = 0;
        int RowOfMemory = 0;

        //Get all fields 
        var fields = str.GetType().GetFields();

        //After MSDN
        //The alignment of the type is the size of its largest element (1, 2, 4, 8, etc., bytes) or the specified packing size, whichever is smaller.
        foreach (var item in fields)
        {
            if (Marshal.SizeOf(item.FieldType) > largestsize)
                largestsize = Marshal.SizeOf(item.FieldType);
        }
        if (largestsize < Pack) Pack = largestsize;

        //Find and remove padding from all memory rows 
        foreach (var item in fields)
        {
            int size = Marshal.SizeOf(item.FieldType);
            if (RowOfMemory != 0 && (RowOfMemory + size) > Pack)
            {
                int paddingsize = Math.Abs(Pack - RowOfMemory);
                buffer.RemoveRange(index, paddingsize);
                RowOfMemory = size % Pack;
            }
            else if ((RowOfMemory + size) < Pack)
            {
                RowOfMemory += size;
            }
            else if ((RowOfMemory + size) == Pack)
            {
                RowOfMemory = 0;
            }
            index += size;
        }
        if (RowOfMemory != 0)
        {
            int paddingsize = Math.Abs(Pack - RowOfMemory);
            buffer.RemoveRange(index, paddingsize);
        }
        return buffer.ToArray();
    }

I will not count this as a right answer, may be someone found a better solution.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.