Fast serialization from .NET to non .NET applications.

In .NET, in case you need perform data exchange between .NET and non .NET applications, the only built in way do do this, is to serialize data using SOAP serialization. However, creating and sending XML is too slow for time critical applications. Therefore you need to find some alternate solution to this problem. No matter to where you sanding data (network or whatever…), somehow you need to create byte buffer, populated with your data. And, for best performance you need to do this as fast as possible. I aware about 2 ways to create byte array with data populated into it:

• Create MemoryStream and BinaryWriter. Write into steam all data, and at the end call MemoryStream.ToArray().
• Use BitConverter class for each value type you have. Use Encoding.GetBytes for strings. Combine all together using Buffer.BlockCopy().

I presenting here my way, which is faster 10-20 times than first solution and faster 3-5 times than second one. It is very simple: I creating buffer in advance with desired size (i calculating the whole size before serialization). Creating pointer to it, and using cast to copy values inside. After each copy, I moving pointer forward for the amount of copied data. For example, if I copied boolean value, I should move pointer 4 positions forward, because of the size of boolean type (4 bytes). The same for each type, consult MSDN if you need to know size of each type.

Example – serialization of double value:

I know, that the size of double is 8 bytes. Therefore, I creating buffer in same size:

byte[] buffer = new byte[8];

fixed(byte* pBuffer = buffer) //must use fixed keyword – avoids GC move buffer in memory

{

byte* pBufferWriter = pBuffer; //must create another pointer, because pBuffer is readonly

*((double*)pBufferWriter) = yourDecimalValue; //copy value
pBufferWriter+=8; //move pointer forward 8 bytes.

}

Example – deserialization of double value:

…. //take here pointer pBufferReader to byte[] with data

double result = *((double*)pBufferReader);
pBufferReader+=8;

Same is for any value primitive(int, short, bool. etc..).

The real challenge is serialization of strings. To serialize string, first step is to take pointer to it.

fixed(char* pStr = yourStringValue)

from this point, you just need to copy all bytes from pStr to your buffer. Remember, that .NET uses Unicode, and each char is represented by 2 bytes. I suggesting here function I using to copy strings – its performance is much better that coping byte by byte:

public sealed class SerializationHelper
{
/// <summary>
/// The StringCopy optimized to copy chars.
/// </summary>
/// <param name=”dmem”>Pointer to destination</param>
/// <param name=”smem”>Pointer to source.</param>
/// <param name=”charCount”>Count of chars to copy.</param>
public static unsafe void StringCopy(char* dmem, char* smem, int charCount)
{
if (charCount > 0)
{
if ((((int)dmem) & 2) != 0)
{
dmem[0] = smem[0];
dmem++;
smem++;
charCount–;
}
while (charCount >= 8)  //it is eight here, not smile )
{
*((int*)dmem) = *((int*)smem);
*((int*)(dmem + 2)) = *((int*)(smem + 2));
*((int*)(dmem + 4)) = *((int*)(smem + 4));
*((int*)(dmem + 6)) = *((int*)(smem + 6));
dmem += 8;
smem += 8;
charCount -= 8;
}
if ((charCount & 4) != 0)
{
*((int*)dmem) = *((int*)smem);
*((int*)(dmem + 2)) = *((int*)(smem + 2));
dmem += 4;
smem += 4;
}
if ((charCount & 2) != 0)
{
*((int*)dmem) = *((int*)smem);
dmem += 2;
smem += 2;
}
if ((charCount & 1) != 0)
{
dmem[0] = smem[0];
}
}
}

}

After you finishing copying, don’t forget to advance buffer pointer .

Deserialization of string very simple:

string myString = new string((char*)pBuffer); // pBuffer is pointer to your buffer

Suppose, you have need copy block of memory to your buffer (maybe it is internal class, serialized before) – you can use the following method:

public static unsafe void ByteCopy(byte* ps, byte* pd, int count)
{
// Loop over the count in blocks of 4 bytes, copying an
// integer (4 bytes) at a time:
for (int n = 0; n < count / 4; n++)
{
*((int*)pd) = *((int*)ps);
pd += 4;
ps += 4;
}

// Complete the copy by moving any bytes that weren’t
// moved in blocks of 4:
for (int n = 0; n < count % 4; n++)
{
*pd = *ps;
pd++;
ps++;
}
}

As you can test in your applications, the performance is awesome. However, performance always must cost something. In this case, it costs readability of code and errors prone. It is very easy to do mistake here, for example forwarding pointer too many or too less. You must be very careful using this, and double check every line of code. You may ask: why not create helper functions to encapsulate serialization of every type. The answer is performance penalty. In my tests, calling to function works 2 times slower than performing the same code “on the fly”. However, I leave freedom to you to choose what is better for your needs. By the way, calling static function is faster than calling instance one.