Friday, December 04, 2009

NETCF: Count your bytes correctly

Pirate

I got a stress related issue reported in which the code tried allocating a 5MB array and do some processing on it but that failed with OOM (Out of Memory). It also stated that there was way more than 5 MB available on the device and surely it’s some bug in the Execution Engine.

Here’s the code.

try{
byte[] result;
long GlobalFileSize = 5242660; //5MB
result = new byte[GlobalFileSize];
string payload = Encoding.UTF8.GetString(result, 0, result.Length);
System.Console.WriteLine("len " + payload.Length);
}
catch (Exception e)
{
System.Console.WriteLine("Exception " + e);
}

The problem is that the user didn’t count his bytes well. The array is 5MB and it actually gets allocated correctly. The problem is with the memory complexity of the UTF8.GetString which allocates further memory based on it’s input. In this particular case the allocation pattern goes something like

  5MB  -- Byte Array allocation (as expected and works)

5MB -- Used by GetString call
10MB -- Used by GetString call
5MB -- Used by GetString call
10MB -- Used by GetString call


So GetString needed a whooping 30MB and the total allocation is around 35MB which was really not available.


Morale of the story: Count your bytes well, preferably through a tool like Remote Performance Monitor

No comments: