NETCF: Count your bytes correctly

I got a stress related issue reported in which the code tried allocating a 5MB array and do some processing on it but that failed with OOM (Out of Memory). It also stated that there was way more than 5 MB available on the device and surely it’s some bug in the Execution Engine.

Here’s the code.

try{
    byte[] result;
    long GlobalFileSize = 5242660;  //5MB
    result = new byte[GlobalFileSize];
    string payload = Encoding.UTF8.GetString(result, 0, result.Length);
    System.Console.WriteLine("len " + payload.Length);
}
catch (Exception e)
{
    System.Console.WriteLine("Exception " + e); 
} 

The problem is that the user didn’t count his bytes well. The array is 5MB and it actually gets allocated correctly. The problem is with the memory complexity of the UTF8.GetString which allocates further memory based on it’s input. In this particular case the allocation pattern goes something like

  5MB  -- Byte Array allocation (as expected and works)

  5MB  -- Used by GetString call
 10MB  -- Used by GetString call
  5MB  -- Used by GetString call
 10MB  -- Used by GetString call

So GetString needed a whooping 30MB and the total allocation is around 35MB which was really not available.

Morale of the story: Count your bytes well, preferably through a tool like Remote Performance Monitor