CLR Memory Leak

On a recent client engagement I had to investigate what appeared a to be a memory leak in a managed application. The program was running for about a week now and it appeared to slowly degrade in performance over time. Although it appeared healthy at only about 245 MB of memory used, I decided to investigate. The fastest way in my opinion to track down leaks in a running environment is to attach Windbg and use SOS:

• Download the appropriate release of the system debugger. Its location on the Microsoft Download site changes every now and then, but right now it is located at:
  • http://www.microsoft.com/whdc/devtools/debugging/installx86.Mspx for 32 bit platforms
  • http://www.microsoft.com/whdc/devtools/debugging/install64bit.mspx for 64 bit platforms.
• Start Windbg and attach it to your application.
• Load SOS: .loadby sos mscorwks
• Dump the heap stats:!dumpheap -stat

The heap stats will dump all the types along with the number of objects allocated and the the total memory consumed, sorted by memory consumed. Usually the last entry in the list is type that is being leaked. In my case some 2 million objects were present so the leak was quite obvious.

One you tracked down the type being leaked it is usually very easy to find the cause and fix it just by a simple code analysis, but if you are having problems with that the solution is to find a few objects of the leaked type. Use: !dumpheap -type <LeakedTypeName> and then dump the tree references that prevents the Garbage Collector from reclaiming the object: !gcroot <AddressOfObject>

There is an excellent post on the subject from Rico Mariani that details the process: http://blogs.msdn.com/ricom/archive/2004/12/10/279612.aspx.

There are also various SOS cheat sheets out there, the one I found refreshingly simple and succinct is this one: http://geekswithblogs.net/.netonmymind/archive/2006/03/14/72262.aspx

In my case the leak cause was very obvious and was caused by not removing an event handler from an object event after the object was no longer needed. Unfortunately this pattern of leak is very frequent because of the default event handling code generated by pressing the Tab key in Visual Studio.

The following snippet of code illustrates the problem I’m talking about:

class Worker
        {
            public void Work(object state)
            {
                // do some work here ...
                OnFinished();
            }

            private void OnFinished()
            {
                if (null != Finished)
                {
                    Finished(this, EventArgs.Empty);
                }
            }

            public event EventHandler Finished;
        }

        static int RunningCount;
        static AutoResetEvent eventDone;

        static void Main(string[] args)
        {
            RunningCount = 10000;
            eventDone = new AutoResetEvent(false);
            for (int i = 0; i < 10000; ++i)
            {
                Worker leaked = new Worker();
                leaked.Finished += new EventHandler(leaked_Finished);
                ThreadPool.QueueUserWorkItem(new WaitCallback(leaked.Work));
            }
            eventDone.WaitOne();
        }

        static void leaked_Finished(object sender, EventArgs e)
        {
            int count = Interlocked.Decrement(ref RunningCount);
            if (0 == count)
            {
                eventDone.Set();
            }
        }

To correct the problem in my example you have to remove the handler from the Worker Finished event when it completes:

class Worker
        {
            public void Work(object state)
            {
                 // do some work here ...
                OnFinished();
            }

            private void OnFinished()
            {
                if (null != Finished)
                {
                    Finished(this, EventArgs.Empty);
                }
            }

            public event EventHandler Finished;
        }

        static int RunningCount;
        static AutoResetEvent eventDone;
        static EventHandler callbackFinished;

        static void Main(string[] args)
        {
            RunningCount = 10000;
            eventDone = new AutoResetEvent(false);
            callbackFinished = new EventHandler(leaked_Finished);
            for (int i = 0; i < 10000; ++i)
            {
                Worker leaked = new Worker();
                leaked.Finished += callbackFinished;
                ThreadPool.QueueUserWorkItem(new WaitCallback(leaked.Work));
            }
            eventDone.WaitOne();
        }

        static void leaked_Finished(object sender, EventArgs e)
        {
            Worker leaked = (Worker)sender;
            leaked.Finished -= callbackFinished;
            int count = Interlocked.Decrement(ref RunningCount);
            if (0 == count)
            {
                eventDone.Set();
            }
        }