analyze time spent in garbage collector

springy

Can Performance Profiler help me in finding out where in my code Gargabe Collection occurs the most time?

There exist 3 performance counters "# Gen X Collections" but they are just increasing steadily -- associating their delta changes with source code would be an interresting thing, wouldn't it? (At least as long GC's are not concurrent.)

Brian Donahue

I don't think it's worthwhile to try to tie GC collections down to code. If you run that code on a different system I'd imagine your GC profile could change completely.

What you can control, possibly, is the rate of allocations of objects. The GC algorithm works independently of the code being run.

springy

Sorry for being not precise enough:

I'm speaking of routines/methods which get called some million times and where it's very likely that short living (=Gen0 and Gen1) memory is allocated and freeded at recurring places -- which I want to isolate.

What I've read about your alpha/beta programm "Memory Tracker" seems to be which I'm looking for. Unfortunately it doesn't work -- either it's not compatible with .NET 4 / 64bit or something different.

Brian Donahue

I'm not sure what you want to accompilsh, but if you think memory tracker would do the job, you would probably get what you need with WinDbg and SOS using !dumpheap -stat.

springy

My goal is to find the regions where problems are. So !dumpheap could only help when I know where to stop and debug this using SOS.

ANTS Memory Profiler does not help, too, since there are no memory leaks -- but there is heavy GC usage which I want to reduce.

The problem is that there are many hidden places where objects could be created, for example when using LINQ with lambdas which create closures. This is happening behind the scenes, there is no "new SomeObject()" constructor call you easily could spot.

Brian Donahue

Just curious to know how Memory Tracker would have helped - it was based on StrikeControl and basically made a graphoc display of what you could get in a slightly less-attractive fashion using !dumpheap.

Guess you could use Windows performance monitor and set up an alert when the app's % time spent in GC gets to a certain level and have this trigger a dumpheap and a stack dump with CDB. I've never done this but it sounds possible.

springy

I don't know if it would have helped. But from the choice of the 2 ANTS Profilers and the Memory tracker the last one was most promising.

Since Memory Profiler already forces non-concurrent garbage collections maybe changes are higher in finding or at least narrowing down problem areas with this tool?

springy

Another sample of hidden object creation, a LINQ query using "let" (against a LINQ-to-SQL AdventureWorks DB):

from h in this.TransactionHistory.AsEnumerable()
let x = h.ActualCost / h.Quantity
where x > 0
let y = char.GetUnicodeCategory(h.TransactionType)
orderby h.TransactionDate, y
select h

This gets translated to ILASM code whose C# translation looks like this:

   .Select (
      h => 
         new  
         {
            h = h, 
            x = (h.ActualCost . (Decimal)(h.Quantity))
         }
   )
   .Where (temp0 => (temp0.x > 0))
   .Select (
      temp0 => 
         new  
         {
            temp0 = temp0, 
            y = Char.GetUnicodeCategory (temp0.h.TransactionType)
         }
   )
   .OrderBy (temp1 => temp1.temp0.h.TransactionDate)
   .ThenBy (temp1 => temp1.y)
   .Select (temp1 => temp1.temp0.h)

So although I'm using only a well known object (of type TransactionHistory) there have been created 2 additional anonymous (and temporary) classes for each instance of TransactionHistory I`m retrieving from the DB.
While the temporary object most likely won't get promoted to Gen1 this still puts a load on GC collections which would be avoidable at all by refactoring the code into something smarter.

springy

I see, Red Gate has no motivation in improving their products.

Chris.Allen

Hi- I think your original question is good and interesting (and it sparked off a lot of internal discussion). I'm by no means an expert on garbage collection but I do know ANTS performance profiler.

Have you tried including all the memory counters including the Gen 0/Gen 1 promoted (bytes/sec) counters - these are most useful as they give rates of activity rather than absolute values? You should then be able to correlate GC activity with your code (although, as Brian points out- it is an indirect correlation).

springy

I started the original post with these counters -- at least the pure counting ones.

Even though it's a performance problem maybe it's easier to tackle this problem using Memory profiler since Memory profiler already disabled concurrent garbage collections.

My main problem is that I see the counters increasing rapidly but don't have a clue where and why so many objects get created -- at least they don't get leaked. Most likely the main cause are these hidden LINQ closures, iterator implementation classes and hidden classes created by LINQ expressions.