w3wp crashes soon after profiling starts

mkoskinen

Hi, I'm an issue running ANTS on a production server. After about 15 mins of profiling, the worker processes dies, the trace ends, and a new w3wp starts. By adding some perfmon counters I notice that just before w3wp crashes requests start queueing up and none of them get processed. Eventually w3wp blows up with too many requests. Here are some details

- using ANTS performance profiler 9.1 in sampling profile mode by attaching to w3wp
- Additional profiler options are all disabled to make things as simple as possible
- profiling a single .net 4 web application
- IIS logs don't show any unusual requests coming in around the time w3wp dies

Here's the relevant ANTS Log:

18 Jun 2015 12:55:23.619 [49] INFO  RedGate.Profiler.Logic.Controllers.Session.Type.AttachRunner - Initializing attach-to-process session
18 Jun 2015 12:55:23.978 [49] INFO  RedGate.Profiler.Engine.Logger - ANTS Performance Profiler Engine has started.
18 Jun 2015 12:55:23.978 [49] INFO  RedGate.Profiler.Engine.Logger - Version: 9.1.0.224.
18 Jun 2015 12:55:23.978 [49] INFO  RedGate.Profiler.Engine.Startup.Basic.PipeConnection - Listening for connections on \.pipeANTSProfiler7.
18 Jun 2015 12:55:24.119 [46] INFO  RedGate.Profiler.Logic.Controllers.Session.Type.BaseRunner - Launching session
18 Jun 2015 12:55:24.134 [46] INFO  RedGate.Profiler.Logic.Controllers.Session.Type.BaseRunner - Options: IL timings turned off
18 Jun 2015 12:55:24.134 [46] INFO  RedGate.Profiler.Logic.Controllers.Session.Type.BaseRunner - Options: Sampling mode
18 Jun 2015 12:55:24.275 [81] INFO  RedGate.Profiler.Engine.Startup.Basic.Attacher - Starting to attach to process ID 0x6234
18 Jun 2015 12:55:24.321 [81] INFO  RedGate.Profiler.Engine.Startup.Basic.Attacher - Found runtime version v4.0.30319
18 Jun 2015 12:55:24.399 [Profiler pipe monitor] INFO  RedGate.Profiler.Engine.Startup.Basic.PipeConnection - Process ID 0x6234 'w3wp.exe' has connected.
18 Jun 2015 12:55:24.602 [Profiler pipe monitor] INFO  RedGate.Profiler.Engine.Startup.Performance.PerformanceOptionsEncoder - Process settings are: not storing source files in log, inlining is disabled, method-level timings, sampling, thread affinity, including functions without source, ignoring 'trivial' function calls, simplifying excessively complicated call graphs, .ignoring derived ASP.NET source code, loading pdbs from , not profiling SQL calls, not gathering SQL parameters, profiling HTTP calls, profiling async events, 
18 Jun 2015 12:55:24.618 [Profiler pipe monitor] INFO  RedGate.Profiler.Engine.Startup.Basic.PipeConnection - Suggesting that target write to C:ANTS Performance Profiler 9	mp5d94_8d277d4bcab1aa4_0
18 Jun 2015 12:55:24.774 [Profiler pipe monitor] INFO  RedGate.Profiler.Engine.Logs.ILog - Log file was generated by ANTS version 9.1.0.224.
18 Jun 2015 12:55:25.133 [Profiler pipe monitor] INFO  RedGate.Profiler.Engine.Reports.Basic.Profile - Processed 74 bytes from the log.
18 Jun 2015 12:55:25.164 [Profiler pipe monitor] INFO  RedGate.Profiler.Engine.Startup.Basic.PipeConnection - Connected and receiving data through C:ANTS Performance Profiler 9	mp5d94_8d277d4bcab1aa4_0.
18 Jun 2015 13:13:40.438 [Profiler pipe monitor] INFO  RedGate.Profiler.Engine.Startup.Basic.PipeConnection - Retrying read of 0-length message
18 Jun 2015 13:13:42.325 [Profiler pipe monitor] ERROR RedGate.Profiler.Engine.Startup.Basic.PipeConnection - Failed to retrieve message from core
18 Jun 2015 13:13:42.325 [Profiler pipe monitor] WARN  RedGate.Profiler.Engine.Startup.Basic.PipeConnection - Received a 0-length message
18 Jun 2015 13:13:45.506 [Profiler pipe monitor] INFO  RedGate.Profiler.Engine.Startup.Basic.PipeConnection - Retrying read of 0-length message
18 Jun 2015 13:13:47.417 [Profiler pipe monitor] ERROR RedGate.Profiler.Engine.Startup.Basic.PipeConnection - Failed to retrieve message from core
18 Jun 2015 13:13:47.417 [Profiler pipe monitor] WARN  RedGate.Profiler.Engine.Startup.Basic.PipeConnection - Received a 0-length message
18 Jun 2015 13:13:47.417 [Profiler pipe monitor] INFO  RedGate.Profiler.Engine.Startup.Basic.PipeConnection - Retrying read of 0-length message
18 Jun 2015 13:13:49.302 [Profiler pipe monitor] ERROR RedGate.Profiler.Engine.Startup.Basic.PipeConnection - Failed to retrieve message from core
18 Jun 2015 13:13:49.302 [Profiler pipe monitor] WARN  RedGate.Profiler.Engine.Startup.Basic.PipeConnection - Received a 0-length message
18 Jun 2015 13:13:50.875 [31] INFO  RedGate.Profiler.Engine.Reports.Basic.Profile - Profiler overhead calculated as 41 ticks when making a method call, and 51 ticks to the method prologue/epilogue.
18 Jun 2015 13:13:54.276 [31] INFO  RedGate.Profiler.Engine.Reports.Basic.Profile - Refreshed profiling data. 27754784 bytes, 27754710 new.
18 Jun 2015 13:13:54.307 [31] INFO  RedGate.Profiler.Engine.Reports.Basic.Profile - Processed 27754784 bytes
1 processes with 204 threads
68457 functions called by 207910 stack traces
0 hit count maps, 0 timing maps

110: 1 entries, 56 bytes
107: 1 entries, 34 bytes
108: 1 entries, 16 bytes
3: 32 entries, 48694 bytes
Function info: 190 entries, 18536762 bytes
105: 55 entries, 142688 bytes
104: 19 entries, 207922 bytes
Stack info: 189 entries, 1566210 bytes
109: 177 entries, 243950 bytes
101: 17749 entries, 567968 bytes
Timing data: 17454 entries, 850528 bytes
100: 80 entries, 27072 bytes
6: 273 entries, 5852652 bytes
18 Jun 2015 13:13:58.981 [Profiler pipe monitor] WARN  RedGate.Profiler.Engine.Startup.Basic.PipeConnection - Pipe has been closed; most likely the process being profiled has terminated.
18 Jun 2015 13:13:59.013 [Profiler pipe monitor] INFO  RedGate.Profiler.Engine.Startup.Basic.Pipe - Recycling pipe

Jessica R

Hello,

Thanks for your detailed post and apologies for not replying sooner!

Regarding this issue, can I please check--is there anything in Event Viewer (Windows log>Application) regarding the profiler or the web application when this crash happens? If there's no details there then we'll likely need to get a dump file to better understand what's going wrong--if this is the case, please let me know and I'll send over the details on gathering and sharing the dump file with us!

I'm also curious though, if it's possible for you to profile the site by starting it through the profiler instead of by attaching to process, can you please try this to see if it crashes the same way?

(If you were evaluating the tool, you can extend it for another two weeks by entering "i need more time" as the serial key to activate)