Faulting appl/mod: w3wp.exe; ntdll.dll; TCP error code 10061
netbaspect
Posts: 6
I'm trying to profile a Windows Workflow Service application hosted locally on IIS (Windows 7), but when I start the profiling I consequently get the following errors:
First error
Second error
When profiling is off, everything works perfectly.
Info
First error
Faulting application name: w3wp.exe, version: 7.5.7601.17514, time stamp: 0x4ce7afa2 Faulting module name: ntdll.dll, version: 6.1.7601.17514, time stamp: 0x4ce7c8f9 Exception code: 0xc0000374 Fault offset: 0x00000000000c40f2 Faulting process id: 0x1d34 Faulting application start time: 0x01cd1d5b6266cdc5 Faulting application path: C:\Windows\system32\inetsrv\w3wp.exe Faulting module path: C:\Windows\SYSTEM32\ntdll.dll Report Id: 3f93e3da-894f-11e1-9c9a-0026187ad9e0
Second error
Timestamp: 18-4-2012 12:08:52 Message: HandlingInstanceID: c4a706f9-e6e7-4930-ae08-bb6eb1413ff5 An exception of type 'System.ServiceModel.EndpointNotFoundException' occurred and was caught. --------------------------------------------------------------------------------------------- 04/18/2012 14:08:52 Type : System.ServiceModel.EndpointNotFoundException, System.ServiceModel, Version=3.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 Message : Could not connect to http://localhost:81/CertigoEnterprise/Runtime/PolicyJobProcess/RenewalControllerJobProcess.svc. TCP error code 10061: No connection could be made because the target machine actively refused it 127.0.0.1:81. Source : mscorlib Help link : Data : System.Collections.ListDictionaryInternal TargetSite : Void HandleReturnMessage(System.Runtime.Remoting.Messaging.IMessage, System.Runtime.Remoting.Messaging.IMessage) Stack Trace : Server stack trace: at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream() at System.ServiceModel.Channels.HttpOutput.Send(TimeSpan timeout) at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.SendRequest(Message message, TimeSpan timeout) at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout) at System.ServiceModel.Channels.ClientReliableChannelBinder`1.RequestClientReliableChannelBinder`1.OnSend(TRequestChannel channel, Message message, TimeSpan timeout) at System.ServiceModel.Channels.ReliableChannelBinder`1.Send(Message message, TimeSpan timeout, MaskingMode maskingMode) at System.ServiceModel.Channels.ReliableChannelBinder`1.Send(Message message, TimeSpan timeout) at System.ServiceModel.Security.SecuritySessionClientSettings`1.ClientSecuritySessionChannel.SendCloseMessage(TimeSpan timeout) at System.ServiceModel.Security.SecuritySessionClientSettings`1.ClientSecuritySessionChannel.CloseOutputSession(TimeSpan timeout) at System.ServiceModel.Security.SecuritySessionClientSettings`1.SecurityRequestSessionChannel.CloseOutputSession(TimeSpan timeout) at System.ServiceModel.Security.SecuritySessionClientSettings`1.ClientSecuritySessionChannel.CloseSession(TimeSpan timeout, Boolean& wasAborted) at System.ServiceModel.Security.SecuritySessionClientSettings`1.ClientSecuritySessionChannel.OnClose(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Close(TimeSpan timeout) at System.ServiceModel.Channels.LayeredChannel`1.OnClose(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Close(TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.OnClose(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Close(TimeSpan timeout)
When profiling is off, everything works perfectly.
Info
-
Windows 7, x64, 4GB
Windows Firewall turned off
IIS Anonymous Authentication and App Pools run under NetworkService
ANTS Performance Profiler 6.2, 6.3
Method-level timings; only methods with source
Do not record SQL and file I/OI performance; Don't profile child processes
Original port; Manually specified ASP.NET account
Comments
Unfortunately resulting in the same errors.
- Windows application event log (shows pertinent .Net errors)
- ANTS log (Help->Open log folder)
We would probably need a full memory dump to work out what was happening at the time of the heap corruption and I guess, try to prevent it.
How do I create such a memory dump?
A colleague of mine has the same problem. We can browse our application. Browse the .svc. But when a call goes out to one of the Workflow based services.... BOOM!
We have profiled these services in the past... so it's interesting why this suddenly is happening.
ftp://support.red-gate.com/utilities/autodumper.zip
Extract all files
Open a command prompt and cd to the extracted files folder
cscript adplus.vbs -crash -pn w3wp.exe
Should make a huge .dmp file in a subfolder of adplus
If you can actually get this to work, we can set up a private FTP to upload the dump to.
Sometimes the cause of this is a corrupt PDB and sometimes assemblies that use app protection or obfucscation can cause it.
I'd wager you can profile fine on "least detailed" mode, because all of the things that try to analyze the assembly so it can get line-level timings will not be executed.
No dumps are created...
ADPlus Version: 6.03.002 ADPlus Version Date: 02/21/2006 OS Version: 6.1 OS Build: 7601 DateTimeStamp: Date_04-20-2012__Time_11-09-2828 ADPlus was run on: MSDEV68 Debuggers Installation Directory: C:\PD\Command\autodumper Command line arguments used were: -crash -p 1788 Script engine Version: 5.8.16385 Selected debugger: cdb.exe Output Directory: C:\PD\Command\autodumper\Crash_Mode__Date_04-20-2012__Time_11-09-2828 g_Debug_Log = True g_CheckFreeSpace = True ------ Current Actions for Exceptions ---- AccessViolation [av] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog InvalidHandle [ch] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog IllegalInstruction [ii] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog IntegerDivide [dz] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog FloatingDivide [c000008e] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog IntegerOverflow [iov] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog InvalidLockSequence [lsq] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog StackOverflow [sov] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog CPlusPlusEH [eh] return: GN GN 1st chance: Log;Time;Stack 2nd chance: Log;Time;Stack;FullDump;EventLog UnknownException [*] return: GN GN 1st chance: Log;Time;Stack 2nd chance: Log;Time;Stack;FullDump;EventLog NET_CLR [clr] return: GN GN 1st chance: VOID 2nd chance: Log;Time;Stack;FullDump;EventLog CONTRL_C_OR_Debug_Break [bpe] return: Q GN 1st chance: Log;Time;EventLog;Stacks;LoadedModules;MatchingSymbols;MiniDump 2nd chance: Wake_Debugger [wkd] return: Q GN 1st chance: Log;Time;EventLog;Stacks;LoadedModules;MatchingSymbols;MiniDump 2nd chance: DLL_Load [ld] return: GN GN 1st chance: Log 2nd chance: Log DLL_UnLoad [ud] return: GN GN 1st chance: Log 2nd chance: Log Process_Shut_Down [epr] return: VOID VOID 1st chance: Log;Time;EventLog;Stacks;FullDump 2nd chance: Application_hang [aph] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog Ctl_C_Console_app [cce] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog Data_misaligned [dm] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog Guard_page_violation [gp] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog In_page_IO_error [ip] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog Invalid_system_call [isc] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog Stack_buffer_overflow [sbo] return: GN GN 1st chance: Log;Time;Stack;MiniDump 2nd chance: Log;Time;Stack;FullDump;EventLog Starting to attach the debugger to each process Attaching to 1788 - W3WP.EXEThe least detailed profile does seem to create a more stable situation, but eventually, after a few minutes, the process randomly dies as well.