Causes and debugging methods of HttpException "Request timed out"

Problem Brief

inASP.NETIn the program, we often encounter error messages as"Request Timed Out"ofHttpException,The error message in the browser is shown below,

At the same time, a warning message will appear in the application log as follows

How to reproduce this anomaly?

How is the thread interrupted in the process of processing logic and throws a timeout exception?

How to debug this kind of exception?

This article we fromCLRSource code level to analyze the cause of the problem and the corresponding debugging method.

reappear question

First we try to reproduce the problem first, atASP.NETprogramweb.configOne of themexecutionTimeout'S setting, the default value is110Seconds, which means that the execution time of a request exceeds110In secondstimeoutResulting inHttpException "Request Timed Out". This setting can beIIS Configuration Editororweb.configMake changes.

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <appSettings />
    <connectionStrings />
    <system.web>
        <compilation>
        </compilation>        
        <authentication mode="Windows" />
        <httpRuntime executionTimeout="10" />        
    </system.web>

</configuration>

In order to quickly reproduce this problem we willexecutionTimeoutThe default value is changed to10second. Then create a request timeoutasp.netThe page is as follows.

<%@ Page Language="C#" AutoEventWireup="true" Inherits="System.Web.UI.Page" %>
<script runat="server">
    protected void Page_Load(object sender, EventArgs e)
    {
        System.Threading.Thread.Sleep(120000);
    }

</script>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
</head>
<body>
    <form id="form1" runat="server">
    <div>
    
    </div>
    </form>
</body>
</html>

Deploy the page onIISGo up, then request the page, wait10Seconds to25The problem reappeared in about seconds.

Principle of production

There is a noteworthy place here, why our page logic is executingSleepBut thrownHttpException, Who interrupted the execution thread, how was it interrupted?

To study this problem, we must first usewindbgLet's take a look at the call stack information where this exception occurs.

bywindbg attachTow3wpProcess and enter the command'sxe clr'captureclrfirst chance exception

CLR exception - code e0434f4d (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
KERNELBASE!RaiseException+0x68:
000007fb`bbb789cc 488b8c24c0000000 mov rcx,qword ptr [rsp+0C0h] ss:000000da`6bf0da20=38a1bff3b7640000

0:023> KL
Child-SP RetAddr Call Site
000000da`6bf0d960 000007fb`a0da98ef KERNELBASE!RaiseException+0x68
000000da`6bf0da40 000007fb`a0d03e07 mscorwks!RaiseTheExceptionInternalOnly+0x2ff
000000da`6bf0db30 000007fb`a0cc58cf mscorwks!Thread::HandleThreadAbort+0x19b
000000da`6bf0dbd0 000007fb`a0cbe815 mscorwks!Thread::HandleThreadInterrupt+0x7b
000000da`6bf0dc30 000007fb`a1280769 mscorwks!Thread::UserSleep+0x89
000000da`6bf0dc90 000007fb`415d09ee mscorwks!ThreadNative::Sleep+0xf9
000000da`6bf0de40 000007fb`41630f39 App_Web_kng1ridv!ASP.sleep120_aspx.Page_Load(System.Object, System.EventArgs)+0xe
…
000000da`6bf0fae0 000007fb`beb53501 KERNEL32!BaseThreadInitThunk+0x1a
000000da`6bf0fb10 00000000`00000000 ntdll!RtlUserThreadStart+0x21

On top of the stackfirst chanceexceptionforThreadAbortException。

0:023> !dso
OS Thread Id: 0x12e0 (23)
RSP/REG Object Name
000000da6bf0d968 000000da0aac3ea0 System.Threading.ThreadAbortException
000000da6bf0da30 000000da0aac3ea0 System.Threading.ThreadAbortException
000000da6bf0da48 000000da0aac3ea0 System.Threading.ThreadAbortException
000000da6bf0db30 000000da0aac3ea0 System.Threading.ThreadAbortException
000000da6bf0dc98 000000da0aa93c90 System.Web.HttpContext
000000da6bf0dca0 000000da0aac3498 System.ComponentModel.EventHandlerList
000000da6bf0dd70 000000da0aac3498 System.ComponentModel.EventHandlerList
000000da6bf0dd90 000000da0aa93c90 System.Web.HttpContext

We can see from this call stack that this exception comes fromThread::UserSleep -> Thread::HandleThreadInterrupt ->Thread::HandleThreadAbort

Since this call stack isCLRUnmanaged code, we need a copy.NETFrameworkCLRSource code for reference, you can download a copy to MicrosoftShare Source CLI. turn up\sscli20_20060311\sscli20\clr\src\vm\threads.cppThis source file, a series of packages encapsulated by this classCLRThread related properties and methods.

Because we calledSleepMethod, first lookSleepFunction implementation. Original thread callSleepAfter the method enters the system kernel, the kernel puts it into a waiting state, onceCLRSleepExreturnWAIT_IO_COMPLETIONAnd passm_State & TS_InterruptedTwo flags to judgeinterrupt, The system will start to executeHandleThreadInterruptMethod, and thenHandleThreadInterruptThrow in methodThreadAbortException。

// Implementation of Thread.Sleep().
void Thread::UserSleep(INT32 time)
{
    CONTRACTL {
        THROWS;
        GC_TRIGGERS;
    }
    CONTRACTL_END;

    INCONTRACT(_ASSERTE(!GetThread()->GCNoTrigger()));

    DWORD   res;

    GCX_PREEMP();

    // A word about ordering for Interrupt.  If someone tries to interrupt a thread
    // that's in the interruptible state, we queue an APC.  But if they try to interrupt
    // a thread that's not in the interruptible state, we just record that fact.  So
    // we have to set TS_Interruptible before we test to see whether someone wants to
    // interrupt us or else we have a race condition that causes us to skip the APC.
    FastInterlockOr((ULONG *) &m_State, TS_Interruptible);

    // If someone has interrupted us, we should not enter the wait.
    if (IsUserInterrupted())
    {
        HandleThreadInterrupt(FALSE);
    }

    ThreadStateHolder tsh(TRUE, TS_Interruptible | TS_Interrupted);

    FastInterlockAnd((ULONG *) &m_State, ~TS_Interrupted);

retry:

    res = ClrSleepEx (time, TRUE);

    if (res == WAIT_IO_COMPLETION)
    {
        // We could be woken by some spurious APC or an EE APC queued to
        // interrupt us. In the latter case the TS_Interrupted bit will be set
        // in the thread state bits. Otherwise we just go back to sleep again.
        if ((m_State & TS_Interrupted))
        {
            HandleThreadInterrupt(FALSE);
        }

        // Don't bother with accurate accounting here.  Just ensure we make progress.
        // Note that this is not us doing the APC.
        if (time == 0)
        {
            res = WAIT_TIMEOUT;
        }
        else
        {
            if (time != (INT32) INFINITE)
            {
                time--;
            }

            goto retry;
        }
    }
    _ASSERTE(res == WAIT_TIMEOUT || res == WAIT_OBJECT_0);
}

The next question is what makesClrSleepExDid the method return? According to the notes we can seeAPCProbably the reason for interrupting the waiting thread. Next we continue to passdebugVerify it. according tomsdnUser mode can be calledkernel32!QueueUserAPCMethod to send aAPCTo the specified thread. We set a breakpoint on this method and reproduce the problem. The call stack was caught as a result.

0:022> kL
Child-SP RetAddr Call Site
000000da`6bbfe808 000007fb`a10ce75b KERNEL32!QueueUserAPCStub
000000da`6bbfe810 000007fb`a0e37e88 mscorwks!Thread::Alert+0xdb
000000da`6bbfe880 000007fb`a103d0d4 mscorwks!Thread::UserInterrupt+0x30
000000da`6bbfe8b0 000007fb`a11f5699 mscorwks!Thread::UserAbort+0x3984d0
000000da`6bbfe9d0 000007fb`997c4519 mscorwks!ThreadNative::Abort+0x169
000000da`6bbfebd0 000007fb`988f835f mscorlib_ni!System.Threading.Thread.Abort(System.Object)+0x39
000000da`6bbfec20 000007fb`97e94547 System_Web_ni!System.Web.RequestTimeoutManager+RequestTimeoutEntry.TimeoutIfNeeded(System.DateTime)+0xa63d4f
000000da`6bbfec60 000007fb`99772bdb System_Web_ni!System.Web.RequestTimeoutManager.CancelTimedOutRequests(System.DateTime)+0x167
…
000000da`6bbff8a0 00000000`00000000 ntdll!RtlUserThreadStart+0x21

Look at the information on the stack, which thread is beingabort。

0:022> !dso
OS Thread Id: 0x14e4 (22)
RSP/REG Object Name
000000da6bbfe9c0 000000da0aa84400 System.Threading.Thread
000000da6bbfea28 000000d9cab1efe8 System.Web.HttpApplication+CancelModuleException
000000da6bbfea38 000000da0aa84400 System.Threading.Thread
000000da6bbfea88 000000d9caa4c810 System.String

0:022> !do 000000da0aa84400
Name: System.Threading.Thread
MethodTable: 000007fb998b86c0
EEClass: 000007fb994bea18
Size: 104(0x68) bytes
(C:\windows\assembly\GAC_64\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
Fields:
MT Field Offset Type VT Attr Value Name
000007fb998f01b0 4000690 8 ....Contexts.Context 0 instance 000000d9caa285c0 m_Context
000007fb998b2bd0 4000691 10 ....ExecutionContext 0 instance 000000da0aace570 m_ExecutionContext
000007fb998b7c20 4000692 18 System.String 0 instance 0000000000000000 m_Name
000007fb998b8438 4000693 20 System.Delegate 0 instance 0000000000000000 m_Delegate
000007fb998aefe0 4000694 28 System.Object[][] 0 instance 0000000000000000 m_ThreadStaticsBuckets
000007fb998bed40 4000695 30 System.Int32[] 0 instance 0000000000000000 m_ThreadStaticsBits
000007fb998c05a0 4000696 38 ...ation.CultureInfo 0 instance 0000000000000000 m_CurrentCulture
000007fb998c05a0 4000697 40 ...ation.CultureInfo 0 instance 0000000000000000 m_CurrentUICulture
000007fb998b7510 4000698 48 System.Object 0 instance 0000000000000000 m_ThreadStartArg
000007fb998ba628 4000699 50 System.IntPtr 1 instance da6b789d60 DONT_USE_InternalThread
000007fb998bee90 400069a 58 System.Int32 1 instance 2 m_Priority
000007fb998bee90 400069b 5c System.Int32 1 instance 7 m_ManagedThreadId
000007fb998ac8d8 400069c 2d8 ...LocalDataStoreMgr 0 shared static s_LocalDataStoreMgr
>> Domain:Value 000000d9c90512d0:0000000000000000 000000d9ca176610:0000000000000000 <<
000007fb998b7510 400069d 2e0 System.Object 0 shared static s_SyncObject
>> Domain:Value 000000d9c90512d0:000000d9caa23bf0 000000d9ca176610:000000da0aa2d410 <<

0:022> !threads
ThreadCount: 10
UnstartedThread: 0
BackgroundThread: 9
PendingThread: 0
DeadThread: 1
Hosted Runtime: no
PreEmptive Lock
ID OSID ThreadOBJ State GC GC Alloc Context Domain Count APT Exception
3 1 150c 000000d9ca125a90 8220 Enabled 0000000000000000:0000000000000000 000000d9c90512d0 0 Ukn
19 2 23c8 000000d9ca12f510 b220 Enabled 000000da0aa8a238:000000da0aa8c200 000000d9c90512d0 0 MTA (Finalizer)
20 3 1a98 000000d9ca174ab0 80a220 Enabled 0000000000000000:0000000000000000 000000d9c90512d0 0 MTA (Threadpool Completion Port)
21 4 23f4 000000d9ca175b60 1220 Enabled 0000000000000000:0000000000000000 000000d9c90512d0 0 Ukn
22 5 14e4 000000da6b779c50 180b220 Disabled 000000d9cab1f000:000000d9cab20b18 000000d9ca176610 1 MTA (Threadpool Worker)
XXXX 6 0 000000da6b77d9d0 10820 Enabled 0000000000000000:0000000000000000 000000d9c90512d0 0 Ukn
23 7 12e0 000000da6b789d60 3a0b221 Enabled 000000da0aafd860:000000da0aafe2a8 000000d9ca176610 1 MTA (Threadpool Worker)
15 8 2180 000000da6b78a330 880a220 Enabled 0000000000000000:0000000000000000 000000d9c90512d0 0 MTA (Threadpool Completion Port)
24 9 13a4 000000da6b7903a0 220 Enabled 000000d9caae2e48:000000d9caae4aa8 000000d9c90512d0 0 Ukn
25 a e5c 000000d9ca107180 220 Enabled 000000d9cab20c68:000000d9cab22b18 000000d9c90512d0 0 Ukn

As a result, we see23No. thread isabortThe goal.23What is thread No.? View23The thread call stack happens to be oursSleepMethod thread.

0:023> kL
Child-SP RetAddr Call Site
000000da`6bf0db08 000007fb`bbb611f2 ntdll!NtDelayExecution+0xa
000000da`6bf0db10 000007fb`a10c369d KERNELBASE!SleepEx+0xaa
000000da`6bf0dbb0 000007fb`a0cbe7fd mscorwks!EESleepEx+0x2d
000000da`6bf0dc30 000007fb`a1280769 mscorwks!Thread::UserSleep+0x71
000000da`6bf0dc90 000007fb`415d09ee mscorwks!ThreadNative::Sleep+0xf9
000000da`6bf0de40 000007fb`41630f39 App_Web_kng1ridv!ASP.sleep120_aspx.Page_Load(System.Object, System.EventArgs)+0xe
…
000000da`6bf0f750 000007fb`a0d1970a mscorwks!UnManagedPerAppDomainTPCount::DispatchWorkItem+0x157
000000da`6bf0f7f0 000007fb`a0dcf3a4 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x1ba
000000da`6bf0f890 000007fb`bc53167e mscorwks!Thread::intermediateThreadProc+0x78
000000da`6bf0fae0 000007fb`beb53501 KERNEL32!BaseThreadInitThunk+0x1a
000000da`6bf0fb10 00000000`00000000 ntdll!RtlUserThreadStart+0x21

At this point we can prove our idea, it turns outASP.NETStarted oneRequestTimeoutManagerTo check the timeout thread, and then sendAPCWay to interrupt the timeout thread to make it throwThreadAbortExceptionEnd wait, finallyThreadAbortExceptionBe packed intoHttpExceptionThrow, recorded in the log and displayed on the client.

Debugging method

In realityTimeoutNot necessarily in the threadSleepNot allThreadAbortExceptionAll representTimeoutProblem occurred. We still have to summarize the debugging methods for similar problems.

First, observetimeoutTime, is it100About seconds, whether it matches the value in the configuration(Note that this is not worse than a second, because the viewRequestTimeoutManagerLogic can find that he is every15Check the timeout thread in seconds, all exist15Errors around seconds.

Second, if it is not the reason why the timeout is set too small, it is necessary to catchdumpWay to see what operation the specific timeout request timed out. Because what we end up withThreadAbortExceptionThere is no call stack information.

How to get it rightdumpIt is a question worth studying. We can set the breakpoint at a certain point on the following call stack. Note that the closer the breakpoint is to the top of the stack, the more times this method may be calleddumpThe greater the possibility.

byILSpyViewCancelTimedOutRequestswithTimeoutIfNeededSource code, you can see that entering this method does not necessarily mean that a thread will beabort, So we have to grab from the bottom layer. The final conclusion is that if we want to studyTimeoutofThreadAbortException, We caninmscorlib_ni!System.Threading.Thread.AbortSet a breakpoint abovedump, And then look at the thread on the stack, and finally find that is beingabortThe thread and call stack information of the corresponding execution.

// System.Web.RequestTimeoutManager
privatevoidCancelTimedOutRequests(DateTime now)
{
if(Interlocked.CompareExchange(refthis._inProgressLock,1,0)!=0)
{
return;
}
ArrayList arrayList =newArrayList(this._requestCount);
for(inti =0;i<this._lists.Length;i++)
{
DoubleLinkList obj;
Monitor.Enter(obj =this._lists[i]);
try
{
DoubleLinkListEnumerator enumerator =this._lists[i].GetEnumerator();
while(enumerator.MoveNext())
{
arrayList.Add(enumerator.GetDoubleLink());
}
}
finally
{
Monitor.Exit(obj);
}
}
intcount = arrayList.Count;
for(intj =0;j<count;j++)
{
((RequestTimeoutManager.RequestTimeoutEntry)arrayList[j]).TimeoutIfNeeded(now);
}
Interlocked.Exchange(refthis._inProgressLock,0);
}

// System.Web.RequestTimeoutManager.RequestTimeoutEntry
internalvoidTimeoutIfNeeded(DateTime now)
{
Thread thread =this._context.MustTimeout(now);
if(thread!=null)
{
this.RemoveFromList();
thread.Abort(newHttpApplication.CancelModuleException(true));
}
}

Intelligent Recommendation

The declared package &quot;com.how2java.j2se&quot; does not match the expected package &quot;&quot;

The declared package "com.how2java.j2se" does not match the expected package "" I encountered this error, because the first written document, after loading the server caused it, de...

Java method quot

Table of contents 1. Method quot 2. Method quoting symbol Third, the instance method of the reference object Fourth, instance method of reference class Fifth, reference the constructor 1. Method quot ...

The applet quot Vant weapp

Official website introduction: https://vant-contrib.gitee.io/vant-weapp/quickstart Be sure to follow the steps! Intersection 1. Install NPM package Perform the command to install the NPM package in th...

The second development control and quot

Direction of first, secondary development 1.1 Add class library 1.2 Add components in the class library 1.3 Add Form to reference and inherit the Text 1.4 Add a ErrorProvider control Write custom logi...

6. Method quot

6. Method quot When we use Lambda, if there is only one method in the method, there is only one method (including the constructed method), and we can use the method to reference to further simplify th...