Explained: ASP.NET Formula for Reducing Contention

From Guidance Share

Jump to: navigation, search

- J.D. Meier, Srinath Vasireddy, Ashish Babbar, John Allen and Alex Mackman


Contents

Formula for Reducing Contention

The formula for reducing contention can give you a good empirical start for tuning the ASP.NET thread pool. Consider using the Microsoft product group-recommended settings that are shown in Table 6.1 if the following conditions are true:

  • You have available CPU.
  • Your application performs I/O bound operations such as calling a Web method or accessing the file system.
  • The ASP.NET Applications/Requests In Application Queue performance counter indicates that you have queued requests.


Table 6.1: Recommended Threading Settings for Reducing Contention

Configuration setting

Default value (.NET Framework 1.1)

Recommended value

maxconnection

2

12 * #CPUs

maxIoThreads

20

100

maxWorkerThreads

20

100

minFreeThreads

8

88 * #CPUs

minLocalRequestFreeThreads

4

76 * #CPUs


To address this issue, you need to configure the following items in the Machine.config file. Apply the recommended changes that are described in the following section, across the settings and not in isolation. For a detailed description of each of these settings, see "Thread Pool Attributes" in Chapter 17, "Tuning .NET Application Performance."

  • Set maxconnection to 12 * # of CPUs. This setting controls the maximum number of outgoing HTTP connections that you can initiate from a client. In this case, ASP.NET is the client. Set maxconnection to 12 * # of CPUs.

Set maxIoThreads to 100. This setting controls the maximum number of I/O threads in the .NET thread pool. This number is automatically multiplied by the number of available CPUs. Set maxloThreads to 100.

  • Set maxWorkerThreads to 100. This setting controls the maximum number of worker threads in the thread pool. This number is then automatically multiplied by the number of available CPUs. Set maxWorkerThreads to 100.
  • Set minFreeThreads to 88 * # of CPUs. This setting is used by the worker process to queue all the incoming requests if the number of available threads in the thread pool falls below the value for this setting. This setting effectively limits the number of requests that can run concurrently to maxWorkerThreads – minFreeThreads. Set minFreeThreads to 88 * # of CPUs. This limits the number of concurrent requests to 12 (assuming maxWorkerThreads is 100).
  • Set minLocalRequestFreeThreads to 76 * # of CPUs. This setting is used by the worker process to queue requests from localhost (where a Web application sends requests to a local Web service) if the number of available threads in the thread pool falls below this number. This setting is similar to minFreeThreads but it only applies to localhost requests from the local computer. Set minLocalRequestFreeThreads to 76 * # of CPUs.


Note The recommendations that are provided in this section are not rules. They are a starting point. Test to determine the appropriate settings for your scenario. If you move your application to a new computer, ensure that you recalculate and reconfigure the settings based on the number of CPUs in the new computer.


If your ASPX Web page makes multiple calls to Web services on a per-request basis, apply the recommendations.

The recommendation to limit the ASP.NET runtime to 12 threads for handling incoming requests is most applicable for quick-running operations. The limit also reduces the number of context switches. If your application makes long-running calls, first consider the design alternatives presented in the "Avoid Blocking on Long-Running Tasks" section. If the alternative designs cannot be applied in your scenario, start with 100 maxWorkerThreads, and keep the defaults for minFreeThreads. This ensures that requests are not serialized in this particular scenario. Next, if you see high CPU utilization and context-switching when you test your application, test by reducing maxWorkerThreads or by increasing minFreeThreads.

The following occurs if the formula has worked:

  • CPU utilization increases.
  • Throughput increases according to the ASP.NET Applications\Requests/Sec performance counter.
  • Requests in the application queue decrease according to the ASP.NET Applications/Requests In Application Queue performance counter.

If using the recommended settings does not improve your application performance, you may have a CPU bound scenario. By adding more threads you increase thread context switching. For more information, see "ASP.NET Tuning" in Chapter 17, "Tuning .NET Application Performance."


References

For more information, see Knowledge Base article 821268, "PRB: Contention, Poor Performance, and Deadlocks When You Make Web Service Requests from ASP.NET Applications," at http://support.microsoft.com/default.aspx?scid=kb;en-us;821268.


Consider minIoThreads and minWorkerThreads for Intermittent Burst Load

If you have burst load scenarios that are intermittent and short (0 to 10 minutes), then the thread pool may not have enough time to reach the optimal level of threads. The use of minIoThreads and minWorkerThreads allows you to configure a minimum number of worker and I/O threads for load conditions.

At the time of this writing, you need a supported fix to configure the settings. For more information, see the following Microsoft Knowledge Base articles:


Threading Explained

ASP.NET processes requests by using threads from the .NET thread pool. The thread pool maintains a pool of threads that have already incurred the thread initialization costs. Therefore, these threads are easy to reuse. The .NET thread pool is also self-tuning. It monitors CPU and other resource utilization, and it adds new threads or trims the thread pool size as needed. You should generally avoid creating threads manually to perform work. Instead, use threads from the thread pool. At the same time, it is important to ensure that your application does not perform lengthy blocking operations that could quickly lead to thread pool starvation and rejected HTTP requests.


Configuration Overview

Most ASP.NET tuning is performed by modifying configuration parameters in the system - wide Machine.config file and the application-specific Web.config file. Figure 17.10 shows an architectural view of ASP.NET and its relationship to several key configuration elements located in Machine.config.


image: ASPNETTuningConfig.gif


Figure 17.10: Mapping of the key configuration elements with the request processing cycle


You have a number of options for tuning ASP.NET applications, most of which involve tuning the settings in Machine.config. This configuration file has many sections, but the following sections are most critical to performance.


<processModel> The attributes on the <processModel> element apply to the ASP.NET worker process (aspnet_wp.exe) and to all applications being hosted in the worker process on an IIS 5 Web server. Many of the settings are tuned by default and do not require further changes. The default settings are as follows.


  <processModel enable="true" timeout="Infinite" idleTimeout="Infinite" 
             shutdownTimeout="0:00:05" requestLimit="Infinite" 
             requestQueueLimit="5000" restartQueueLimit="10" 
  memoryLimit="60" 
             webGarden="false" cpuMask="0xffffffff" userName="machine" 
             password="AutoGenerate" logLevel="Errors" 
             clientConnectedCheck="0:00:05" comAuthenticationLevel="Connect" 
             comImpersonationLevel="Impersonate" 
             responseDeadlockInterval="00:03:00" maxWorkerThreads="20" 
             maxIoThreads="20"/>


For a detailed description of each attribute, see "<processModel> Element" in the.NET Framework documentation at http://msdn.microsoft.com/library/en-us/cpgenref/html/gngrfProcessmodelSection.asp.


<httpRuntime> The <httpRuntime> element configures the ASP.NET runtime settings. You can specify these at the machine, site, application, and subdirectory levels. The default settings from Machine.config are as follows.


  <httpRuntime executionTimeout="90" maxRequestLength="4096" 
            useFullyQualifiedRedirectUrl="false" minFreeThreads="8" 
            minLocalRequestFreeThreads="4" appRequestQueueLimit="100" 
            enableVersionHeader="true"/> 

For a detailed description of each attribute, see "<httpRuntime> Element" in the .NET Framework documentation at http://msdn.microsoft.com/library/en-us/cpgenref/html/gngrfHttpRuntimeSection.asp.


Thread Pool Attributes

Figure 17.11 shows thread pool configuration options for ASP.NET in context.


Image: ASPNETThreadPoolAttributes.gif


Figure 17.11: ASP.NET thread pool configuration options


The following list describes key attributes (in <processModel> and <httpRuntime>) in the machine.config file related to ASP.NET ThreadPool.

The list also discusses the scenarios where each attribute applies:

  • maxconnection. If your application makes calls to a remote Web service and the requests are waiting for the call to complete, you can increase the CPU utilization and your application performance by changing the maxconnection attribute on the <ConnectionManagement> element in Machine.config. The default values are as follows.


  <connectionManagement>
    <add address="*" maxconnection="2"/>
  </connectionManagement>

Increasing maxconnection enables more calls to be executed concurrently to a remote Web service. This attribute does not affect local Web service calls. An increase in the number of concurrent calls causes an increase in the utilization of threads that are used to make the remote calls.

Increasing maxconnection also can lead to an increase in CPU utilization. This increase in CPU utilization is caused by the fact that more incoming requests can be processed by ASP.NET instead of having the incoming requests wait for their turn to call the Web service. You need to balance the maxconnection with the other attributes discussed in this list and the actual CPU utilization.

  • maxWorkerThreads and maxIoThreads. These attributes define the maximum number of worker and I/O threads that the ASP.NET worker process is allowed to create. These values do not reflect the actual number of threads that are created by the worker process. The maximum value for these attributes is 100 per processor.

As described earlier, if you increase maxconnection, you cause increased concurrent processing, which requires a greater number of worker and I/O threads. Consider the following guidelines when tuning these attributes:

    • Change these attributes only if your processor utilization is below the threshold limits defined by your application's performance objectives.
    • Avoid increasing these attributes if the requests are not waiting on an I/O call but are actually performing CPU-intensive work. Increasing these attributes in this circumstance can negatively affect performance because the already stressed processor now has to handle increased thread context switching.
    • If your application makes a short-running I/O call (for example, to a remote Web service), you might not need to increase these values because the calling threads are not blocked for an excessive period.
    • If your application makes a long-running I/O call and your system has idle CPU, you can safely consider increasing these attributes along with the other related attributes discussed in this section. If your system does not have idle CPU, then you probably should not increase these attributes.

If you have a Web service on the same server as your Web application, consider the following to decide when to increase the default values:

    • Increase these attributes only if your processor utilization is below the threshold limits defined by your application's performance objectives.
    • Avoid increasing these attributes if requests are not waiting on an I/O call but are performing CPU-intensive tasks. Increasing these attributes in this situation can negatively affect performance because the already stressed processor now has to handle increased thread switching.
    • maxconnection and minFreeThreads. These attributes do not have any effect in scenarios where you make only local Web services calls.
  • minFreeThreads. This attribute defines the number of threads that can be used for work other than processing incoming requests to the worker process. This attribute prevents the ASP.NET process from using a thread from the thread pool to handle a new HTTP request if this would mean that the free thread count drops below this limit. The attribute is specified on the <httpRuntime> element and has a default value of 8.

You can use this attribute to help prevent deadlocks by ensuring that a thread is available to handle callbacks from pending asynchronous requests. A deadlock can occur if all of the threads in the thread pool are currently in use handling incoming HTTP requests, and one or more of those requests are waiting for asynchronous callbacks. In this situation, there are no available threads to service the callback. You can set minFreeThreads to ensure that some free threads are available to process the callbacks.

Increasing minFreeThreads means that you reserve more threads to make remote calls. In all cases, you should ensure that maxWorkerThreads – minFreeThreads >=12. Twelve is the optimum number of threads that should be made available to the ASP.NET worker process to service requests. This value means that ASP.NET cannot execute more than twelve requests concurrently. This limit is based on a series of performance tests, which demonstrate that normally the worker process uses four of these threads. If the processor is fully utilized (greater than 95 percent utilization) and your application makes long-running calls, the worker process is likely to use all twelve threads.

You might want to increase the attribute from the default values in the following scenarios:

    • You have increased maxWorkerThreads and maxIoThreads.
    • You have increased maxconnection to service a greater number of back-end calls, and hence require more threads to be made available for this purpose.
    • You might need to consider changing this attribute whenever you make long running calls which block your execution. This is most beneficial when the work is not computationally expensive for the server where you make the changes.
  • minLocalRequestFreeThreads. For a Web application using the Web service located on the same computer, you should consider decreasing the value of minLocalRequestFreeThreads when you need to give priority to processing local calls. This attribute defines the minimum number of free threads that ASP.NET keeps available so that requests from localhost are not queued. Before processing a local request, the runtime checks to see if at least this minimum number of worker threads are available. If fewer threads are available, the request is queued.

The default setting is four, so if only three worker threads are available the local request is queued. When this value is decreased, ASP.NET starts to use threads more aggressively, resulting in less local queuing.

Note Requests from remote clients start to queue when the free threads in the thread pool fall below the value of minFreeThreads.

If you decrease the value of minLocalRequestFreeThreads value without changing the minFreeThreads attribute, you are effectively telling the worker process to give priority to completing calls from the local server.

Personal tools