Timeout

Timeout policy (v5.0 onwards)

Purpose

To ensure the caller never has to wait beyond the configured timeout.

To enforce a timeout on actions having no in-built timeout.

Premise: 'Don't wait forever'

Waiting forever (having no timeout) is a bad design strategy: it specifically leads to the blocking up of threads or connections (itself often a cause of further failure), during a faulting scenario.

Beyond a certain wait, success is unlikely.

Syntax

TimeoutPolicy timeoutPolicy = Policy
  .Timeout([int|TimeSpan|Func<TimeSpan> timeout]
           [, TimeoutStrategy.Optimistic|Pessimistic]
           [, Action<Context, TimeSpan, Task> onTimeout])

TimeoutPolicy timeoutPolicy = Policy
  .TimeoutAsync([int|TimeSpan|Func<TimeSpan> timeout]
                [, TimeoutStrategy.Optimistic|Pessimistic]
                [, Func<Context, TimeSpan, Task, Task> onTimeoutAsync])

Parameters:

• timeout: the time after which the execute delegate or func should be abandoned. Can be specified as an int (number of seconds), TimeSpan, or func returning a TimeSpan
• timeoutStrategy (optional): whether to time out optimistically or pessimistically (see below)
• onTimeout/Async (optional): an action to run when the policy times-out an executed delegate or func. The action is run before the TimeoutRejectedException (see below) is thrown.

Throws:

• TimeoutRejectedException, when an execution is abandoned due to timeout.

Operation

TimeoutPolicy supports optimistic and pessimistic timeout.

Optimistic timeout

TimeoutStrategy.Optimistic assumes that delegates you execute support co-operative cancellation (ie honor CancellationTokens), and that the delegates express that timeout by throwing OperationCanceledException (as is standard for most .NET library code).

If the code executed through the policy is your own code, the recommended pattern is to call cancellationToken.ThrowIfCancellationRequested() at suitable intervals in the cancellable work.

The policy combines a timing-out CancellationToken into any passed-in CancellationToken, and uses the fact that the executed delegate honors cancellation to achieve the timeout. You must use Execute/Async(...) (or similar) overloads taking a CancellationToken, and the executed delegate must honor the token passed in to the lambda expression:

Policy timeoutPolicy = Policy.TimeoutAsync(30, TimeoutStrategy.Optimistic);
HttpResponseMessage httpResponse = await timeoutPolicy
    .ExecuteAsync(
        async ct => await httpClient.GetAsync(requestEndpoint, ct), // Execute a delegate which responds to a CancellationToken input parameter.
        CancellationToken.None // CancellationToken.None here indicates you have no independent cancellation control you wish to add to the cancellation provided by TimeoutPolicy.
    );

You can also combine your own CancellationToken (perhaps to carry independent cancellation signalled by the user). For example:

CancellationTokenSource userCancellationSource = new CancellationTokenSource();
// userCancellationSource perhaps hooked up to the user clicking a 'cancel' button, or other independent cancellation

Policy timeoutPolicy = Policy.TimeoutAsync(30, TimeoutStrategy.Optimistic);

HttpResponseMessage httpResponse = await timeoutPolicy
    .ExecuteAsync(
        async ct => await httpClient.GetAsync(requestEndpoint, ct), 
        userCancellationSource.Token
        );
    // GetAsync(...) will be cancelled when either the timeout occurs, or userCancellationSource is signalled.

We recommend using optimistic timeout wherever possible, as it consumes less resource. Optimistic timeout is the default.

Pessimistic timeout

TimeoutStrategy.Pessimistic recognises that there are cases where you may need to execute delegates which have no in-built timeout, and do not honor cancellation.

TimeoutStrategy.Pessimistic is designed to allow you nonetheless to enforce a timeout in these cases, guaranteeing still returning to the caller on timeout.

What is meant by timeout in this case is that the caller 'walks away': stops waiting for the underlying delegate to complete. An underlying delegate which does not honour cancellation is not magically cancelled - see What happens to the timed-out delegate? below.

Additional notes: Pessimistic timeout for async executions

For asynchronous executions, the extra resource cost is marginal: no extra threads or executing Tasks involved.

Note that pessimistic timeout for async executions will not timeout purely synchronous delegates which happen to be labelled async. It expects that the executed async code conforms to the standard async pattern, returning a Task representing the continuing execution of that async work (for example when the executed delegate hits the first internal await statement).

Unit tests attempting to demonstrate async timeout policy against a Thread.Sleep(...) will fail due to the intentional design choice that async timeout policy is optimised for the majority well-behaved async case, not for the actually-synchronous edge case. To write unit-tests against async timeout policy, test with await Task.Delay(..., cancellationToken), not with Thread.Sleep(...). Detailed discussion and examples: #318; #340; #623.

Additional notes: Pessimistic timeout for sync executions

For synchronous executions, the ability of the calling thread to 'walk away' from an otherwise un-timeout-able action comes at a cost: to allow the current thread to walk away, the policy executes the user delegate as a Task on a ThreadPool thread.

Because of this cost we do not recommend pessimistic synchronous TimeoutPolicy in scenarios where the number of concurrent requests handled is potentially high or unbounded. In such high/unbounded scenarios, that cost (effectively doubling the number of threads used) may be very expensive.

We recommend pessimistic synchronous TimeoutPolicy in conjunction with explicitly limiting the parallelism of calls on that codepath. Options to control parallelism include:

• using Polly BulkheadPolicy (which is a parallism-throttle) upstream of the Timeout policy
• using a concurrency-limiting TaskScheduler upstream of the Timeut policy
• using a circuit-breaker policy upstream of the TimeoutPolicy, with the circuit-breaker configured to break if too many downstream calls are timing out. The prevents an excessive number of calls being put through to the downstream system (and blocking threads) when it is timing out
• any other in-built parallelism controls of the calling environment.

What happens to the timed-out delegate?

A key question with any timeout policy is what to do with the abandoned (timed-out) task.

Pessimistic timeout

Polly will not risk the state of your application by unilaterally terminating threads. Instead, for pessimistic executions, TimeoutPolicy captures and passes the abandoned execution to you as the Task parameter of the onTimeout/onTimeoutAsync delegate.

This prevents these tasks disappearing into the ether (with pessimistic executions, we are talking by definition about delegates over which we expect to have no control by cancellation token: they will continue executing until they either belatedly complete or fault).

The task property of onTimeout/Async allows you to clean up gracefully even after these otherwise ungovernable calls. When they eventually terminate, you can dispose resources, carry out other clean-up, and capture any exception the timed-out task may eventually raise (important, to prevent these manifesting as UnobservedTaskExceptions):

Policy.Timeout(30, TimeoutStrategy.Pessimistic, (context, timespan, task) => 
    {
        task.ContinueWith(t => { // ContinueWith important!: the abandoned task may very well still be executing, when the caller times out on waiting for it! 

            if (t.IsFaulted) 
            {
                logger.Error($"{context.PolicyKey} at {context.OperationKey}: execution timed out after {timespan.TotalSeconds} seconds, eventually terminated with: {t.Exception}.");
            }
            else if (t.IsCanceled)
            {
               // (If the executed delegates do not honour cancellation, this IsCanceled branch may never be hit.  It can be good practice however to include, in case a Policy configured with TimeoutStrategy.Pessimistic is used to execute a delegate honouring cancellation.)  
               logger.Error($"{context.PolicyKey} at {context.OperationKey}: execution timed out after {timespan.TotalSeconds} seconds, task cancelled.");
            }
            else
            {
               // extra logic (if desired) for tasks which complete, despite the caller having 'walked away' earlier due to timeout.
            }

            // Additionally, clean up any resources ...

        });
    });

Note: In the async case of the above code, do not code await task.ContinueWith(...), as that will make the onTimeoutAsync: delegate await the completion of the abandoned task which the policy has just walked away from ... causing the policy to wait the full execution time of task and defeating the timeout. The async case must just attach the continuation, TPL-style; not await it.

Optimistic timeout

For optimistic executions, it is assumed the CancellationToken will cause the timed-out execution to clean up (if necessary) and then terminate by throwing for cancellation, per standard co-operative cancellation semantics.

This terminates the timed-out task and expresses that termination back to the caller. There is no separate, continuing, walked-away-from task as there was in the pessimistic case. The Task parameter passed to onTimeout/onTimeoutAsync is therefore intentionally always null for optimistic timeout.

(Another way of understanding this is that in optimistic timeout, the time-governed work is executed on the caller's codepath and thus expresses the full detail of its termination directly back to the caller. If the cancellation of the timed-out work was additionally expressed to the onTimeout/onTimeoutAsync delegate this would cause it to be expressed in two places, leading to ambiguity about where to process or handle its termination.)

Further reading

For a good discussion on walking away from executions you cannot cancel (pessimistic timeout), see Stephen Toub on How do I cancel non-cancelable async operations?.

Configuration recommendations

Every action which could block a thread, or block waiting for a resource or response, should have a timeout. [Michael Nygard: Release It!].

Use optimistic timeout with HttpClient calls

Do not use TimeoutStrategy.Pessimistic with calls through HttpClient! All HttpClient calls exist in versions taking a CancellationToken, so co-operative timeout with TimeoutStrategy.Optimistic is possible.

Combining timeout with retries

• For a timeout-per-try, place a TimeoutPolicy inside a RetryPolicy with PolicyWrap
• For a timeout applying to an operation overall, including any retries (eg: up to N tries, but if the whole operation takes longer than one minute, time out), place a TimeoutPolicy outside a RetryPolicy with PolicyWrap.

More in ordering policies with PolicyWrap can be found here.

For the specific case of applying an overall timeout to all tries where a retry policy is applied as a DelegatingHandler within HttpClient (perhaps configured through HttpClientFactory), note that the HttpClient.Timeout property can/will also provide this overall timeout: see our HttpClientFactory doco for more detail.

Thread safety and policy reuse

Thread safety

The operation of TimeoutPolicy is thread-safe: multiple calls may safely be placed concurrently through a policy instance.

Policy reuse

TimeoutPolicy instances may be re-used across multiple call sites.

When reusing policies, use an OperationKey to distinguish different call-site usages within logging and metrics.