Module org.eclipse.jetty.server

Package org.eclipse.jetty.server.handler

Class QoSHandler

java.lang.Object

org.eclipse.jetty.util.component.AbstractLifeCycle

org.eclipse.jetty.util.component.ContainerLifeCycle

org.eclipse.jetty.server.Handler.Abstract

org.eclipse.jetty.server.Handler.AbstractContainer

org.eclipse.jetty.server.Handler.Wrapper

org.eclipse.jetty.server.handler.ConditionalHandler

org.eclipse.jetty.server.handler.ConditionalHandler.Abstract

org.eclipse.jetty.server.handler.QoSHandler

All Implemented Interfaces:

Handler, Handler.Container, Handler.Singleton, Request.Handler, org.eclipse.jetty.util.component.Container, org.eclipse.jetty.util.component.Destroyable, org.eclipse.jetty.util.component.Dumpable, org.eclipse.jetty.util.component.Dumpable.DumpableContainer, org.eclipse.jetty.util.component.LifeCycle, org.eclipse.jetty.util.thread.Invocable

@ManagedObject
public class QoSHandler
extends ConditionalHandler.Abstract

A quality of service Handler that conditionally limits the number of concurrent requests, to provide more predictable end-user experience in case descendant Handlers have limited capacity.

This Handler limits the number of concurrent requests to the number configured via setMaxRequestCount(int). If more requests are received, they are suspended (that is, not forwarded to the child Handler) and stored in a priority queue.

The maximum number of suspended request can be set with setMaxSuspendedRequestCount(int) to avoid out of memory errors. When this limit is reached, the request will fail fast with status code 503 (not available).

Priorities are determined via getPriority(Request), that should return values between 0 (the lowest priority) and positive numbers, typically in the range 0-10.

When a request that is being processed completes, the suspended request that current has the highest priority is resumed.

This Handler is ideal to avoid contending on slow/limited resources such as a JDBC connection pool, avoiding the situation where all server threads blocked contending on the limited resource, therefore leaving threads free to process other requests that do not require access to the limited resource.

Requests are resumed in priority order, so that when the server is under load, and there are many requests suspended to be processed, high priority request are processed first. For example, load balancer "ping" requests may have the highest priority, followed by requests performed by admin users, etc. so that regardless of the load, "ping" and "admin" requests will always be able to access the web application.

Nested Class Summary

Nested classes/interfaces inherited from class org.eclipse.jetty.server.handler.ConditionalHandler

ConditionalHandler.Abstract, ConditionalHandler.ConnectorPredicate, ConditionalHandler.DontHandle, ConditionalHandler.ElseNext, ConditionalHandler.InetAddressPatternPredicate, ConditionalHandler.MethodPredicate, ConditionalHandler.PathSpecPredicate, ConditionalHandler.PredicateSet, ConditionalHandler.Reject, ConditionalHandler.SkipNext

Nested classes/interfaces inherited from class org.eclipse.jetty.server.Handler.Abstract

Handler.Abstract.NonBlocking

Nested classes/interfaces inherited from class org.eclipse.jetty.util.component.AbstractLifeCycle

org.eclipse.jetty.util.component.AbstractLifeCycle.AbstractLifeCycleListener, org.eclipse.jetty.util.component.AbstractLifeCycle.StopException

Nested classes/interfaces inherited from interface org.eclipse.jetty.util.component.Container

org.eclipse.jetty.util.component.Container.InheritedListener, org.eclipse.jetty.util.component.Container.Listener

Nested classes/interfaces inherited from interface org.eclipse.jetty.util.component.Dumpable

org.eclipse.jetty.util.component.Dumpable.DumpableContainer

Nested classes/interfaces inherited from interface org.eclipse.jetty.server.Handler

Handler.AbstractContainer, Handler.Collection, Handler.Container, Handler.Sequence, Handler.Singleton, Handler.Wrapper

Nested classes/interfaces inherited from interface org.eclipse.jetty.util.thread.Invocable

org.eclipse.jetty.util.thread.Invocable.Callable, org.eclipse.jetty.util.thread.Invocable.InvocationType, org.eclipse.jetty.util.thread.Invocable.ReadyTask, org.eclipse.jetty.util.thread.Invocable.Task

Nested classes/interfaces inherited from interface org.eclipse.jetty.util.component.LifeCycle

org.eclipse.jetty.util.component.LifeCycle.Listener

Nested classes/interfaces inherited from interface org.eclipse.jetty.server.Request.Handler

Request.Handler.AbortException

Field Summary

Fields inherited from class org.eclipse.jetty.util.component.AbstractLifeCycle

FAILED, STARTED, STARTING, STOPPED, STOPPING

Fields inherited from interface org.eclipse.jetty.util.component.Dumpable

KEY

Fields inherited from interface org.eclipse.jetty.util.thread.Invocable

__nonBlocking, NOOP

Constructor Summary

Constructors

Constructor	Description
QoSHandler()
QoSHandler(Handler handler)

Method Summary

Modifier and Type	Method	Description
protected void	doStart()
protected void	doStop()
protected void	failSuspended(Request request, Response response, org.eclipse.jetty.util.Callback callback, int status, Throwable failure)	Fails the given suspended request/response with the given error code and failure.
int	getMaxRequestCount()
Duration	getMaxSuspend()	Get the max duration of time a request may stay suspended.
int	getMaxSuspendedRequestCount()
protected int	getPriority(Request request)	Returns the priority of the given suspended request, a value greater than or equal to 0.
int	getSuspendedRequestCount()
boolean	onConditionsMet(Request request, Response response, org.eclipse.jetty.util.Callback callback)	Handle a request that has met the conditions.
protected boolean	onConditionsNotMet(Request request, Response response, org.eclipse.jetty.util.Callback callback)	This method is called when the request has not met the conditions and is not to be handled by this handler.
void	setMaxRequestCount(int maxRequests)	Sets the max number of concurrent requests.
void	setMaxSuspend(Duration maxSuspend)	Sets the max duration of time a request may stay suspended.
void	setMaxSuspendedRequestCount(int maxSuspendedRequests)	Sets the max number of suspended requests.

Methods inherited from class org.eclipse.jetty.server.handler.ConditionalHandler

clear, dump, exclude, exclude, exclude, excludeInetAddressPattern, excludeMethod, excludePath, from, from, handle, include, include, include, includeInetAddressPattern, includeMethod, includePath, nextHandler

Methods inherited from class org.eclipse.jetty.server.Handler.Wrapper

getHandler, getInvocationType, setHandler

Methods inherited from class org.eclipse.jetty.server.Handler.AbstractContainer

findContainerOf, getDescendant, getDescendants, isDynamic, setDynamic, setServer

Methods inherited from class org.eclipse.jetty.server.Handler.Abstract

destroy, getServer

Methods inherited from class org.eclipse.jetty.util.component.ContainerLifeCycle

addBean, addBean, addEventListener, addManaged, contains, dump, dump, dumpObjects, dumpStdErr, getBean, getBeans, getBeans, getContainedBeans, getContainedBeans, installBean, installBean, isAuto, isManaged, isUnmanaged, manage, removeBean, removeBeans, removeEventListener, setBeans, start, stop, unmanage, updateBean, updateBean, updateBeans, updateBeans

Methods inherited from class org.eclipse.jetty.util.component.AbstractLifeCycle

getEventListeners, getState, getState, isFailed, isRunning, isStarted, isStarting, isStopped, isStopping, setEventListeners, start, stop, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.eclipse.jetty.util.component.Container

getCachedBeans, getEventListeners

Methods inherited from interface org.eclipse.jetty.util.component.Destroyable

destroy

Methods inherited from interface org.eclipse.jetty.util.component.Dumpable

dumpSelf

Methods inherited from interface org.eclipse.jetty.util.component.Dumpable.DumpableContainer

isDumpable

Methods inherited from interface org.eclipse.jetty.server.Handler

getServer, setServer

Methods inherited from interface org.eclipse.jetty.server.Handler.Container

getContainer, getDescendant, getDescendants, getDescendants

Methods inherited from interface org.eclipse.jetty.server.Handler.Singleton

getHandlers, getTail, insertHandler, setHandler

Methods inherited from interface org.eclipse.jetty.util.component.LifeCycle

addEventListener, isFailed, isRunning, isStarted, isStarting, isStopped, isStopping, removeEventListener, start, stop

Constructor Details

QoSHandler

public QoSHandler()

QoSHandler

public QoSHandler(Handler handler)

Method Details

getMaxRequestCount

@ManagedAttribute(value="The maximum number of concurrent requests",
                  readonly=true)
public int getMaxRequestCount()

Returns:

the max number of concurrent requests

setMaxRequestCount

public void setMaxRequestCount(int maxRequests)

Sets the max number of concurrent requests.

A negative or zero value indicates to calculate a value based on heuristics, drawn from the number the size of the server thread pool and/or the number of CPU cores.

Parameters:

maxRequests - the max number of concurrent requests

getMaxSuspendedRequestCount

@ManagedAttribute(value="The maximum number of suspended requests",
                  readonly=true)
public int getMaxSuspendedRequestCount()

Returns:

the max number of suspended requests

setMaxSuspendedRequestCount

public void setMaxSuspendedRequestCount(int maxSuspendedRequests)

Sets the max number of suspended requests.

Once the max suspended request limit is reached, the request is failed with a HTTP status of 503 Service unavailable.

A negative value indicate an unlimited number of suspended requests.

Parameters:

maxSuspendedRequests - the max number of suspended requests

getMaxSuspend

public Duration getMaxSuspend()

Get the max duration of time a request may stay suspended.

Returns:

the max duration of time a request may stay suspended

setMaxSuspend

public void setMaxSuspend(Duration maxSuspend)

Sets the max duration of time a request may stay suspended.

Once the duration expires, the request is failed with an HTTP status of 503 Service Unavailable.

Duration.ZERO means that the request may stay suspended forever.

Parameters:

maxSuspend - the max duration of time a request may stay suspended

getSuspendedRequestCount

@ManagedAttribute("The number of suspended requests")
public int getSuspendedRequestCount()

doStart

protected void doStart()
                throws Exception

Overrides:

doStart in class ConditionalHandler

Throws:

Exception

doStop

protected void doStop()
               throws Exception

Overrides:

doStop in class Handler.Abstract

Throws:

Exception

onConditionsMet

public boolean onConditionsMet(Request request,
 Response response,
 org.eclipse.jetty.util.Callback callback)
                        throws Exception

Description copied from class: ConditionalHandler

Handle a request that has met the conditions. Typically, the implementation will provide optional handling and then call the ConditionalHandler.nextHandler(Request, Response, Callback) method to continue handling.

Specified by:

onConditionsMet in class ConditionalHandler

Parameters:

request - The request to handle

response - The response to generate

callback - The callback for completion

Returns:

True if this handler will complete the callback

Throws:

Exception - If there is a problem handling

See Also:

• Request.Handler.handle(Request, Response, Callback)

onConditionsNotMet

protected boolean onConditionsNotMet(Request request,
 Response response,
 org.eclipse.jetty.util.Callback callback)
                              throws Exception

Description copied from class: ConditionalHandler

This method is called when the request has not met the conditions and is not to be handled by this handler. Implementations may return false; send an error response; or handle the request differently.

Specified by:

onConditionsNotMet in class ConditionalHandler

Parameters:

request - The request to handle

response - The response to generate

callback - The callback for completion

Returns:

True if this handler will complete the callback

Throws:

Exception - If there is a problem handling

See Also:

• Request.Handler.handle(Request, Response, Callback)

getPriority

protected int getPriority(Request request)

Returns the priority of the given suspended request, a value greater than or equal to 0.

Priority 0 is the lowest priority.

The set of returned priorities should be stable over time, typically constrained in the range 0-10.

Parameters:

request - the suspended request to compute the priority for

Returns:

the priority of the given suspended request, a value >= 0

failSuspended

protected void failSuspended(Request request,
 Response response,
 org.eclipse.jetty.util.Callback callback,
 int status,
 Throwable failure)

Fails the given suspended request/response with the given error code and failure.

This method is called only for suspended requests, in case of timeout while suspended, or in case of failure when trying to handle a resumed request.

Parameters:

request - the request to fail

response - the response to fail

callback - the callback to complete

status - the failure status code

failure - the failure