SageMakerGatewayRuntime / Client / exceptions / GatewayTimeoutException
GatewayTimeoutException¶
- class SageMakerGatewayRuntime.Client.exceptions.GatewayTimeoutException¶
504 – The inference gateway did not respond in time. The
Messageprefix says whose time limit expired:Global Inference Gateway: timed out waiting for the inference gateway– the gateway’s limit on the wait for the first part of the response.inference gateway: <status> <message>– the inference gateway serving the model returned a 504, or a 408 because it timed out waiting for the request to arrive. Its status code is in the message (the response status is 504), and up to 256 bytes of its error message are quoted.
The inference gateway names a component in the prefix when it reports one:
inference gateway (<component>):, where the component isconfig(the inference gateway’s configuration and proxy layer),scheduler(its request scheduler),model selector(its model selector) ormodel server(the model server).Not modeled as retryable: a timeout does not prove the inference did not run, and the gateway may already have tried another inference gateway. Standard SDK retry modes may still retry a 5xx status, which could run (and bill) the inference again; set max attempts to 1 if a repeated inference is unacceptable.
Example
try: ... except client.exceptions.GatewayTimeoutException as e: print(e.response)
- response¶
The parsed error response. All exceptions have a top level
Errorkey that provides normalized access to common exception atrributes. All other keys are specific to this service or exception class.Syntax
{ 'Message': 'string', 'Error': { 'Code': 'string', 'Message': 'string' } }
Structure
(dict) –
504 – The inference gateway did not respond in time. The
Messageprefix says whose time limit expired:Global Inference Gateway: timed out waiting for the inference gateway– the gateway’s limit on the wait for the first part of the response.inference gateway: <status> <message>– the inference gateway serving the model returned a 504, or a 408 because it timed out waiting for the request to arrive. Its status code is in the message (the response status is 504), and up to 256 bytes of its error message are quoted.
The inference gateway names a component in the prefix when it reports one:
inference gateway (<component>):, where the component isconfig(the inference gateway’s configuration and proxy layer),scheduler(its request scheduler),model selector(its model selector) ormodel server(the model server).Not modeled as retryable: a timeout does not prove the inference did not run, and the gateway may already have tried another inference gateway. Standard SDK retry modes may still retry a 5xx status, which could run (and bill) the inference again; set max attempts to 1 if a repeated inference is unacceptable.
Message (string) –
A description of the error. It starts with a prefix such as
Global Inference Gateway:orinference gateway:that says where the error came from.Error (dict) – Normalized access to common exception attributes.
Code (string) – An identifier specifying the exception type.
Message (string) – A descriptive message explaining why the exception occured.