TCP-connection becomes unusable if request handler does not read entire request with HTTP/1.1 chunked Transfer-Encoding

[fbieler]

Jetty version(s)
12.0.3

Jetty Environment
core

Java version/vendor
openjdk 21-ea 2023-09-19
OpenJDK Runtime Environment (build 21-ea+35-Ubuntu-1ubuntu122.04)
OpenJDK 64-Bit Server VM (build 21-ea+35-Ubuntu-1ubuntu122.04, mixed mode, sharing)

OS type/version
Linux

Description
If using HTTP/1.1 with Transfer-Encoding: chunked and the request handler does not read all chunks before responding the TCP-connection becomes unusable for subsequent requests.
Note that this may happen with jersey and spring-mvc: For e.g. JSON payloads, a response is sent as soon as the request payload was fully parsed. This means that for HTTP/1.1 chunked transfer-encoding a response may be sent without reading the zero-length terminating chunk.
Still, I'm not sure if the request-handlers in those frameworks should be considered incorrectly implemented. Please advise.

Excerpts from log:
After a correctly processed request (i.e. where the entire request was processed) I see:

17431 [qtp380894366-29] DEBUG org.eclipse.jetty.server.internal.HttpChannelState - recycling HttpChannelState@550a61ff{handling=null, handled=true, send=LAST_COMPLETE, completed=true, request=POST@46ce5ced http://localhost/ HTTP/1.1}
17431 [qtp380894366-29] DEBUG org.eclipse.jetty.http.HttpParser - reset HttpParser{s=END,0 of -1}
17431 [qtp380894366-29] DEBUG org.eclipse.jetty.http.HttpParser - END --> START

If only part of the request was processed I see:

17440 [qtp380894366-21] DEBUG org.eclipse.jetty.server.internal.HttpChannelState - recycling HttpChannelState@550a61ff{handling=null, handled=true, send=LAST_COMPLETE, completed=true, request=POST@1703360 http://localhost/ HTTP/1.1}
17441 [qtp380894366-21] DEBUG org.eclipse.jetty.http.HttpParser - close HttpParser{s=CHUNKED_CONTENT,7 of -1}
17441 [qtp380894366-21] DEBUG org.eclipse.jetty.http.HttpParser - CHUNKED_CONTENT --> CLOSE
[...]
17442 [qtp380894366-21] DEBUG org.eclipse.jetty.server.internal.HttpConnection - caught exception HttpConnection@446fa7af::SocketChannelEndPoint@3db1b99[{l=/127.0.0.1:8080,r=/127.0.0.1:59788,OPEN,fill=-,flush=-,to=3/30000}{io=0/0,kio=0,kro=1}]->[HttpConnection@446fa7af[p=HttpParser{s=CLOSE,7 of -1},g=HttpGenerator@27695942{s=START}]=>HttpChannelState@550a61ff{handling=null, handled=false, send=SENDING, completed=false, request=null}] HttpChannelState@550a61ff{handling=null, handled=false, send=SENDING, completed=false, request=null}
org.eclipse.jetty.io.RuntimeIOException: Parser is terminated
	at org.eclipse.jetty.server.internal.HttpConnection.parseRequestBuffer(HttpConnection.java:626)

Full log: server.log

How to reproduce?
I created a minimal test-case:
https://github.com/fbieler/jetty-test

[joakime]

If using HTTP/1.1 with Transfer-Encoding: chunked and the request handler does not read all chunks before responding the TCP-connection becomes unusable for subsequent requests.
Note that this may happen with jersey and spring-mvc: For e.g. JSON payloads, a response is sent as soon as the request payload was fully parsed. This means that for HTTP/1.1 chunked transfer-encoding a response may be sent without reading the zero-length terminating chunk.
Still, I'm not sure if the request-handlers in those frameworks should be considered incorrectly implemented. Please advise.

You seem to be describing asynchronous response generation.

This means the response has started to be sent before the request is fully read.
This is normal, and many projects utilize this behavior to optimize their time to first byte.

Knowing this wont help you though.

Your description of the problem is a very common scenario with spring / rest libraries that use JSON or XML.
Namely the JSON and XML parsers do not (by default) read to EOF.
On an HTTP connection you must read to EOF if you want to use the persistent connection.

Depending on your chosen JSON library you should look for the configuration options to read to EOF (and possibly throw an exception if more content appears after the close of the JSON document).

Example from FasterXML/jackson-jaxrs-providers#108

    public ObjectMapper restObjectMapper() {
        ObjectMapper restObjectMapper = new ObjectMapper();
        restObjectMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
        restObjectMapper.enable(DeserializationFeature.FAIL_ON_TRAILING_TOKENS); // <-- the key change
        return restObjectMapper;
    }

[sbordet]

Still, I'm not sure if the request-handlers in those frameworks should be considered incorrectly implemented. Please advise.

Yes it is incorrectly implemented.

See also:
#4117
FasterXML/jackson-jaxrs-providers#108

[gregw]

Just some extra info. If a handler fully generates the response and completes without reading all the content, then Jetty will make an attempt to read the content, so the connection can be reused. However, Jetty will not block doing so, thus it depends on the state of connection if it can be used again. Thus a good handler should always strive to read all the request content if it wants the connection to be reused.

[fbieler]

Thank you for your comments, especially to @joakime, the json-deserialization-config did work wonders.