Java client: TooLongFrameException: Response entity too large

Hi there!

We are using Fauna’s latest Java client 3.0.0 in a Kotlin 1.3 app that runs on AWS Lambda on JDK 11.

When a query produces a large-ish result (here apparently about 7 MB), we get the below exception upon calling get() on the result of FaunaClient.query. Has anybody else experienced this? Is there a way around it?

Cheers,

Felix

TooLongFrameException: Response entity too large: DefaultHttpResponse(decodeResult: success, version: HTTP/1.1)
HTTP/1.1 200 OK
Content-Type: application/json;charset=utf-8
Date: Thu, 10 Sep 2020 15:31:25 GMT
Vary: Accept-Encoding
X-Compute-Ops: 1357
X-FaunaDB-Build: 20.08.03.rc1-99601a7
X-Query-Bytes-In: 7249
X-Query-Bytes-Out: 7243363
X-Query-Time: 654
X-Read-Ops: 532
X-Storage-Bytes-Read: 0
X-Storage-Bytes-Write: 0
X-Txn-Retries: 0
X-Txn-Time: 1599751884798065
X-Write-Ops: 0
Content-Length: 7243363
Connection: keep-alive
at io.netty.handler.codec.http.HttpObjectAggregator.handleOversizedMessage(HttpObjectAggregator.java:276)
at io.netty.handler.codec.http.HttpObjectAggregator.handleOversizedMessage(HttpObjectAggregator.java:87)
at io.netty.handler.codec.MessageAggregator.invokeHandleOversizedMessage(MessageAggregator.java:404)
at io.netty.handler.codec.MessageAggregator.decode(MessageAggregator.java:254)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
...

(46 additional frame(s) were not displayed)

@Felix, I think we have a limit of 5MB on the content size. There is a reference to it here but does not explicitly mention the maximum limit. Will research more for specifics.

Hi Jay, thanks for your response. None of our documents is larger than a few hundred kB - the query in question retrieves more than one. According to the exception, the problem seems to be that the response’s actual content is longer than what is indicated in the Content-Length header (I guess it’s either that, or it’s a bug in Netty).

@Felix, X-Query-Bytes-Out: 7243363 and X-Read-Ops: 532 indicates that the database is returning 7MB and has executed 532 read ops. Restriction of 5MB is from the driver. Documents could be small but the query is resulting in 7MB response. Do you want to DM me your query, happy to look at it.

Okay, that’s good to know. Is the 5 MB limit on the client side? Is there a way to increase it? Or a way to Paginate based on number of bytes? The problem is that the size of our documents varies a lot, so we have no way of predicting the response size when submitting a query.

5MB limit is on the client side. Currently, there is no way to increase it. Discussing internally more on this.

Thanks, Jay, much appreciated. It’s a little unfortunate that the entire query gets executed, the response goes over the wire, we get billed for it, but we just can’t access the contents of the response. :wink: