Estimate project
← All posts

How to Cancel HTTP Requests on the JVM

This article looks at the problems caused by abandoned HTTP calls and reviews how JVM HTTP clients let you cancel requests in flight, which can save resources.

9 min read
ScalaJavaHTTPAsynchronousJVM

HTTP is among the main means of exposing applications to the outside world and of letting microservices within a system talk to each other. Sooner or later, nearly every backend service sends an HTTP request.

Sometimes, though, the outcome of an HTTP call stops mattering to us. There can be several reasons for that, for instance:

  • we might send a few HTTP requests in parallel and continue processing only the result of whichever responds first (a race)
  • a downstream HTTP request might take too long, so a timeout response is returned upstream or a default value (cached or computed) is used in its place

The usual way of handling HTTP calls we no longer care about is to just abandon them. They complete, but nobody uses the response. Before being abandoned, such a response might even be fully loaded into memory and parsed, e.g. into JSON – a waste of both memory and CPU. Is there a better approach?

As it turns out, there usually is. The majority of the JVM HTTP clients we examined let you cancel an HTTP request that is in progress. Let’s see how this works.

What does canceling an HTTP request mean?

Due to the way HTTP is designed, once a request has been sent, there is generally nothing we can do to stop the server from processing it. So, taking into account any side effects our request may have on the server (if it’s e.g. a POST), we must assume that the request was processed in full.

What, then, is the benefit of canceling it? Besides the server processing the request, there is also the step of sending the response back to the client, which matters most when the response is of non-trivial size. In addition, as noted earlier, the client may parse the response further, and that work should be skipped if the result is going to be thrown away.

So when we cancel an in-flight HTTP request, the aim is to avoid transmitting the response, as well as any parsing of it or loading it into the client’s memory.

Our test

Our test was simple: one endpoint and one request. Once the server receives the request, it waits 2 seconds and then returns a 100MB response. The client sends the request but cancels it after 1 second (by then, no response data should have been sent). The exact meaning of “canceling” depends on the specific client implementation in use.

We checked these clients: async-http-client, Java’s built-in HttpClient, OkHttp and http4s. To compare how they behave, we used three servers built on Jetty, Http4s and akka-http.

TCP: What happens when a request is canceled

On the TCP level, the sequence of events is identical no matter which client is used. For now, we’ll concentrate on HTTP/1.1.

Canceling the request makes the client close the underlying TCP connection on its end by sending a FIN packet. The server acknowledges it, yet the connection stays half-open (the server does not close its end yet).

In our test, the cancellation happens after 1 second, while the server begins sending data after 2 seconds. Once that moment comes and the server sends the first data frames, no acknowledgments arrive for them, because the socket on the client’s side is closed. The client replies with RST (reset) packets instead.

At this point, the server stops sending the rest of the response. In the test, typically around 32K of data got transmitted (rather than the full 100MB), which is an enormous saving.

Keep in mind that the server only “learns” that the socket is closed when it tries to send some response data. As we said in the introduction, server-side processing can’t be interrupted and normally runs to completion.

Below is a capture of a single test run recorded with Wireshark:

Once a request is canceled, the connection gets closed, no matter what the Keep-Alive headers say. Having to set up a new connection rather than reusing an already open one can hurt performance. So it may also be reasonable not to cancel requests at all and keep the connection open instead.

async-http-client

Below is a sample test run using async-http-client. Executing a call returns a custom Future implementation that has a cancel method (the code is in Scala, but you can write an equivalent in any JVM language):

import org.asynchttpclient.DefaultAsyncHttpClient
import org.slf4j.LoggerFactory
import sys.process._

object Run extends App {
  val log = LoggerFactory.getLogger(this.getClass)
  val client = new DefaultAsyncHttpClient()

  val p = "/Applications/Wireshark.app/Contents/MacOS/wireshark -i lo0 -k -a duration:10".run()
  Thread.sleep(3000) // wait for Wireshark to start

  log.info("Sending ...")
  val f = client.executeRequest(
      client.prepareGet("http://localhost:8080/wait").build())
  log.info("Sent ...")
  Thread.sleep(1000)

  f.cancel(true)

  log.info("Done.")
}

On top of that, we kick off a Wireshark capture at the start. The key call is, naturally, f.cancel(true). Any later attempt to .get the result throws an exception.

async-http-client also offers .abort(Throwable), but it appears to execute only some of the logic that runs on cancellation, so cancel looks like the better choice. On the TCP level, however, both behave the same.

Because the response data never arrives, the client doesn’t parse or otherwise process it.

HttpClient

With Java’s built-in HttpClient, the code looks almost the same. Sending a request with .sendAsync likewise returns a Future implementation that can be canceled.

Up to JDK 15, though, canceling does nothing. Starting with JDK 16, it behaves exactly as with async-http-client.

HttpClient using Loom’s virtual threads

Virtual threads are a preview feature in Java 19 and are meant to largely remove the need to work with asynchronous code and Futures. Is it possible to cancel a synchronous HTTP request that’s in progress? It turns out it works perfectly well. Tested on JDK20:

import org.slf4j.LoggerFactory
import java.net.URI
import java.net.http.HttpResponse.BodyHandlers
import java.net.http.{HttpClient, HttpRequest}
import scala.sys.process._

object Run extends App {
  val log = LoggerFactory.getLogger(this.getClass)
  val client = HttpClient.newHttpClient()

  val p = "/Applications/Wireshark.app/Contents/MacOS/wireshark -i lo0 -k -a duration:10".run()
  Thread.sleep(3000) // wait for Wireshark to start

  log.info("Starting virtual thread ...")

  val t = Thread.startVirtualThread(() => {
    log.info("Sending ...")
    val r = client.send(
      HttpRequest
          .newBuilder(new URI("http://localhost:8080/wait"))
            .GET().version(HttpClient.Version.HTTP_1_1)
              .build(),
      BodyHandlers.ofString()
    )
    log.info(s"Received, body length: ${r.body().length}")
  })

  Thread.sleep(1000)
  log.info("Interrupting ...")
  t.interrupt()

  log.info("Done.")
}

In this case, the key operation is t.interrupt(), which cancels the in-flight HTTP request with the effects described earlier.

OkHttp

With OkHttp, both the synchronous and the asynchronous way of sending a request behave much like what we described above. An asynchronous request gives us a Future-like value that can be canceled, while in the synchronous case we can interrupt the thread executing the request.

When an OkHttp request is interrupted, however, the thread’s interrupted flag stays set. As a result, any later blocking operation (e.g. inside a finally block) will throw an InterruptedException right away, which may not be what we want.

Http4s

To compare, let’s examine a request sent with a library from the cats-effect ecosystem. Cats-effect is a library for concurrent programming as well as an asynchronous runtime (available for Scala on the JVM, JS, and Native). It comes with its own concept of lightweight threads (called fibers) and calls for a somewhat different style of programming.

This is our client code:

import cats.effect._
import org.http4s.ember.client.EmberClientBuilder
import org.slf4j.LoggerFactory
import scala.concurrent.duration.DurationInt
import sys.process._

object Run extends IOApp {
  private val log = LoggerFactory.getLogger(this.getClass)

  override def run(args: List[String]): IO[ExitCode] = EmberClientBuilder
    .default[IO]
    .build
    .use { client =>
      for {
        _ <- IO("/Applications/Wireshark.app/Contents/MacOS/wireshark -i lo0 -k -a duration:10".run())
        _ <- IO.sleep(3.seconds)
        _ <- IO(log.info("Sending ..."))
        f <- client
          .expect[String]("http://localhost:8080/wait")
          .start
        _ <- IO(log.info("Sent ..."))
        _ <- IO.sleep(1.second)
        _ <- IO(log.info("Cancelling ..."))
        _ <- f.cancel
        _ <- IO(log.info("Done."))
      } yield ()
    }
    .as(ExitCode.Success)
}

Thanks to the for-comprehension, the nested effect descriptions are executed one after another. The .start operation begins running, in the background on a new fiber, the computation described by the effect it’s called on. In this case, that means the HTTP request is performed concurrently.

A fiber can be canceled with f.cancel – and this is exactly what we rely on to cancel the HTTP request.

Interrupting a (virtual) thread and canceling a fiber differ in several significant ways. One of them is that canceling a fiber back-pressures by default, meaning it waits until the fiber has finished (there may be, for instance, some finalizers). With a thread, you need to call .join() after .interrupt() to get the same result. This design choice lets cats-effect provide even stronger resource safety than the alternatives.

HTTP/2

Running the same test over HTTP/2 (supported by Java’s HttpClient and Jetty) reveals one key difference: once a request is canceled, the connection stays open. A RST_STREAM packet is sent instead, which closes just one of the streams multiplexed over that connection.

As a result, connections can still be reused, so we skip the cost of establishing a new connection when sending a request, while still avoiding the transmission of unneeded data.

Streaming

We additionally tested a scenario with an endpoint that streams data to the client piece by piece. In our test, this meant sending 1KB data fragments every 100 milliseconds over 3 seconds. As before, the client canceled the request after 1 second.

The outcome is much the same: the client got the data from the first second, then sent a FIN packet that closed the socket on its side. When the next data fragment arrives, the client answers with an RST.

For the server, we implemented streaming in two versions, with akka-http and http4s, since both libraries make it easy to define streaming endpoints. Once the client closes the connection, the stream finishes with an IOException: Broken pipe.

So in this scenario, the client-side cancellation reaches the server, which avoids the needless work of producing further stream elements.

Takeaways

Here is a summary of what our investigation revealed:

  • the http clients that support cancellation are async-http-client, Java’s built-in HttpClient (since JDK 16), OkHttp and http4s
  • the http clients that don’t support cancellation include Java’s built-in HttpClient (prior to JDK 16) and akka-http
  • with HTTP/1.1, canceling an HTTP request closes the connection. With HTTP/2, the connection stays open and can be reused for other requests
  • the server “learns” that the connection is closed only when it tries to send the first frames of the response, so some data is always transmitted
  • with project Loom (Virtual Threads), requests sent with HttpClient or OkHttp can be canceled by interrupting the thread they run on; OkHttp, however, leaves the interrupted flag set
  • when the server produces the response data incrementally, cancellation may propagate from the client and close the stream on the server side