Content negotiation, hands-on
Content Negotiation, specified in RFC-9110, is an essential mechanism that lets servers and user agents jointly choose the best-fitting representation of a resource. This article examines its specification, implements it with Spring Boot, and checks how closely Spring Boot follows the RFC standards.
Content negotiation stands out as a key mechanism for making web APIs more accessible. This powerful technique gives API integrators the freedom to choose the data representation that suits them best, while at the same time letting API providers serve a wide variety of integration needs. The process relies on standardized procedures, described at length in RFC-9110, which provides a solid basis for implementing it.
The article has two parts. The first concentrates on the theory of Content Negotiation and looks at its patterns and mechanisms. The second presents real-life examples of content negotiation using the Spring Boot framework.
The theory behind Content Negotiation
As noted earlier, content negotiation exists so that the server and the user agent can agree on a format for request and response content that works for both. The process is standardized in RFC-9110, which describes three separate patterns.
Two patterns for response content
- Proactive content negotiation (server driven negotiation) Here, it’s the server’s job to choose the most suitable representation, based on what it knows about the user agent. That knowledge can come from explicit content negotiation headers (Accept, Accept-Encoding, Accept-Charset, Accept-Language) or from implicit traits, such as parts of the User-Agent header or the network address.
- Reactive content negotiation (agent driven negotiation) In this approach, the user agent picks the best-matching representation once it has received an initial response. Rather than sending an initial representation, the server can return a list of available representations with response code 300 (multiple choices) or 406 (not acceptable). The user agent then uses this information to send another request for the chosen representation.
One pattern for request content
- Request content negotiation The server can signal which format it prefers for request content. To do that, it may add negotiation headers to its response, which the user agent can then reuse in later requests.
Content negotiation headers
The content negotiation standard defines the following headers:
- Accept: This header expresses preferences regarding the media type of message content. In a request, it states the user agent’s preferences for the response content. In a response, it states the server’s preferences for request content in later requests.
- Accept-Charset: The RFC guidelines mark it as deprecated, and it should be avoided where possible; before being deprecated, though, it served to express charset preferences. A wildcard can stand for any charset not listed elsewhere in the field.
- Accept-Encoding: This header is meant for expressing content coding preferences. It lets user agents say which content codings they accept in the response, and it also lets servers state the content coding they prefer for later requests. The identity token means the same as no encoding.
- Accept-Language: lets user agents state which natural languages they prefer in responses. These preferences are expressed with language tags, so users can rank languages. Language tags may carry quality values that reflect how strongly the user prefers them.
- Vary: a response header that lists which parts of a request, besides the method and target, influence how the server chooses the response representation. It can be the wildcard * or a list of request field names that may have contributed to choosing the response representation.
We’ll open this part with a brief summary of the rules to follow when using or processing content negotiation headers.
General rules
- If a content negotiation header is absent from the request, the sender has no preferences in that dimension of negotiation.
- Each content negotiation field may carry several preferences.
- Preferences can be ranked using quality values
“q”from the range0..1, with up to 3 digits after the decimal point, where 1 means most preferred,0.001least preferred, and 0 not accepted. - A quality value follows the preference it applies to and is separated from it by a semicolon (e.g.,
application/json;q=0.8) - When no quality value is given, it is assumed to be 1
- In most content negotiation headers, a wildcard, where allowed, can denote an unspecified value (e.g.
Accept: text/*denotes any text mime type).
Implementing media type selection via content negotiation
To show how content negotiation can work, we’ll build a simple application that returns exchange rates, one step at a time. We’ll write it in Kotlin with the newest Spring Boot version.
Project generation
We start by generating a project using spring initializr.
We’ll set it up like this
- Gradle project with Kotlin DSL
- Kotlin as the language
- Newest stable Spring Boot version
- Java version 17
- A dependency on Spring Web
Stub data
A service returning dummy data is enough to demonstrate how content negotiation works
private val EUR = Currency.getInstance("EUR")
private val USD = Currency.getInstance("USD")
private val PLN = Currency.getInstance("PLN")
@Service
class ExchangeRatesService {
private val exchangeRates: ExchangeRates = ExchangeRates(
listOf(
ExchangeRate(
baseCurrency = EUR,
quotedCurrency = USD,
rate = BigDecimal.valueOf(1.10)
),
ExchangeRate(
baseCurrency = USD,
quotedCurrency = EUR,
rate = BigDecimal.valueOf(0.90)
)
),
Instant.now()
)
fun getAllExchangeRates(): ExchangeRates {
return exchangeRates
}
}
data class ExchangeRates(val exchangeRates: List<ExchangeRate>, val updatedAt: Instant)
data class ExchangeRate(val baseCurrency: Currency, val quotedCurrency: Currency, val rate: BigDecimal)
Exposing an Endpoint That Serves the First Resource Representation in application/json
With that in place, we can move on to an endpoint offering the first representation (application/json) of the response:
package com.example.contentnegotiation.controller
import com.example.contentnegotiation.domain.ExchangeRatesService
import org.springframework.http.MediaType
import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.RequestMapping
import org.springframework.web.bind.annotation.RestController
@RestController
@RequestMapping("/exchange-rates")
class ExchangeRatesController(private val exchangeRatesService: ExchangeRatesService) {
@GetMapping(produces = [MediaType.APPLICATION_JSON_VALUE])
fun getAllExchangeRates(): ExchangeRatesResponse {
val exchangeRates = exchangeRatesService.getAllExchangeRates()
return ExchangeRatesResponse(
exchangeRates = exchangeRates.exchangeRates.map {
ExchangeRateResponseItem(
baseCurrency = it.baseCurrency,
quotedCurrency = it.quotedCurrency,
rate = it.rate
)
},
updatedAt = exchangeRates.updatedAt
)
}
}
Now we start the application and send a test request
curl --location 'http://localhost:8080/exchange-rates'
This is the response we get:
HTTP/1.1 200
Content-Type: application/json
Transfer-Encoding: chunked
Date: Thu, 17 Aug 2023 09:38:43 GMT
{
"exchangeRates": [
{
"baseCurrency": "EUR",
"quotedCurrency": "USD",
"rate": 1.1
},
{
"baseCurrency": "USD",
"quotedCurrency": "EUR",
"rate": 0.9
}
],
"updatedAt": "2023-08-17T09:32:21.407251Z"
}
Adding Another Resource Representation in application/xml
XML serialization requires one more dependency in the Gradle file: com.fasterxml.jackson.dataformat:jackson-dataformat-xml.
Next, we extend the existing endpoint by adding another media type (MediaType.APPLICATION_XML_VALUE) to the ‘produces’ section.
@RestController
@RequestMapping("/exchange-rates")
class ExchangeRatesController(private val exchangeRatesService: ExchangeRatesService) {
@GetMapping(produces = [MediaType.APPLICATION_JSON_VALUE, MediaType.APPLICATION_XML_VALUE])
fun getAllExchangeRates(): ExchangeRatesResponse {
...
We can then restart the application and verify that the XML representation looks right:
curl -i --location 'http://localhost:8080/exchange-rates' \
--header 'Accept: application/xml'
The response should look like this
HTTP/1.1 200
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Thu, 17 Aug 2023 10:12:28 GMT
<ExchangeRatesResponse>
<exchangeRates>
<exchangeRates>
<baseCurrency>EUR</baseCurrency>
<quotedCurrency>USD</quotedCurrency>
<rate>1.1</rate>
</exchangeRates>
<exchangeRates>
<baseCurrency>USD</baseCurrency>
<quotedCurrency>EUR</quotedCurrency>
<rate>0.9</rate>
</exchangeRates>
</exchangeRates>
<updatedAt>2023-08-17T10:12:23.792904Z</updatedAt>
</ExchangeRatesResponse>
Adding a more advanced resource representation: text/html
Here, I want to demonstrate handling a more complex representation – one that can’t be serialized automatically. Let’s take the text/html format. We’ll generate the HTML content with a Kotlin library offering a DSL for exactly that: org.jetbrains.kotlinx:kotlinx-html-jvm.
Next, we write a mapper that builds the HTML representation of the exchange rates resource
package com.example.contentnegotiation.controller
import com.example.contentnegotiation.domain.ExchangeRates
import kotlinx.html.*
import kotlinx.html.stream.appendHTML
import org.springframework.context.i18n.LocaleContextHolder
import org.springframework.stereotype.Component
import java.time.ZoneOffset
import java.time.format.DateTimeFormatter
private val DATE_TIME_FORMATTER = DateTimeFormatter.ofPattern("uuuu-MM-dd HH:mm:ss")
@Component
class HTMLMapper {
fun mapToHTML(exchangeRates: ExchangeRates): String {
val locale = LocaleContextHolder.getLocale()
return buildString {
appendHTML().html {
head {
title { +"Exchange rates" }
style {
unsafe {
+"""
h1, h2, p {
text-align: center;
}
table {
margin-left:auto;
margin-right:auto;
}
th, td {
text-align: center;
padding: 10px;
border:1px solid black;
}
""".trimIndent()
}
}
}
body {
h1 { +"Exchange rates" }
p {
h2 {
+"Updated at: ${DATE_TIME_FORMATTER.format(exchangeRates.updatedAt.atOffset(ZoneOffset.UTC))}"
}
table {
thead {
tr {
th { +"Base currency" }
th { +"Quoted currency" }
th { +"Rate" }
}
}
tbody {
exchangeRates.exchangeRates.map {
tr {
td { +"${it.baseCurrency}" }
td { +"${it.quotedCurrency}" }
td { +"${it.rate}" }
}
}
}
}
}
}
}
}
}
}
To use it in the ExchangeRatesControler, we first need to add a new variant of the endpoint to the controller
package com.example.contentnegotiation.controller
import com.example.contentnegotiation.domain.ExchangeRatesService
import org.springframework.http.MediaType
import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.RequestMapping
import org.springframework.web.bind.annotation.RestController
@RestController
@RequestMapping("/exchange-rates")
class ExchangeRatesController(
private val exchangeRatesService: ExchangeRatesService,
private val htmlMapper: HTMLMapper
) {
@GetMapping(produces = [MediaType.APPLICATION_JSON_VALUE, MediaType.APPLICATION_XML_VALUE])
fun getAllExchangeRates(): ExchangeRatesResponse {
val exchangeRates = exchangeRatesService.getAllExchangeRates()
return ExchangeRatesResponse(
exchangeRates = exchangeRates.exchangeRates.map {
ExchangeRateResponseItem(
baseCurrency = it.baseCurrency,
quotedCurrency = it.quotedCurrency,
rate = it.rate
)
},
updatedAt = exchangeRates.updatedAt
)
}
@GetMapping(produces = [MediaType.TEXT_HTML_VALUE])
fun helloHTML(): String {
return htmlMapper.mapToHTML(exchangeRatesService.getAllExchangeRates())
}
}
After that, we restart the application and confirm that the HTML representation comes back correctly
curl -i --location 'http://localhost:8080/exchange-rates' \
--header 'Accept: text/html'
We should get
HTTP/1.1 200
Content-Type: text/html;charset=UTF-8
Content-Length: 1156
Date: Thu, 17 Aug 2023 11:56:35 GMT
<html>
<head>
<title>Exchange rates</title>
<style>h1, h2, p {
text-align: center;
}
table {
margin-left:auto;
margin-right:auto;
}
th, td {
text-align: center;
padding: 10px;
border:1px solid black;
}</style>
</head>
<body>
<h1>Exchange rates</h1>
<p>
<h2>Updated at: 2023-08-17 13:03:04</h2>
<table>
<thead>
<tr>
<th>Base currency</th>
<th>Quoted currency</th>
<th>Rate</th>
</tr>
</thead>
<tbody>
<tr>
<td>EUR</td>
<td>USD</td>
<td>1.1</td>
</tr>
<tr>
<td>USD</td>
<td>EUR</td>
<td>0.9</td>
</tr>
</tbody>
</table>
</p>
</body>
</html>
Evaluating the Spring Boot Content Negotiation Algorithm
Three different media types are now supported. Time for a few experiments to see how Spring Boot uses content negotiation to choose the best-suited media type.
Support for quality values
To begin with, we’ll verify whether Spring Boot honours “q” values as specified in RFC-9110. We’ll send a handful of requests to find out.
For this request
curl -i --location 'http://localhost:8080/exchange-rates' \
--header 'Accept:text/html;q=0.9,application/xml;q=0.8,application/json;q=0.7'
we get
HTTP/1.1 200
Content-Type: text/html;charset=UTF-8
Content-Length: 1225
Date: Thu, 17 Aug 2023 13:13:40 GMT
...
For this request
curl -i --location 'http://localhost:8080/exchange-rates' \
--header 'Accept:text/html;q=0.8,application/xml;q=0.9,application/json;q=0.7'
we get
HTTP/1.1 200
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Thu, 17 Aug 2023 13:17:58 GMT
...
For this request
curl -i --location 'http://localhost:8080/exchange-rates' \
--header 'Accept:text/html;q=0.7,application/xml;q=0.8,application/json;q=0.9'
we get
HTTP/1.1 200
Content-Type: application/json
Transfer-Encoding: chunked
Date: Thu, 17 Aug 2023 13:20:04 GMT
...
Clearly, Spring Boot processed the ‘q’ value correctly every time, always choosing the supported representation with the highest ‘q’ value.
Missing Accept header
Next, let’s see what happens when no ‘Accept’ header is sent, or when it’s sent with an empty value.
When the Accept header is missing, empty or set to */*, we get the following response:
HTTP/1.1 200
Content-Type: application/json
Transfer-Encoding: chunked
Date: Thu, 17 Aug 2023 13:20:04 GMT
...
How is the representation resolved in this situation?
A look into the Spring code makes it clear that a missing or empty Accept header falls back to */*. That yields three possible representations of the resource, all equally prioritized. When several media types share the same quality value, their order is decided by the compareTo method of the MimeType class. Put simply, the mime types’ alphabetical order settles it.
Unsupported value in Accept header
Let’s see what happens when the Accept header carries a value the server doesn’t support:
For text/plain, we get:
HTTP/1.1 406
Accept: application/json, application/xml, text/html
Content-Length: 0
Date: Thu, 17 Aug 2023 13:49:31 GMT
The response comes with a 406 (Not Acceptable) status code and includes an Accept header listing every supported media type (application/json, application/xml, text/html).
Fetching exchange rates from a Web browser
Curiously, fetching the exchange rates resource from a web browser gives us the HTML representation
This is because web browsers send their own preferences for the content they retrieve. As we can see, text/plain is their top choice
Implementing Internationalization via Content Negotiation
To show how content language negotiation works in Spring, let’s add support for multiple languages… Our HTML content is a perfect candidate for internationalization.
For this, we’ll use the Spring class MessageSource, which can resolve text according to the locale context of a given request. Let’s change HTMLMapper so that it starts using MessageSource:
private const val EXCHANGE_RATES = "exchange-rates"
private const val UPDATED_AT = "updated-at"
private const val BASE_CURRENCY = "base-currency"
private const val QUOTED_CURRENCY = "quoted-currency"
private const val RATE = "rate"
private val DATE_TIME_FORMATTER = DateTimeFormatter.ofPattern("uuuu-MM-dd HH:mm:ss")
@Component
class HTMLMapper(private val messageSource: MessageSource) {
fun mapToHTML(exchangeRates: ExchangeRates): String {
val locale = LocaleContextHolder.getLocale()
return buildString {
appendHTML().html {
head {
title { +messageSource.getMessage(EXCHANGE_RATES, null, locale)}
...
table {
thead {
tr {
th { +messageSource.getMessage(BASE_CURRENCY, null, locale) }
th { +messageSource.getMessage(QUOTED_CURRENCY, null, locale) }
th { +messageSource.getMessage(RATE, null, locale)}
}
}
...
Next, we prepare the translation files. By default, the configuration files from the resource bundle following the naming convention messages_{language_code}.properties are used. The default language belongs in the messages.properties file.
We’ll now add translations for three languages (English, German, French), making English the default
Every language file contains the following properties, translated accordingly:
messages.properties
exchange-rates=Exchange rates
updated-at=Updated at
base-currency=Base currency
quoted-currency=Quoted currency
rate=Rate
Then we have to configure which locales are supported. We do that by providing a LocaleResolver bean
@Configuration
class LocaleConfiguration {
@Bean
fun localeResolver(): LocaleResolver {
val localeResolver = AcceptHeaderLocaleResolver()
localeResolver.setDefaultLocale(Locale.ENGLISH)
localeResolver.supportedLocales = listOf(Locale.ENGLISH, Locale.GERMANY, Locale.FRENCH)
return localeResolver
}
}
With that done, we can restart the application and send a few requests
Missing Accept-Language header
A request with no Accept-Language header
curl -i --location 'http://localhost:8080/exchange-rates' \
--header 'Accept: text/html'
returns a response in the default language, i.e. English
HTTP/1.1 200
Content-Type: text/html;charset=UTF-8
Content-Length: 1225
Date: Fri, 18 Aug 2023 09:35:31 GMT
<html>
<head>
<title>Exchange rates</title>
<style>h1, h2, p {
text-align: center;
}
table {
margin-left:auto;
margin-right:auto;
}
th, td {
text-align: center;
padding: 10px;
border:1px solid black;
}</style>
</head>
<body>
<h1>Exchange rates</h1>
<p>
<h2>Updated at: 2023-08-18 09:27:59</h2>
<table>
<thead>
<tr>
<th>Base currency</th>
<th>Quoted currency</th>
<th>Rate</th>
</tr>
...
Accept-Language with unsupported value
If the Accept-Language header asks for Spanish, which isn’t supported
curl -i --location 'http://localhost:8080/exchange-rates' \
--header 'Accept: text/html' \
--header 'Accept-Language: es'
the response comes back in the default language (in contrast to the Accept header, no 406 response is returned here)
HTTP/1.1 200
Content-Type: text/html;charset=UTF-8
Content-Length: 1225
Date: Fri, 18 Aug 2023 09:35:31 GMT
<html>
<head>
<title>Exchange rates</title>
<style>.....</style>
</head>
<body>
<h1>Exchange rates</h1>
<p>
<h2>Updated at: 2023-08-18 09:27:59</h2>
<table>
<thead>
<tr>
<th>Base currency</th>
<th>Quoted currency</th>
<th>Rate</th>
</tr>
...
List of languages with quality values
If the request includes a list of languages with q values
curl -i --location 'http://localhost:8080/exchange-rates' \
--header 'Accept: text/html' \
--header 'Accept-Language: en;q=0.5,fr;q=0.7,de;q=0.6,es;0.9'
we get the supported language that has the highest q value – French, in this case
HTTP/1.1 200
Content-Type: text/html;charset=UTF-8
Content-Length: 1225
Date: Fri, 18 Aug 2023 09:42:44 GMT
<html>
<head>
<title>Kursy wymian</title>
<style>h1, h2, p {
text-align: center;
}
table {
margin-left:auto;
margin-right:auto;
}
th, td {
text-align: center;
padding: 10px;
border:1px solid black;
}</style>
</head>
<body>
<h1>Kursy wymian</h1>
<p>
<h2>Zaktualizowane: 2023-08-18 09:27:59</h2>
<table>
<thead>
<tr>
<th>Bazowa waluta</th>
<th>Notowana waluta</th>
<th>Kurs</th>
</tr>
...
Fetching exchange rates from a Web browser
Just as with the content type, the browser also sends its content language preferences
These preferences can be set in the browser settings. Here’s how it looks in Google Chrome
The content came back in my browser’s default language
Negotiation of Content Encoding
Content encoding requires three Spring Boot properties to be configured
server.compression.enabled: turns on content compression on the server sideserver.compression.mime-types: defines which content types may be compressedserver.compression.min-response-size: the smallest response size, in bytes, that gets compressed.
Let’s put these properties in the application.properties file
server.compression.enabled=true
server.compression.mime-types=text/html,application/json,application/xml
server.compression.min-response-size=1024
Next, we restart the application and send a few test requests
Request without Accept-Encoding
If we send a request
curl -sI --location 'http://localhost:8080/exchange-rates' \
--header 'Accept: text/html'
we get this response
HTTP/1.1 200
vary: accept-encoding
Content-Type: text/html;charset=UTF-8
Content-Length: 1225
Date: Fri, 18 Aug 2023 12:42:37 GMT
The response isn’t compressed, even though it’s larger than the minimum compression size we configured. We can also see the Vary: Accept-Encoding header, signalling that the Accept-Encoding header may be taken into account when the response is built.
Request with indicated content encoding
If the request specifies gzip as an accepted encoding
curl -sI --location 'http://localhost:8080/exchange-rates' \
--header 'Accept: text/html' \
--header 'Accept-Encoding: gzip'
the response comes back compressed
HTTP/1.1 200
vary: accept-encoding
Content-Encoding: gzip
Content-Type: text/html;charset=UTF-8
Transfer-Encoding: chunked
Date: Fri, 18 Aug 2023 13:25:33 GMT
Quality value used in content encoding
If we send a request whose Accept-Encoding header has two values, gzip and identity, with identity given the higher quality value:
curl -sI --location 'http://localhost:8080/exchange-rates' \
--header 'Accept: text/html' \
--header 'Accept-Encoding: gzip;q=0.1,identity;q=0.3'
we still get a compressed response
HTTP/1.1 200
vary: accept-encoding
Content-Encoding: gzip
Content-Type: text/html;charset=UTF-8
Transfer-Encoding: chunked
Date: Fri, 18 Aug 2023 13:32:52 GMT
So in this case, the higher ‘q’ value isn’t used for prioritization.
Encoding preference with a quality value equal to zero
If we send a request
curl -sI --location 'http://localhost:8080/exchange-rates' \
--header 'Accept: text/html' \
--header 'Accept-Encoding: gzip;q=0'
the response comes back uncompressed
HTTP/1.1 200
vary: accept-encoding
Content-Type: text/html;charset=UTF-8
Content-Length: 1225
Date: Fri, 18 Aug 2023 13:36:40 GMT
This shows that the quality value can be used to rule out particular compression formats.
Adjustments of Vary header
As we saw, enabling response compression caused the Vary header to appear in responses. That isn’t quite what we want, though. Our content negotiation relies on headers such as Accept, Accept-Language, and Accept-Encoding, and each of them can change the response representation.
Sadly, Spring has no dedicated property for changing this behaviour. A servlet filter, however, makes for an easy workaround:
@WebFilter("/exchange-rates")
class ExchangeRateFilter : Filter{
override fun doFilter(request: ServletRequest?, response: ServletResponse?, chain: FilterChain?) {
val httpServletResponse = response as HttpServletResponse
httpServletResponse.setHeader(HttpHeaders.VARY, "${HttpHeaders.ACCEPT}, ${HttpHeaders.ACCEPT_LANGUAGE}, ${HttpHeaders.ACCEPT_ENCODING}")
chain!!.doFilter(request, response)
}
}
For this to take effect, the @ServletComponentScan annotation has to be added too.
@SpringBootApplication
@ServletComponentScan
class ContentNegotiationApplication
fun main(args: Array<String>) {
runApplication<ContentNegotiationApplication>(*args)
}
Then we restart the application and send a sample request
curl -I --location 'http://localhost:8080/exchange-rates'
The response should now include a Vary header listing every field that can change the response representation
HTTP/1.1 200
Vary: accept,accept-language,accept-encoding
Content-Type: application/json
Transfer-Encoding: chunked
Date: Fri, 18 Aug 2023 14:31:51 GMT
Summary
This article explored the theory of content negotiation as described in RFC-9110. Once the theory was covered, we built a practical example with Spring Boot.
Below is an overview of how Spring Boot supports content negotiation, how well it matches the theory, and how it behaves in edge cases:
| Negotiation element | Header | q value taken into account | Behaviour when header is missing | Header values not supported | Included in Vary header | Content negotiation pattern |
|---|---|---|---|---|---|---|
| Media type | Accept | Yes | The supported media type that comes first alphabetically is returned | 406 (Not acceptable) is returned; the response carries an Accept header listing the supported media types | No | Proactive, Reactive |
| Language | Accept-Language | Yes | The default language is used | The default language is used | No | Proactive |
| Encoding | Accept-Encoding | No | No encoding is used | No encoding is used | Yes | Proactive |
It’s worth highlighting that Spring Boot ships with a built-in, very flexible content negotiation mechanism. It can be regarded as a kind of proactive content negotiation, in which a 406 response code appears only in the rare situations where the server can’t offer an acceptable representation.
That said, it doesn’t always follow the RFC-9110 specification strictly in every scenario. This showed most clearly in encoding negotiation and in how the Vary header was attached, both of which we had to adjust.