Estimate project
← All posts

Rust Patterns in Depth vol. 1: Newtypes in Rust

An in-depth technical look at the Newtype pattern in the Rust programming language. The article examines how newtypes differ semantically from type aliases and explains how wrappers with private fields can enforce domain invariants. It covers newtypes as resource handlers for graph structures, applying the typestate pattern to data validation (e.g., Email and Filter types), and ways of hiding implementation details to stay semver compatible. It also warns about typical pitfalls, like 'hidden conversions' during serialization and excessive boilerplate in simple data structures.

16 min read
RustProgramming LanguagesBest Practices

Newtypes are likely the simplest pattern used in the Rust programming language – so simple that calling it a pattern is a stretch, yet there are particular situations where it proves very handy. At the same time, it’s easy to overuse, which results in somewhat too verbose code. Let’s look into the pattern and think about where it fits best.

What are newtypes?

In essence, a newtype is a structure with a single field that wraps some underlying type:

struct Handler(usize);

This defines a newtype Handler that stores a usize inside. It’s a very typical case for me: when I have an underlying vector and don’t want to index it with a bare usize, I create a Handler wrapper instead.

It’s worth noting how newtypes differ from type aliases. The following is not a newtype:

type OtherHandler = usize;

OtherHandler is merely an alias for usize. The crucial difference is that a newtype really introduces a new type, whereas the type keyword just gives an existing type another name. As a result, in this example OtherHandler and usize are interchangeable, while Handler is an entirely distinct type:

fn takes_handler(handler: Handler) {}
fn takes_usize(handler: usize) {}

fn main() {
    let raw = 0usize;
    let handler = Handler(0);
    // Note that I just assign an `usize` - its the exact same type
    let other_handler = 0usize;
    // Following would not compile, `Handler` is *not* an `usize`
    // let handler2 = 0usize;

    takes_usize(raw);
    // `takes_usize` accepts `OtherHandler`
    takes_usize(other_handler);
    // But doesn't accept `Handler`
    // takes_usize(handler);

    // `takes_handler` accepts `Handler` only
    takes_handler(handler);
    // Passing `usize` or `OtherHandler` fails
    // takes_handler(raw);
    // takes_handler(other_handler);
}

With the formal difference between type aliases and newtypes clear, let’s turn to another API aspect – hermetization of the inner field. Should the newtype’s only field be public? Technically, either choice is valid, but personally I reserve the name newtype for types whose field is private. The reason lies in what newtypes are for. We reach for this pattern when we technically need the information carried by an underlying type, but we use it with semantics specific to the API and don’t want the underlying type to be passed in directly.

Following that reasoning – once the newtype’s field is public, an API user holding the underlying type can hand it straight to the function, which more or less defeats the point of newtypes. For example:

struct PubHandler(pub usize);

fn takes_pub_handler(handler: PubHandler) {}

fn main() {
    let raw = 0usize;
    let handler = PubHandler(0);

    takes_pub_handler(handler);
    // Passing `usize` fails
    // takes_pub_handler(raw);
    // But there is nothing stopping us to just create it in-place
    take_pub_handler(PubHandler(raw));
}

A newtype like this introduces a little explicit noise, which may be exactly what we’re after, but in my view it mostly serves different use cases. That’s why I personally call types with public fields wrapper types and types with private fields newtypes. This also matches the common convention that follows Rust by Example. Bear in mind that this is more of an idiomatic guideline – in the Rust ecosystem, you may see what I call “wrapper types” referred to as “newtypes”.

Why use newtypes?

As I said earlier, the simplest use of newtypes is passing around information represented by the underlying type, but with different semantics. Because the meaning differs, we usually don’t want just any value of the underlying type to be accepted – we want some control over it.

The two most frequent use cases are:

  1. Accepting only values that we created ourselves earlier.
  2. Attaching validation to the value.

Quite often we want both – values returned by our API are correct from the start, and we may still offer a way to build the newtype from the underlying type, with additional validation.

Use case – newtype as resource handlers

The simplest newtype example, which I’ve used several times, is the Handler type shown above. Suppose we’re building a library that internally maintains a graph – perhaps a computation graph. One of the most common ways to represent a graph is to store the nodes in a container and add some means of representing the connections between them – possibly in separate containers.

With such a representation, GraphNode may be a fairly costly structure that we’d rather not pass around. We could use Rc<GraphNode> instead, but often the simple approach is to keep a vector of nodes and pass around a node’s index rather than the whole node:

struct MyGraph {
    nodes: Vec<GraphNode>,
}

impl MyGraph {
    fn create_node(&mut self) -> usize {
        nodes.push(GraphNode::default());
        nodes.len() - 1
    }
}

Then, whenever we want to do something with the nodes, we pass their indices:

impl MyGraph {
    fn add_connection(&mut self, from: usize, to: usize) {
        self.nodes[from].add_connection(to)
    }
}

This works perfectly well, but the drawback is that the user can feed arbitrary numbers into the API, which may lead to trouble – in this case, most likely a panic:

fn main() {
    let mut graph = MyGraph::default();
    let node1 = graph.create_node();
    let node2 = graph.create_node();

    // All is fine
    graph.add_connection(node1, node2);

    // Here we get a panic
    graph.add_connection(100, 200);
}

It may not be catastrophic – Rust would make sure invalid indices can’t be accessed, and the documentation can always state that indices not previously returned by the create_node method must never be passed in.

Still, we can do better and make invalid indices impossible to create:

struct NodeHandler(usize);

impl MyGraph {
    fn create_node(&mut self) -> NodeHandler {
        let index = self.nodes.len();
        self.nodes.push(GraphNode::default());
        NodeHandler(index)
    }

    fn add_connection(&mut self, from: NodeHandler, to: NodeHandler) {
        self.nodes[from.0].add_connection(to.0)
    }
}

The code is almost the same as before, yet it prevents certain problems. One might ask, though – why would anyone pass a value we never returned? Perhaps they assumed the values could safely be stored in a database, then restored and reused, which isn’t the case. Nobody is likely to pass 100 directly as a node index, but invalidating a value obtained from someone else is a fairly probable mistake.

Be very careful about any extra assumptions you make regarding the values you receive. In this example, the add_connection method may assume that the from and to parameters are nodes created by MyGraph, but it can’t safely assume they come from the same instance of MyGraph. Consider the following example:

impl MyGraph {
    fn add_connection(&mut self, from: NodeHandler, to: NodeHandler) {
        unsafe {
            self.nodes.get_unchecked(from.0).add_connection(to.0)
        }
    }
}

It’s tempting to think that since we’re the sole producer of NodeHandler, we can trust it completely and apply the crucial optimization of skipping the bound check. However, we can quite easily end up with unsound code:

fn main() {
    let graph1 = MyGraph::default();
    let graph2 = MyGraph::default();
    let node1 = graph1.create_node();
    let node2 = graph2.create_node();
    let node3 = graph3.create_node();
    // `node3` index here is `1`, and the `self.nodes.get_unchecked(1)` becomes an unsound call
    graph1.add_connection(node3, node2);
}

If this optimization really mattered, it could be implemented, but that would require additional instrumentation with lifetimes.

In this kind of scenario, Newtypes would usually help deliver a developer-friendly API, but they would rarely help with performance optimizations.

Types validation

The second use of newtypes came up recently in our company Book Club. At first I was a bit skeptical, but I believe that’s because it was approached from an angle that isn’t very typical for Rust. Nonetheless, it’s a very common application of newtypes in the Rust programming language.

In the earlier example, newtypes guaranteed that the graph node handler was created by our crate and wasn’t just some random usize value fed into our system. Another case for newtypes is similar – we want to restrict the values passed to a function, but they don’t have to be values we created earlier. Maybe we only want to be sure the value went through some specific processing – and that processing would often be validation.

Here’s a real-life example. Say we’re building a chess tool that filters games. The filter object would look like this:

struct Filter {
    pub player: Option<String>,
    pub color: Option<Color>,
}

impl ChessService {
    fn fetch_games(&self filter: Filter) -> Vec<Game> {
        todo!()
    }
}

The Filter struct describes how games are filtered. In the simplified version, we only want games of the given player. Additionally, if the color is set, we only get games this player played with that particular color.

The basic assumption we’d like to hold for the Filter struct is easy to spot – the color field should probably never be set when player is None. One way to handle this is to validate the filter within the fetch_games method:

impl ChessService {
    fn fetch_games(&self, filter: Filter) -> Vec<Game> {
        if filter.player.is_none() && filter.color.is_some() {
            panic!("Invalid filter: color cannot be set without player");
        }
        todo!()
    }
}

It works, but there’s a small problem – if the same filter gets passed to fetch_games or other methods several times, it has to be validated every time. The performance cost isn’t large, but it’s work that can easily be avoided. The newtype pattern solves this:

struct ValidatedFilter(Filter);

impl ValidatedFilter {
    fn new(filter: Filter) -> Self {
        if filter.player.is_none() && filter.color.is_some() {
            panic!("Invalid filter: color cannot be set without player");
        }
        Self(filter)
    }
}

impl ChessService {
    fn fetch_games(&self, filter: ValidatedFilter) -> Vec<Game> {
        todo!()
    }
}

Now validation happens just once, when the filter is created, and the rest of the system can treat it as valid. Unlike in the previous case, here we can even gain a bit of performance, because the validation doesn’t depend on any other entities.

Of course, a newtype isn’t strictly required here – the ValidatedFilter struct could just as well contain all the fields of Filter, but that would mean duplicating code.

This use of a newtype can be seen as a basic application of the typestate pattern. The typestate pattern is common in Rust; the idea is to use the type system to express an object’s state. Here, the Filter type is a potential filter that may still be under construction, while the ValidatedFilter type is a filter in the validated state. This is also frequently combined with builders – Filter would serve as a builder, perhaps with methods for setting the fields, and the final Filter::build(self) method would run the validation and return the finished ValidatedFilter.

One more example of using a newtype for validation is an Email type in a system where we need emails to be valid, and perhaps also want to filter allowed domains or apply deny-lists. Even basic email validation is fairly complex, and in some systems it can become arbitrarily complex.

While it’s hard to picture how the performance of email validation would affect a system that depends on the email format, I can easily picture a system whose stability truly depends on that format. And here’s something I’ve learned: if the compiler doesn’t check something, sooner or later I’ll forget to do it – or someone on my team will. This is how the code will look:

struct Email(String);

impl Email {
  fn validate(str: String) -> Result<Self, ValidationError> {
    // Validation code...

    Self(str)
  }
}

The newtype pattern helps ensure we never skip email validation – any code that depends on the actual email format is expected to always use the newtype. Code that handles email as a String or &str is fine, as long as that piece of code doesn’t depend on the format.

Note also that in this case it can be quite useful to add functions converting to a string – perhaps a From conversion (but only from Email to String, never the reverse!), perhaps an AsRef or even a Deref trait exposing the inner string. Turning Email back into a String causes no trouble – even if it gets modified or processed before reaching the critical part of the system, it will have to be validated again to become a proper email.

We can go even further to make this newtype more convenient. The TryFrom trait lets us implement conversion from String to Email – and it’s not a direct, unchecked conversion. Because the conversion is fallible, we get the chance to validate:

impl TryFrom<String> for Email {
  type Error = ValidationError;

  fn try_from(value: String) -> Result<Self, Self::Error> {
    Self::validate(value)
  }
}

We can add yet more helpers and allow deserializing straight into Email – just remember to always run validation when doing so. When using serde for deserialization, we should write a custom deserialization implementation or use the try_from serde attribute.

Hiding underlying type

There’s another reason to use the newtype pattern, particularly valuable when building libraries – hiding the underlying type completely. Until now, we focused mostly on the underlying data value and treated the newtype mainly as a guard that keeps invalid input out of our system.

Sometimes, though, we may want to avoid exposing the underlying type in order to keep some flexibility for future API changes.

Let’s go back to the graph example. We said the graph consists of GraphNode values kept in a hidden vector. At some point, however, we may want to change that implementation. Perhaps the vector will go away and we’ll carry Rc<NodeData> instead. Or perhaps a vector won’t be sufficient at some stage, and we’ll want a HashMap<Uuid, GraphNode> or even a database, where usize wouldn’t be a suitable index type.

By using the newtype pattern in the library API, we hand the user a type that expresses its semantics rather than its implementation details. This makes it easier to swap out the whole implementation without a major version bump while staying semver-compatible.

Is a newtype always a goto?

We’ve covered two main situations in which the newtype pattern helps deliver a better API. So the question is – should we push for newtypes wherever they seem to make sense?

Personally, I’m against that. Newtypes can improve an API, but they are just as good at adding boilerplate.

Here’s an example. Suppose we’re writing a graphics library and want to offer a Color type like this:

struct Color(u8, u8, u8);

The three elements of Color are the RGB values, each 8-bit. But we’re very enthusiastic about newtypes, so we “improve” our API:

struct Channel(u8);
struct Color(Channel, Channel, Channel);

impl Color {
    fn new(r: Channel, g: Channel, b: Channel) -> Self {
        Self(r, g, b)
    }
}

The trouble is that for this specific use, we almost certainly want to allow building a Channel from a u8, and vice versa:

impl From<u8> for Channel {
    fn from(value: u8) -> Self {
        Self(value)
    }
}

impl From<Channel> for u8 {
    fn from(value: Channel) -> Self {
        value.0
    }
}

Now a user of our crate can indeed create a blue color:

fn main() {
    let blue = Color(Channel::from(0), Channel::from(0), Channel::from(255));
}

With the Into trait, it can look even nicer:

fn main() {
    let blue = Color::new(0.into(), 0.into(), 255.into());
}

Yet I honestly see no benefit of this syntax compared to a plain Color::new(0, 0, 255). We’ve added a boilerplate requirement without gaining anything meaningful.

My rule of thumb for using newtypes: use one if you won’t offer a constructor that builds the newtype directly from the underlying type – whether via a method or the From trait. If that’s the case, it means there is some constraint on the shape of the value, checked whenever the type is returned. Note, however, that a constructor performing validation is not what I mean by direct creation. So if the constructor is fn ValidatedFilter::new(Filter) -> Result<Filter>, it’s very much a case for the newtype pattern.

Common pitfall – hidden conversion

A theme I try to carry through this whole article is that a newtype should ensure its creation is controlled. There’s one typical pitfall I’ve seen a few times when newtypes were used carelessly – the hidden conversion.

As I noted in the color example, newtypes sometimes lead to piles of from / into calls, which to me signals that the pattern is being abused.

And while the From trait is a very explicit conversion, when implementing newtypes we should be especially careful that every creation path – conversions in particular – goes through the proper construction that keeps the type consistent. The usual slip-up involves serialization. I already brought this up in the Email example, but it bears repeating – don’t blindly #[derive(Deserialize)] on your newtypes. It may be a serde::Deserialize, but it would cover every deserialization point you might encounter – your protocol implementation, your binary mmaping allocator.

A very concrete counterexample of applying newtypes comes from a project I used to maintain – CosmWasm. It has one particular, very widely used type: Addr. Addr is a newtype over a string representing a valid Cosmos address. It’s well documented and created through validation, but it has one flaw – it can be deserialized directly. There were very specific reasons for that, and I don’t want to get into whether it’s a justified exception. The reasons mainly had to do with the type being frequently serialized and deserialized in storage; given the nature of CosmWasm, performance mattered, and so did skipping validation when it wasn’t needed.

There was, however, one case where Deserialize caused a real problem – the Addr type could be used directly as a field in an incoming smart contract message. And such a message had one particular property – it came from an untrusted user. This small slip made it possible to skip address validation by accident, which could lead to all sorts of failures. I don’t know of any critical bugs caused by it, but I did run into a few situations where we lost time on testing because of it – for instance, configuration errors that could have been caught early instead of surfacing as deep errors.

The issue was addressed through guidelines – seasoned CosmWasm devs know never to put Addr in message types for exactly this reason. We did similar things with numbers in the past – for example, Wasm implementation limitations forced us to avoid i128.

It’s only one real-life example, but it taught me a valuable lesson about working with newtypes.

Final word

The key point of the newtype pattern is controlling how the type is created, whatever the reason. Use it whenever it gives you useful type-checked preconditions – and use it with care! Always remember to review every creation path, and enjoy a clean, Rusty codebase.