Estimate project
← All posts

Core principles for building applications in Scala

A few principles worth keeping in mind when writing code with Scala and functional programming.

8 min read
ScalaJavaFunctional ProgrammingConcurrencyMacrosImmutability

Scala, a functional programming language with static typing, has been around for nearly 20 years. Over that period, several mainstream ways of writing Scala code have taken shape. Among them are working with effect systems, various techniques for running code asynchronously, and relying on a range of libraries and frameworks.

Although these approaches differ considerably, they all share a common set of principles. Put another way, whichever stack you prefer when building applications in Scala, there are things you routinely do, and things you never do.

These principles grow out of three pillars: the language itself, the standard library, and the community. Let’s examine a few of these practices more closely, at times comparing the Scala approach to the way applications are built in the Java ecosystem.

Immutable data structures

Among the most basic and far-reaching characteristics of Scala code is that data structures are immutable by default. This approach brings several advantages (along with a few drawbacks):

  • nobody “else” can change the data “somewhere else”
  • since state outside a function can’t change, the code is easier to follow; this is also known as local reasoning
  • data is protected from corruption, because concurrent modification is impossible
  • better thread-safety and, overall, simpler concurrent programming

Other programming platforms also favour immutable data structures for concurrent programming. That Scala makes it the default may be one of the reasons concurrent programming became one of its niches.

Scala backs immutable data structures at many levels. To begin with, the language itself provides constructs such as case classes, class fields that are vals (rather than vars) by default, enums (in Scala 3), and ADTs (sealed families of classes). Equally important is how easy it is to create modified copies of data structures thanks to the automatically generated copy methods.

Perhaps even more significant is that the standard library is designed around immutable data structures – whenever you reach for a Set, a List, or Map, you get the immutable variant by default.

Lastly, the majority of Scala libraries accept and return immutable data structures. It’s simply the way data is handled in Scala. Naturally, mutable data structures are still available when you need them. But that’s more of an exception than the rule.

There’s nothing preventing you from using immutable data structures on other platforms, such as node.js, Java or .NET. In those cases, though, the language offers only limited support. What matters more is that the standard libraries and popular community libraries are built on mutable data structures. Even where immutability is achievable, it can turn out to be very impractical.

For instance, nearly every Java library relies on the mutable collection interfaces from the standard library. Whatever Java library you use, it will almost surely work with the standard, mutable collections. That’s practically impossible to change: Java is going to stay a language that puts mutability first.

Expressions and values

One Scala feature that may not catch your eye at first, yet soon becomes something you can’t live without, is that everything in Scala is an expression. After getting accustomed to it, programming in a language that separates statements from expressions feels clumsy and needlessly restrictive.

This shows how flexible and versatile Scala’s syntax is. Looking at the three pillars we described at the start, being expression-oriented is a trait that belongs purely to the language.

A related Scala practice is modelling various concepts as values. Take functions first: defining a function-value requires very lightweight syntax. In Scala, it simply comes naturally. The standard library strengthens this further by relying on higher-order functions (meaning functions that accept other functions as parameters or return functions). So does the entire ecosystem, which follows the same approach.

It goes beyond functions, though. A major advantage of Scala is how flexibly you can define abstractions. In practice, this frequently means being able to model various concepts as values. Perhaps unexpectedly, introducing this extra layer of indirection unlocks interesting new possibilities.

A flagship example are functional effect systems such as ZIO or cats-effect. In them, the whole computation is modelled as a value. This gives us lazy, controlled evaluation of effectful code. Paired with a custom runtime, that makes declarative concurrency possible, along with lightweight threads that support principled interruptions, or fearless refactoring.

Another example comes from HTTP libraries, where you often start by creating a value describing an endpoint or an HTTP request. Naturally, this value is immutable, which makes it possible to refine it step by step and to separate concerns: the networking layer from the domain layer.

Macros, not reflection

The third principle follows straight from Scala’s emphasis on correctness through statically checked types. Once more, we’ll mainly compare the “Scala approach” with what Java provides.

Scala gives you great flexibility in defining abstractions, but that can only take you so far. Sooner or later, you’ll have to either write big blocks of boilerplate or find some way to have the code generated for you. Typical cases are generating JSON encoders/decoders, wiring up the object graph with dependency injection, or mapping classes onto database tables.

So the question becomes: how should this code-generation process be defined? Which language do we use, and how do users specify what ought to be generated?

In Java, this is typically done with annotations, which a framework such as Spring then reads via reflection at run-time. The framework then serves as an interpreter, producing bytecode or creating instances of predefined classes. I’ve long held the view that Java overuses annotations and that they effectively form a parallel, limited, and unsafe programming language with underspecified, ad-hoc interpreters.

So what’s the alternative? Scala’s answer is to generate code at compile time. Its most widespread form, type-directed term derivation, is known as implicits (in Scala 2) or givens (in Scala 3). In the same way an IDE can frequently infer the types of a piece of code, the Scala compiler can infer code from a type. Scala 3 has improved the “developer experience” of this feature.

Scala goes even further, however – inlines and macros directly replace annotation-based code generation. They make it possible to generate Scala code at compile-time in a type-safe manner, using the entire Scala language to describe the process and fitting seamlessly with the rest of Scala’s features.

Annotations can still steer a macro and supply extra meta-data – but that’s just one possibility; examining the types and terms handed to the macro is another. Scala offers plenty of examples of macros in use, such as the jsoniter-scala library or macwire for dependency injection at compile time.

Macros are frequently described as difficult to grasp. And even though writing a macro can indeed be anything but trivial, the same is true – I’d claim to an even greater extent – of writing annotation processors or bytecode generators.

Using macros is another matter entirely; typically it’s no different from invoking a method. Error messages not always being perfect is an issue, but one that comes down to the ecosystem maturing, not a fundamental flaw of the whole approach.

Even though code generation driven by annotations and based on reflection still dominates the Java ecosystem, there are signs that it, too, is shifting toward generation at compile time. One example is Hibernate’s metamodel (which relies on annotation processors). Still, the limited expressiveness of annotations as a sub-language continues to be a problem. Kotlin’s serialization plugin (https://github.com/Kotlin/kotlinx.serialization) is another example.

Explicit when necessary, implicit when not

The final principle isn’t a direct result of how Scala or its standard library were designed, but of the “culture” of functional programming as a whole, and of the Scala community specifically.

On nearly every platform there’s a temptation to add some kind of “magic”. The motivation is good: we want things automated and happening by themselves. Trouble begins, though, when we as users of this “magic” can’t identify and grasp what is going to happen, when, and for what reason. That clashes with the predictability and explorability of our code bases, which play a major role in delivering maintainable software.

A textbook case of such “magic” is classpath scanning and “auto-discovery”, which switches certain components on conditionally, just because they happen to be on the classpath. In Scala code, this is almost completely absent.

Would you want your operating system to change its behaviour simply because you downloaded something from the Internet? Yet classpath-scanning & auto-instrumentation magic does exactly that. Scala prefers features to be switched on explicitly, even at the cost of three extra lines of code.

A further example is that Scala applications always define their main methods explicitly. For whatever reason, Java code frequently lacks such a clear entry point, relying instead on our trusty old companions: annotations and reflection. Perhaps Java is the odd one out here, or perhaps Scala is bringing a bit of sanity to things – I can’t say. What’s certain is that there’s no need to fear the main.

What’s the origin of this? When we write Scala code, or follow the “functional” style in general, we deal with functions that can take other functions as parameters and return values which we then hand on to other functions. That, after all, is the essence of functional programming: functions treated as first-class values.

As a result, we can trace code paths directly – using the simple yet highly effective “go-to definition” feature of our IDEs.

Summary

None of the characteristics above is exclusive to Scala in general. What makes for such a good mix is the way they combine. Language design isn’t everything; defaults matter too, as immutable data structures and immutable collections in the standard library demonstrate.

A regular, surprisingly small grammar in an expression-oriented language brings new possibilities – for instance, concise representation of concepts as values.

Extending compilation with type-safe code generation is more efficient, safer, and more predictable than processing annotations at run-time.

Lastly, being explicit about what happens in your code, when and why, makes our codebases easier to explore and understand, even when we have to find our way through infrastructure code supplied by libraries.

All of this stands on three pillars: the Scala language, its standard library, and the libraries and frameworks provided by the community. Which principles would you add to this list when developing code with functional programming and Scala?