Concurrent Programming with Cats Effect
This blog post is a beginner-friendly tutorial covering the concurrency features that Cats Effect offers.
Concurrency is difficult. As Brian Goetz puts it in “Java Concurrency in Practice”:
It is hard to write correct programs, and it is even harder to write correct concurrent ones. A concurrent program simply has more ways to go wrong than a sequential one does.
Luckily for us, Scala offers plenty of excellent tools that help tame concurrent programming, such as Akka, ZIO, Finagle or Cats Effect. In this article, I will concentrate on what the last of these has to offer.
What is Fiber?
The fundamental building block of concurrency in CE is the fiber. Put simply, a fiber is a lightweight logical thread representing a sequence of actions. In CE, that means a list of operations suspended in the IO monad and chained together with flatMap.
Compared to system threads, fibers have a small footprint. Above all, they use very little memory, so they are not nearly as scarce a resource as threads. There is no need for fiber pools – we can simply spawn a new fiber any time we need one.
In contrast to system threads, fibers don’t depend on the operating system’s scheduler for context switching. They rely instead on a scheduler at the framework level, which decides when and how they run. A given fiber is not tied to any specific thread. The Cats Effect runtime is able to multiplex tens of thousands of fibers onto a handful of system threads. On the JVM, the runtime typically uses a fixed-size, CPU-bound thread pool, while in single-threaded environments such as Scala.js it can even work on just one thread.

Managing fibers
CE lets us fork computations using the start method. It returns a Fiber instance exposing a join method. Calling it makes the fiber at the call site wait until the joined fiber has finished.
import cats.effect.*
import scala.concurrent.duration.*
val task = for {
_ <- IO.println("Task started")
_ <- IO.sleep(1.second)
_ <- IO.println("Task completed")
} yield ()
for {
fiber <- task.start
_ <- IO.println("Hello from main fiber")
//we wait approx. 1s till forked fiber completes
_ <- fiber.join
} yield ()
Fiber also offers a cancel method, which cancels the fiber together with the computations running on it. cancel does not give control back until every resource allocated in the fiber has been correctly finalized and all the underlying operations have been canceled.
Using onCancel(fin: IO[Unit]), we can set up a finalizer that runs whenever a task gets canceled.
import cats.effect.*
import scala.concurrent.duration.*
val task = for {
_ <- IO.sleep(5.seconds)
_ <- IO.println("Computations complete")
} yield ()
for {
fiber <- task.onCancel(IO.println("Task canceled!")).start
_ <- IO.sleep(1.second)
//fiber would complete after 5s, but is canceled after 1s
_ <- fiber.cancel
} yield ()
join returns an Outcome, whose subtypes are Succeeded, Errored and Canceled. Pattern-matching on the outcome lets us check the status of the finished fiber.
Whatever value the fiber returns is exposed as a field of the Succeeded outcome. That is how we can pass values from the forked task back to the call site.
import cats.effect.*
import scala.concurrent.duration.*
val task = IO.sleep(500.millis) *> IO.pure("Hello world!")
for {
fiber <- task.start
_ <- fiber.join.flatMap {
case Outcome.Succeeded(value) => value.flatMap(
v => IO.println(s"Computed value: $v")
)
case Outcome.Errored(e) => IO.println(s"Error: $e")
case Outcome.Canceled() => IO.println("Task was canceled!")
}
} yield ()
There is also an alternative version of join: joinWith(onCancel: F[A]). It lets us provide a fallback computation for when the fiber gets canceled. It has two specialized forms: joinWithNever, a shorthand for joinWith(IO.never), and joinWithUnit (a shorthand for joinWith(IO.unit)).
Timeouts
The timeout method lets us set a time limit after which the fiber gets canceled.
task
.onCancel(IO.println("Task has timed out!"))
.timeout(500.millis)
.start
With timeoutTo, we can provide a fallback for the canceled computation.
task
.onCancel(IO.println("Task has timed out!"))
.timeoutTo(500.millis, planB)
.start
Just like cancel, the timeout methods always wait until every underlying resource allocated in the fiber has been correctly shut down. To avoid waiting, we can use timeoutAndForget. Once invoked, it returns control to the call site right away and runs the finalizers asynchronously. Keep in mind that if the finalizers never complete, the fiber will leak.
Structured concurrency
A fiber that is neither joined nor canceled runs in parallel with the call site fiber. It finishes only when its computations complete or throw an error. If the computation on the fiber never terminates, neither will the fiber.
val task = for {
time <- IO.realTimeInstant
_ <- IO.println(f"Current date and time is $time")
_ <- IO.sleep(1.second)
} yield ()
for {
_ <- task.foreverM.start //task is running as an infinite loop
//and fiber is never canceled
_ <- IO.sleep(5.seconds)
_ <- IO.println("Bye")
} yield ()
This may result in resource leaks. Fibers are quite lightweight, but spawning enough of them can still exhaust the available memory.
start and join and similar methods are CE’s concurrency primitives. They are useful, but should be handled with care. A little carelessness is enough to leak resources or end up with deadlocks.
To avoid leaks, we need to either cancel unneeded fibers explicitly or rely on safer methods such as background. background ties the fiber’s lifecycle to the Resource datatype. As soon as the resource is finalized, the underlying fiber is canceled too.
val backgroundTask: Resource[IO, Unit] = task
.onCancel(IO.println("Closing the background task"))
.background
.void
backgroundTask.surround {
for {
_ <- IO.sleep(5.seconds)
_ <- IO.println("Bye") //after foreground task is complete
//the background fiber will be canceled
} yield ()
}
The fiber’s state follows the syntactic structure of the code. Managing fiber lifecycles this way is known as structured concurrency.
CE offers one more structured concurrency tool: Supervisor. A supervisor is a resource capable of spawning fibers whose lifecycle is tied to its own. When the supervisor is finalized, it also ensures that all of its child fibers are terminated.
Supervisor has two modes, selected by passing the await flag when creating the resource. With await set to false, the supervisor cancels all of its live threads when it shuts down; this is the default. With await set to true, the supervisor’s termination blocks until every fiber has finished.
Supervisor[IO](await = false).use { supervisor =>
for {
_ <- supervisor.supervise(
IO.println("A").andWait(1.second).foreverM
)
_ <- supervisor.supervise(
IO.println("B").andWait(2.second).foreverM
)
_ <- IO.sleep(5.seconds) //fibers will print A and B
//for 5 seconds and then terminate
} yield ()
}
Adding the import cats.effect.implicits.* import clause also brings the supervise extension method for IO into scope. It lets us call supervise in much the same way as we call start.
import cats.effect.implicits.*
Supervisor[IO](await = true).use { supervisor =>
IO.println("Hello")
.andWait(1.second)
.foreverM
.timeout(5.seconds)
.supervise(supervisor)
}
Concurrency operators
CE offers a number of handy higher-level abstractions for typical concurrency-related operations.
IO.race, for example, takes two IO, runs them in parallel and waits for whichever finishes first. The slower one gets canceled. race returns a result of type IO[Either[A, B]].
val raced: IO[Either[String, Int]] = IO.race(
IO("Faster").delayBy(5.seconds),
IO(999).delayBy(10.seconds)
)
IO.racePair gives us finer control over what happens to the losing fiber. It returns the Outcome of the winning task together with the Fiber object instance, which we can use to cancel or join the slower fiber. At first glance the return type may look somewhat convoluted: Either[(OutcomeIO[A], FiberIO[B]), (FiberIO[A, OutcomeIO[B])]. When the left task wins, we receive the left value of the Either: a tuple with the left fiber’s outcome and a handle to the right fiber. When the right one wins, we receive the right value, this time holding the right fiber’s outcome and a handle to the left one.
IO.racePair(IO(10), IO(5).delayBy(2.seconds)).flatMap {
case Left((resL, fibR)) =>
fibR.cancel *> IO(s"Left fiber outcome: $resL")
case Right((fibL, resR)) =>
fibL.cancel *> IO(s"Right fiber outcome: $resR")
}
When two computations need to run in parallel and we have to wait for both to finish, there are a few options. IO.both returns a tuple with both results: IO[(A, B)]. Should either operation be canceled or fail, the other one is canceled as well.
IO.both(IO(1).delayBy(1.second), IO("Hello"))
IO.bothOutcome does not stop the other fiber, even when the first one gets canceled. It returns a tuple with the outcomes of both fibers.
The Cats library offers a range of operators for running effects in parallel. Because they are implemented as extension methods, using them requires the right imports to bring the appropriate implicits into scope.
import cats.syntax.all.*
parTupled is quite close to IO.both. It is called on a tuple of two IO and runs both of them in parallel, producing a result of type IO[(A, B)]. Its semantics match those of tupled, except that tupled runs the effects one after another.
To run any number of effects concurrently, we can reach for the parallel versions of traverse and sequence: parTraverse and parSequence.
val r: IO[List[Int]] = List(
IO(1).delayBy(1.second),
IO(2).delayBy(2.seconds),
IO(3).delayBy(3.seconds)
).parSequence
The parTraverseN and parSequenceN variants let us cap how many tasks run simultaneously. They take an extra Int parameter defining the maximum parallelism.
By default, parTraverse and parSequence both keep the tasks in order, which may introduce a slight performance overhead. parTraverseUnordered and parSequenceUnordered, by contrast, don’t preserve order, so they may run faster.
Finally, there are two handy symbolic operators: &> and <&. They, too, run two effects in parallel. If either IO fails, the other is canceled, and if the entire computation is canceled, both actions are canceled. &> returns the right-hand value, and <& the left-hand one.
//we'll get 1 after approx. 2s
IO(1).delayBy(1.second) <& IO(2).delayBy(2.seconds)
These operators can be seen as parallel equivalents of the sequential *>, <*, << and >>.
Summary
The aim of this article is to give a beginner-level introduction to the concurrency features available in Cats Effect. There is a lot more to CE than what is covered here. For example, I left out the powerful data structures built for managing concurrent access to data, such as Ref or Queue. That might make a good topic for a separate article. I hope you found this post useful. Take care!