Caterpillar

Let's talk about Rust today, because it has ad-hoc effects, no principled effect system, and has real problems because of that. Here's a rather simple example that involves a higher-order function:

let name = names().find(|name| name.is_available());

Okay, so names is a function that returns some kind of Iterator. We use find and an anonymous function that we pass to it, to find the first name that is available. All good!

But what if is_available can fail? Maybe it connects to a database? Well, find expects its argument to return a bool, so the best we can do is panic (another effect!) in there, which is not always desirable. Fortunately, there is try_find:

let name = names().try_find(|name| name.is_available())?;

Now is_available can return a Result, which try_find passes on, for us to handle. Again, all good! Except, maybe, that try_find is not available in stable versions. And note how we needed a completely different method to deal with this slightly different scenario.

But whatever, let's take a look at what would happen instead, if is_available is async. It could call a database, remember? Now using Iterator is no longer an option, because that is inherently synchronous. We need an asynchronous iterator, which exists in the form of Stream.

let name = stream::iter(names())
    .filter(|name| name.is_available())
    .next()
    .await;

Okay, so we convert our iterator into a stream. Then we find out that, annoyingly, Stream (or StreamExt, to be precise) does not have a find method. But we can emulate that with filter and next. (Please note that I left code that handles pinning out of this example and the next one. It would be slightly more complicated and not relevant to the topic at hand.)

So we needed a completely new API to deal with async, but other than that, all good? Not quite! If is_available talks to a database, then it's unrealistic that it wouldn't be async and fallible. How can we deal with that?

let name = stream::iter(names())
    .filter_map(|name| async move {
        name.is_available()
            .await
            .map(|is_available| is_available.then_some(name))
            .transpose()
    })
    .next()
    .await
    .transpose()?;

As you might have guessed, yes, we needed even more methods and do a whole song and dance. Now, I'm not claiming that this is the best way to do it (it's just the best I could come up with). But my point is this: The filter method from my initial example is not able to abstract over its argument being fallible, or async, or both. In Rust, it's impossible to write a function that does.

Starting tomorrow, let's speculate about how this could be much better, in a language with effects as a first-class citizen!