Thinking about streams in WASI
Posted on
Streams are an essential feature of any large modular system. Whether they're in the form of channels, pipes, sockets, iterators, async coroutines, or other things, streams represent sequences of data over time, and can connect two pieces of software with minimal coordination.
WASI is being designed to support large modular systems, so WASI needs a solid stream design. This is a blog post about streams in WASI.
WASIp2 has been out a while, so we can now look at some experience with it in practice. WASIp3 is a ways in the future yet, so we have some time to think about what we want to do differently. So let's take a look!
WASIp2 streams
WASIp2 streams have overall worked fairly well, and are used in APIs like wasi-filesystem, wasi-sockets, and wasi-http, providing a common interface for streaming functionality. However, some of their big ideas are limited in practice by the lack of integrated async.
To block or not to block
WASIp2 has input-stream and output-stream types, in wasi-io.
These have pretty ordinary bytestream APIs, although they do do one thing
that's different from what people may expect: Instead of being simply
blocking, or simply non-blocking, or even blocking-or-not depending on a
dynamic flag (O_NONBLOCK) as in POSIX, WASIp2 streams have separate
functions for doing blocking and nonblocking I/O. The goal of this is to
fix the composition problem.
A lot of people have heard of the "what color is your function?"
blog post. In a sense, streams that are blocking or non-blocking also have
different "colors" which can present barriers to composition. Pass a
non-blocking stream in as the stdin of grep in Unix and you get this:
grep: (standard input): Resource temporarily unavailable
So Unix's grep needs a stream with the "blocking" color. Pass it the wrong
color and it doesn't work.
The observation behind WASI's streams is that one typically has to write different code to work with using blocking versus non-blocking streams. It's not something that can be meaningfully switched at runtime based on a flag. So WASI has separate API calls for blocking and non-blocking, so programmers can just request what they want, and the stream can work in that way without "color" being a property of the stream.
It's a good idea in theory, though in WASIp2 it is somewhat hampered by two
issues. One is that the output-stream API ended up being a little more
cluttered than one might have wanted. The flush concept, in particular, is
something I hope we can improve in WASIp3.
The other issue is that composability in WASI for anything involving I/O is
limited by WASI's lack of async features. The poll function needs to know
about all types of things that might be polled for, which means that virtualizing
any API that involves I/O requires virtualizing the poll function, which then
requires virtualizing everything else that uses I/O at the same time.
It is possible to virtualize things, and tools like WASI-virt do so, but the virtualization ends up having to completely wrap everything, rather than being able to virtualize individual APIs individually.
And, input-stream is a different type from output-stream, so it's not possible
to directly connect stream output from one place to stream input to another.
And the reason they have to be separate types is that WASIp2 has no way to suspend
the writer of a stream while the reader is reading. One can connect readers and
writers in various ways using custom code in the host, but it can't be done
from within WASI.
Fixing all these issues properly really needs async features built into the platform.
Fortunately, it was already clear from the beginning that async features would be
critical for many use cases, so there's already a plan to follow up WASIp2 with WASIp3
to add async as a headline feature. The idea is to replace poll, input-stream,
output-stream, and all the rest of wasi-io.
Streams versus files
From a POSIX perspective, the other big difference between WASIp2 streams and POSIX streams is they WASIp2 doesn't have the "file is-a stream" property. Streams are separate resources from files. You can request streams for reading from or writing to files. And similar for sockets and other things. Libraries like libc may hide this, but this is what's happening at the WASI level.
I blogged about some of the reasons for doing this in "What does everything is a file do?". Long story short, decoupling streams from files means better composition because we can compose anything that works in terms of stream types without worrying about whether they actually need to be file streams.
These ideas will likely be more important in WASIp3, once we get more powerful composition with async. Having them in WASIp2 has made things like supporting libc more complex, and has made it less familiar to people familiar with POSIX. At the same time, it has also been a step helping us get ready for WASIp3 where these ideas will have more advantages.
Several people have observed that these changes aren't particularly useful if everyone is just going to use libc, which hides them. But the reason they make sense for WASI is that in Wasm, not everything will want to use libc. In the "native" world, people are often accustomed to thinking of C as being the only thing that really matters, with all other languages simply being ways of producing the kinds of programs that C could produce, and communicating with each other and the OS as if they were C programs. But Wasm from the beginning has been on a path of supporting other languages, including adding features like Garbage Collection to the underlying platform. So to meet the needs of Wasm, WASI needs to address languages that don't use libc. That does make some things more complex for people who only care about C, but the tradeoff is we get to aim for a more unified ecosystem.
Also, even for C and C-like languages, WASI covers many areas not covered by traditional libc APIs, such as HTTP, and in these areas, users will more often be using new APIs or new frameworks.
WASIp3 streams
The biggest feature of WASIp3 is expected to be the addition of async support.
Luke Wagner's talk on async in the Component Model is a good introduction, and discusses how WASI can fix the function coloring problem. Because Wasm VMs are in control of the callstack, they'll be able to do stack switching as needed to do things like bridge between async code calling sync functions. And this is also the way they'll be able to bridge between blocking and non-blocking streams. That talk goes into detail on a lot of material; here, I'll just highlight a few aspects of how it affects streams.
When async is a builtin feature of the platform, it enables a lot of different possibilities. Imagine being able to link together code written in different languages, in the same process on the same thread and using a single underlying async event loop. That's powerful. That's where this is going.
With my standards hat on, none of what I'm describing here is in an official proposal yet, but these general ideas are shared by several involved parties. If you have thoughts or questions or concerns or suggestions, please reach out and let's talk!
A unified stream type
Some of the ideas can look surprising at first, especially if you're like me
and have an existing intuition based on existing platforms that are not designed
around async. For example, with async we can have a single stream type, rather
than having separate input-stream and output-stream types. It's still
unidirectional, but it works as an input stream when it appears as a function
argument, and works as an output stream when it appears as a function return
value. In effect, streams are less like first-class handles that can be
passed around anywhere, and more like coroutine enables, that simply mediate
between a caller and a coroutine callee that coexist and have a stream of values
flowing between them.
For example, consider this signature for a hypothetical gzip function:
gzip: func(input: stream<u8>) -> stream<u8>
The argument is a stream<u8>, which is a bytestream. Because it's an argument,
it's an input stream that the function can read from. The return type is
also stream<u8>. Being a return value, it's an output stream that the function
can create and write to.
Because it's the same type on input and output , it could compose with other functions using ordinary function composition. For example, if we also have this:
gunzip: func(input: stream<u8>) -> stream<u8>
Then we could do compositions like gzip(gunzip(data)) to decompress and
recompress the data (as a silly example).
These ideas aren't new. For example, gRPC also has a single conceptual
stream type that works as input when it's an argument and output when it's
a return value.
Nevertheless, these ideas are things I've had to take time to understand, as I'm more accustomed to thinking of input and output streams as being independent handles (file descriptors) that can be passed around, and these kinds of streams work in different ways. I've explored questions like how does error handling work and should closing an output stream be allowed to fail? and what happens if someone passed you an output-stream and then ... and where do output-stream errors go?. One big-picture takeaway, every time I think "what if someone passes you an output stream and then...", I find that the answer is that I need to step back a level and think about the system in a broader way.
My posts here are a start on exploring these topics, though clearly much more will be needed as WASIp3 progresses. And WASI will need bridges to connect with existing languages and libraries that work in terms of channel-like concepts, or other streaming concepts.
But put all these pieces together, and we get the most plausible path
I'm aware of having been proposed to an ecosystem with a single stream type
that connects all the streaming things, which seems like a valuable goal
to pursue for WASI.
Stream of T
Another advantage of being a builtin type is that the stream type can
more easily be parameterized on the data type. Bytestreams can do a lot,
but with a rich type system with things like resource handles, being
able to pass handles over a stream will be a powerful connecting primitive.
From a Unix perspective, streams-of-handles is much like passing file descriptors over a Unix-domain socket, but hopefully with a much less complex API.
From a higher-level language perspective, streams of typed values look a lot like iterators.
Summary
WASIp2 streams are here today. They are a transitional step between WASIp1-style Unix streams and WASIp3's coroutine-style streams. But because WASIp2 lacks integrated async, they have some limitations.
WAISp3 will add integrated async, and make streams much more flexible and powerful.