sunfishcode's blog
A blog by sunfishcode

Capabilities and Filesystems

Posted on

When we first designed the filesystem API in WASI, we had a lot of questions. When a Wasm module has an import, what does that mean? How are import names resolved? What do we know about the state of the world outside of what the spec calls the "store"? Capabilities are pretty cool; what's the best way to incorporate capabilities into filesystem APIs?

At the time, we mainly wanted to keep our options open and avoid hard-coding ambient authority anywhere, because we knew from other systems that retrofitting capability-based security onto an ecosystem designed without it can be prohibitively difficult. So we followed the example of CloudABI, which in turn draws a lot from Capsicum, which combine filesystem APIs and capabilities in practical systems.

Today, we've learned a lot about how WASI is being used, and about what imports are and how they're resolved in the component model, and how the overall security model of the component model works. And with this knowledge, it's worth taking a high-level look at WASI's filesystem API to see if we can make something that works with the tools we have today and fits with the way people want to use it today.

WASI's current design

In WASI's current design, all filesystem API functions require a handle value to be passed to them as an argument. Instead of having a plain open function which takes a string and resolves it in some implied namespace, there's just an openat-like method in the descriptor resource, which takes a descriptor as its receiver parameter.

    resource descriptor {
        ...

        open-at: func(
            /// Flags determining the method of how the path is resolved.
            path-flags: path-flags,
            /// The relative path of the object to open.
            path: string,
            /// The method by which to open the file.
            open-flags: open-flags,
            /// Flags to use for the resulting descriptor.
            %flags: descriptor-flags,
        ) -> result<descriptor, error-code>;

        ...
    }

The implied "self" argument handle identifies a directory tree, and the path string is resolved as a relative path within that tree. Programs are then given an initial set of handles to work with via the "preopens" mechanism.

Unlike in POSIX, this openat-style function enforces sandboxing, preventing absolute paths, or .. or symlinks which would reference files or directories outside of the handle's namespace.

All these ideas come directly from CloudABI and indirectly from Capsicum, and similar ideas were being adopted in Google's Fuchsia OS.

And to be sure, there are reasons why this approach makes sense, for security, modularity, and portability. The areas around the roots of filesystems tend have higher concentrations of security and portability hazards. Roots are where many "interesting" directories like /etc, /tmp, /proc, and C:/Windows live. If we desire secure and portable programs, we need to avoid them being especially aware of those directories. Prohibiting absolute paths at the WASI level strongly resonates with that desire.

And while we knew that banning absolute paths would break a lot of things, we had the idea to add compatibility back in, using the techniques of libpreopen. The basic idea is to emulate absolute paths in libc, while still avoiding them at the WASI level.

The cool thing about this approach is that, in theory, it achieves the best of both worlds. We get compatibility for existing code and toolchains when we want that, and we also get the option to have programs that don't use those compatibility layers and in return get stronger security guarantees.

Another theoretical win of this approach is that it avoids baking the concept of a "current working directory" into the system. The current working directory is basically a file descriptor for a directory, except that unlike normal file descriptors, this file descriptor is implicit. In Unix, this means it needs its own special system calls like chdir to do what open does, and fchdir to do what dup2 does. So it has arbitrary restrictions, anywhere the specialized system calls don't cover things that the general system calls can do. And, this implied file descriptor is per-process, making it inflexible for multi-threaded applications. So instead, we kept the current working directory concept out of WASI and aimed for an approach of emulating it in libc instead.

This overall design means that Wasm toolchains and libraries can chose for themselves whether to use the compatibility bridges if they need that functionality, or to lean into the capability model and avoid having a filesystem root or current working directory imposed on them if they don't.

How it's going

This approach basically works, but it has several papercuts.

It's less familiar. Practically everyone is familiar with traditional filesystems and absolute paths and current working directories. And even though Unix and Windows have concepts of directory handles, they're not widely used, so not everyone is familiar with them.

And it's less compatible. Practically every source language has a standard library with an open function that takes a string and opens a file in an implied namespace. WASI's current approach requires all source languages to use libc, or to do their own emulation. And lots of programs end up depending on absolute paths, or symlinks to absolute paths, or the ability to canonicalize paths into absolute paths, and WASI's current approach makes all that awkward.

And, there still are places where things don't work quite right. In theory some of these are fixable by doing more work in libc, but some are not.

And unlike on Unix where one can often get away by assuming that everyone will be using libc, on Wasm, many toolchains are not using libc. And indeed, we don't want every toolchain to have to use libc.

New tools

When we started looking at capability-based security, one of the big ideas is to avoid ambient authority.

But what we know now is, that there is no such thing as purely ambient authority. Any system can be virtualized, such that any ambient authority can be sandboxed and redirected. It's not always cheap or easy to do so, but it's always doable. So fundamentally, ambient authority is always a question of degree and granularity rather than being an absolute.

So instead of just saying "ambient authority bad", we need to look at systems and see the degree to which they achieve the Principle of Least Authority (PoLA) in practice, and analyze what forces help or hinder it.

In Wasm, let's imagine a component that imports a function named do-stuff, with no arguments. This function has the appearance of using ambient authority, because it's just a function with no handle arguments. It would seem that anyone can call it anywhere.

   do-stuff: func();

However, Wasm components don't have a global scope. It's not true that anyone can call it anywhere. It has to be linked to something, at link time. And it doesn't need to be linked to the same thing in every instance that needs it. So whoever controls the linking process can chose to link this do-stuff import to a do-stuff export of their choosing. They can even link it to a do-stuff wrapper around another do-stuff implementation. That means it can be virtualized or attenuated or refused entirely.

The fact that there are no handle values being passed in as arguments just means that instead of a runtime capability, this do-stuff function provides a link-time capability. That might have consequences for granularity, but it'll depend on how it gets used. So instead of calling it ambient authority and prohibiting it, we can consider it, and think instead about how it'll get used in practice.

Type safety and granularity

One of the reasons for wanting to avoid filesystem roots is that rooted filesystem namespaces tend to bundle up many different logical resources into a single namespace, which means the granularity of access granted by that namespace is very coarse.

We could push for finer-grained filesystem capabilities, however there are options at the conceptual level. Instead of exposing new capabilities through filesystem APIs, we'd really rather be exposing new capabilities through dedicated APIs using resources, because that's much easier to make fine-grained, typed, and robust.

So if we push toward more typed APIs, then we can let WASI's filesystem API just focus on providing access to files, and not be burdened with the pressure to be finer-grained.

And there's one additional piece of this puzzle. What if make something like do-stuff use link-time authority, and then some time in the future decide that we really need it to use runtime authority? For example, suppose some program needs to have the ability to call two different do-stuff backends, deciding between them at runtime. If we were designing a new API, that'd be a use case for handles. But if we didn't use handles at the start, are we stuck, at least until we can do a semver break?

This is likely a ways off in the future, but we now at least have an idea for a future component-model feature, where we allow component instances to implement resources. We could then have handles to component instances, and the instance exports could satisfy the require methods of the resource type. If we had that feature, then any interface with link-time authority functions could be converted into an interface with runtime authority functions. This possible future path gives us more confidence that we'll be able to evolve in the future when we need to.

What does this look like in wasi-filesystem?

We could add a plain open function, which takes a path argument and resolves it relative to a filesystem root that comes with the instance that the function is imported from. That makes this open function use a link-time capability. It could support absolute paths, symlinks to absolute paths, and all the rest. And, it could support a current working directory concept.

And then, we could add similar non--at versions of all the -at functions.

This new API could live in a new interface that could coexist with the current interface.

Now, I believe we still don't want to be defaulting to exposing the entire host filesystem to WASI programs. So we'll still want to have a simple VFS system in Wasm engines, with roughly the same level of virtual "mounting" that today's preopens provide, just using a link-time authority namespace instead of actual preopen handles.

What about ghosts?

Ghosts are when logical resources are identified by integers or strings or similar things, across code boundaries, and "ghosts" conceptually relay the resources needed to resolve the references. In the case of filesystem APIs, ghosts are present in the form of filesystem paths being passed around, which requires any code accepting such a path to have access to the same filesystem namespace.

If a component ecosystem grows up where it's common to use filesystem paths to identify resources between components, rather than handles, the ecosystem becomes less composable and, has weakened security properties, and requires more expensive virtualization.

If we add an open function, there is a greater risk that the WASI ecosystem will acquire some ghosts, in the form of components passing around filenames and assuming a common filsystem namespace, making them more expensive and complex to virtualize.

Now that we have resources though, we can see that they're relatively easy to use, which makes me more optimistic that we don't have to worry as much about people using filesystem paths in place of handles. This is something we can continue to monitor as the ecosystem grows. For now, I think we can be confident enough that we don't need to preemptively worry about it.

Conclusion

With the new tools and understandings in Wasm components, I think we can re-evaluate how wasi-filesystem works, to make it more familiar and more compatible with existing tools, libraries, and applications.