Go Fundamentals - Sample
Table of Contents
Chapter 14.12: Using the FS Interface
Previously, we had been using filepath.WalkDir to walk the directory tree. This worked by walking the file system directly. As a result, we have to make sure that the file system is in the correct state before we can use it. As we have seen, this can cause a lot of setup work to be done. Either we have to keep a completely different folders for each test scenario we have, or we have to create all of the files and folders at the start of the test before we begin. This is a lot of work, and is often error prone.
$ go doc fs.WalkDir
package fs // import "io/fs"
func WalkDir(fsys FS, root string, fn WalkDirFunc) error
WalkDir walks the file tree rooted at root, calling fn for each file or
directory in the tree, including root.
All errors that arise visiting files and directories are filtered by fn:
see the fs.WalkDirFunc documentation for details.
The files are walked in lexical order, which makes the output deterministic
but requires WalkDir to read an entire directory into memory before
proceeding to walk that directory.
WalkDir does not follow symbolic links found in directories, but if root
itself is a symbolic link, its target will be walked.
--------------------------------------------------------------------------------
Go Version: go1.22.0
fs.WalkDir function.If we use the fs.WalkDir function, Listing 14.1, instead, which takes, as its first argument, a fs.FS implementation. Instead of walking the file system the fs.FS is walked instead.
func Walk(cab fs.FS) ([]string, error) {
var entries []string
err := fs.WalkDir(cab, ".", func(path string, d fs.DirEntry, err error) error {
// if there was an error, return it
// if there is an error, it is most likely
// because an error was an encoutered trying
// to read the top level directory
if err != nil {
return err
}
// if the entry is a directory, handle it
if d.IsDir() {
// name of the file or directory
name := d.Name()
// if the directory is a dot return nil
// this may be the root directory
if name == "." || name == ".." {
return nil
}
// if the directory name is "testdata"
// or it starts with "."
// or it starts with "_"
// then return filepath.SkipDir
if name == "testdata" || strings.HasPrefix(name, ".") || strings.HasPrefix(name, "_") {
return fs.SkipDir
}
return nil
}
// append the entry to the list
entries = append(entries, path)
// return nil to tell walk to continue
return nil
})
return entries, err
}fs.WalkDir function.In Listing 14.2 we can continue to use the same code inside of the fs.WalkFunc function as we did before. Our test code only needs two small changes now to make it work. The first is we an implementation of the fs.FS interface to pass to the Walk function. For now, we can use the os.DirFS function, Listing 14.3. The implementation will be backed directly by the file system.
$ go doc os.DirFS
package os // import "os"
func DirFS(dir string) fs.FS
DirFS returns a file system (an fs.FS) for the tree of files rooted at the
directory dir.
Note that DirFS("/prefix") only guarantees that the Open calls
it makes to the operating system will begin with "/prefix":
DirFS("/prefix").Open("file") is the same as os.Open("/prefix/file").
So if /prefix/file is a symbolic link pointing outside the /prefix tree,
then using DirFS does not stop the access any more than using os.Open does.
Additionally, the root of the fs.FS returned for a relative path,
DirFS("prefix"), will be affected by later calls to Chdir. DirFS is
therefore not a general substitute for a chroot-style security mechanism
when the directory tree contains arbitrary content.
The directory dir must not be "".
The result implements io/fs.StatFS, io/fs.ReadFileFS and io/fs.ReadDirFS.
--------------------------------------------------------------------------------
Go Version: go1.22.0
os.DirFS function.File Paths
The other change we need to make is the expected paths. Before, we were expecting paths such as /data/a.txt returned from our Walk function. However, when working with fs.FS implementations, the paths returned are relative to the root of the implementation. In this case, we are using os.DirFS("data") to create the fs.FS implementation. This places data at the root of the file system implementation and paths returned from the Walk function will be relative to this root.
Note: Paths are expected to use
/as the path separator, regardless of the operating system.
We need to update our test code to for the relative paths, a.txt, to be returned instead of /data/a.txt.
func Test_Walk(t *testing.T) {
t.Parallel()
exp := []string{
"a.txt",
"b.txt",
"e/f/g.txt",
"e/f/h.txt",
"e/j.txt",
}
cab := os.DirFS("data")
act, err := Walk(cab)
if err != nil {
t.Fatal(err)
}
es := strings.Join(exp, ", ")
as := strings.Join(act, ", ")
if es != as {
t.Fatalf("expected %s, got %s", es, as)
}
}$ go test -v
=== RUN Test_Walk
=== PAUSE Test_Walk
=== CONT Test_Walk
--- PASS: Test_Walk (0.00s)
PASS
ok demo 0.952s
--------------------------------------------------------------------------------
Go Version: go1.22.0
Walk function.As we can see from the test output in Listing 14.4, we have successfully updated our code to use the fs.FS interface.