Can't watch multiple paths in fsevents

[jazzdan]

Which version of macOS are you using?

ProductName:	Mac OS X
ProductVersion:	10.13.4
BuildVersion:	17E199

Please describe the issue that occurred.

When there are multiple paths in the Path slice in the EventStream struct calling Start() causes an error to be printed from fsevents itself:

2018-04-30 13:55 main[37509] (FSEvents.framework) FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)

Are you able to reproduce the issue? Please provide steps to reproduce and a code sample if possible.

Here's a version of example/main.go which illustrates this problem:

// +build darwin

package main

import (
	"bufio"
	"fmt"
	"io/ioutil"
	"log"
	"os"
	"runtime"
	"time"

	"github.com/fsnotify/fsevents"
)

func main() {
	path, err := ioutil.TempDir("", "fsexample")
	if err != nil {
		log.Fatalf("Failed to create TempDir: %v", err)
	}
	fmt.Println(path)
	path2, err := ioutil.TempDir("", "fsexample2")
	if err != nil {
		log.Fatalf("Failed to create TempDir: %v", err)
	}
	fmt.Println(path2)
	dev, err := fsevents.DeviceForPath(path)
	if err != nil {
		log.Fatalf("Failed to retrieve device for path: %v", err)
	}
	log.Print(dev)
	log.Println(fsevents.EventIDForDeviceBeforeTime(dev, time.Now()))

	es := &fsevents.EventStream{
		Paths:   []string{path, path2},
		Latency: 500 * time.Millisecond,
		Device:  dev,
		Flags:   fsevents.FileEvents | fsevents.WatchRoot}
	es.Start()
	ec := es.Events

	log.Println("Device UUID", fsevents.GetDeviceUUID(dev))

	go func() {
		for msg := range ec {
			for _, event := range msg {
				logEvent(event)
			}
		}
	}()

	in := bufio.NewReader(os.Stdin)

	if false {
		log.Print("Started, press enter to GC")
		in.ReadString('\n')
		runtime.GC()
		log.Print("GC'd, press enter to quit")
		in.ReadString('\n')
	} else {
		log.Print("Started, press enter to stop")
		in.ReadString('\n')
		es.Stop()

		log.Print("Stopped, press enter to restart")
		in.ReadString('\n')
		es.Resume = true
		es.Start()

		log.Print("Restarted, press enter to quit")
		in.ReadString('\n')
		es.Stop()
	}
}

var noteDescription = map[fsevents.EventFlags]string{
	fsevents.MustScanSubDirs: "MustScanSubdirs",
	fsevents.UserDropped:     "UserDropped",
	fsevents.KernelDropped:   "KernelDropped",
	fsevents.EventIDsWrapped: "EventIDsWrapped",
	fsevents.HistoryDone:     "HistoryDone",
	fsevents.RootChanged:     "RootChanged",
	fsevents.Mount:           "Mount",
	fsevents.Unmount:         "Unmount",

	fsevents.ItemCreated:       "Created",
	fsevents.ItemRemoved:       "Removed",
	fsevents.ItemInodeMetaMod:  "InodeMetaMod",
	fsevents.ItemRenamed:       "Renamed",
	fsevents.ItemModified:      "Modified",
	fsevents.ItemFinderInfoMod: "FinderInfoMod",
	fsevents.ItemChangeOwner:   "ChangeOwner",
	fsevents.ItemXattrMod:      "XAttrMod",
	fsevents.ItemIsFile:        "IsFile",
	fsevents.ItemIsDir:         "IsDir",
	fsevents.ItemIsSymlink:     "IsSymLink",
}

func logEvent(event fsevents.Event) {
	note := ""
	for bit, description := range noteDescription {
		if event.Flags&bit == bit {
			note += description + " "
		}
	}
	log.Printf("EventID: %d Path: %s Flags: %s", event.ID, event.Path, note)
}

Running this results in:

[~/go/src/github.com/fsnotify/fsevents/example]> ./main
/var/folders/6p/7hwgf0d92_3bckf61qbwz1n80000gp/T/fsexample985416144
/var/folders/6p/7hwgf0d92_3bckf61qbwz1n80000gp/T/fsexample2184687343
2018/04/30 14:03:40 16777224
2018/04/30 14:03:40 69444979
2018-04-30 14:03 main[38203] (FSEvents.framework) FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)
2018/04/30 14:03:40 Device UUID 32F97B79-2B30-4389-A5C1-C2586D6EB0E6
2018/04/30 14:03:40 Started, press enter to stop

2018/04/30 14:03:41 Stopped, press enter to restart

2018-04-30 14:03 main[38203] (FSEvents.framework) FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)
2018/04/30 14:03:42 Restarted, press enter to quit

Any events produced in either of these directories are not reported like they are if you're only watching one directory.

I'm running this on an iMac with a fusion drive, which is the only machine-specific thing I can think of which is causing this. I imagine though that a fusion drive appears as one device as far as FSEvents is concerned?

I'm also unsure which path you are supposed to use to look up the device.

[jazzdan]

Many usages of FSEventStreamStart on GitHub seem to just pass an array of one thing, including https://github.com/rjeczalik/notify and https://github.com/strongloop/fsevents. This makes me think that this isn't supported by FSEvents at all, but the documentation seems to indicate that it is supported:

pathsToWatch

A CFArray of CFStringRefs, each specifying a path to a directory, signifying the root of a filesystem hierarchy to be watched for modifications.

[jazzdan]

Hmm, this looks like a "known" issue?

https://facebook.github.io/watchman/docs/troubleshooting.html#fseventstreamstart-register_with_server-error-f2d_register_rpc--null--21

[nicks]

for what it's worth, it seems like this only happen when a Device is specified on EventStream. if you use per-host streams (rather than per-disk streams), everything works ok.

[nathany]

@jazzdan Thanks for reporting this issue and doing the research.

@nicks We need #38 for host mode though, correct?

If this limitation isn't going away, I do wonder how to present the API to make it clear.