bump dependencies
Some checks failed
Go Build / build (push) Has been cancelled

This commit is contained in:
raf 2025-01-10 16:21:32 +03:00
commit 8e7365b93f
No known key found for this signature in database
GPG key ID: EED98D11B85A2819
212 changed files with 13881 additions and 9534 deletions

View file

@ -1,7 +1,7 @@
freebsd_task:
name: 'FreeBSD'
freebsd_instance:
image_family: freebsd-13-2
image_family: freebsd-14-1
install_script:
- pkg update -f
- pkg install -y go
@ -9,5 +9,6 @@ freebsd_task:
# run tests as user "cirrus" instead of root
- pw useradd cirrus -m
- chown -R cirrus:cirrus .
- FSNOTIFY_BUFFER=4096 sudo --preserve-env=FSNOTIFY_BUFFER -u cirrus go test -parallel 1 -race ./...
- sudo --preserve-env=FSNOTIFY_BUFFER -u cirrus go test -parallel 1 -race ./...
- FSNOTIFY_BUFFER=4096 sudo --preserve-env=FSNOTIFY_BUFFER -u cirrus go test -parallel 1 -race ./...
- sudo --preserve-env=FSNOTIFY_BUFFER -u cirrus go test -parallel 1 -race ./...
- FSNOTIFY_DEBUG=1 sudo --preserve-env=FSNOTIFY_BUFFER -u cirrus go test -parallel 1 -race -v ./...

View file

@ -1 +0,0 @@
go.sum linguist-generated

View file

@ -5,3 +5,6 @@
# Output of go build ./cmd/fsnotify
/fsnotify
/fsnotify.exe
/test/kqueue
/test/a.out

View file

@ -1,8 +1,36 @@
# Changelog
Unreleased
----------
Nothing yet.
1.8.0 2023-10-31
----------------
### Additions
- all: add `FSNOTIFY_DEBUG` to print debug logs to stderr ([#619])
### Changes and fixes
- windows: fix behaviour of `WatchList()` to be consistent with other platforms ([#610])
- kqueue: ignore events with Ident=0 ([#590])
- kqueue: set O_CLOEXEC to prevent passing file descriptors to children ([#617])
- kqueue: emit events as "/path/dir/file" instead of "path/link/file" when watching a symlink ([#625])
- inotify: don't send event for IN_DELETE_SELF when also watching the parent ([#620])
- inotify: fix panic when calling Remove() in a goroutine ([#650])
- fen: allow watching subdirectories of watched directories ([#621])
[#590]: https://github.com/fsnotify/fsnotify/pull/590
[#610]: https://github.com/fsnotify/fsnotify/pull/610
[#617]: https://github.com/fsnotify/fsnotify/pull/617
[#619]: https://github.com/fsnotify/fsnotify/pull/619
[#620]: https://github.com/fsnotify/fsnotify/pull/620
[#621]: https://github.com/fsnotify/fsnotify/pull/621
[#625]: https://github.com/fsnotify/fsnotify/pull/625
[#650]: https://github.com/fsnotify/fsnotify/pull/650
1.7.0 - 2023-10-22
------------------

View file

@ -1,7 +1,7 @@
Thank you for your interest in contributing to fsnotify! We try to review and
merge PRs in a reasonable timeframe, but please be aware that:
- To avoid "wasted" work, please discus changes on the issue tracker first. You
- To avoid "wasted" work, please discuss changes on the issue tracker first. You
can just send PRs, but they may end up being rejected for one reason or the
other.
@ -20,6 +20,124 @@ platforms. Testing different platforms locally can be done with something like
Use the `-short` flag to make the "stress test" run faster.
Writing new tests
-----------------
Scripts in the testdata directory allow creating test cases in a "shell-like"
syntax. The basic format is:
script
Output:
desired output
For example:
# Create a new empty file with some data.
watch /
echo data >/file
Output:
create /file
write /file
Just create a new file to add a new test; select which tests to run with
`-run TestScript/[path]`.
script
------
The script is a "shell-like" script:
cmd arg arg
Comments are supported with `#`:
# Comment
cmd arg arg # Comment
All operations are done in a temp directory; a path like "/foo" is rewritten to
"/tmp/TestFoo/foo".
Arguments can be quoted with `"` or `'`; there are no escapes and they're
functionally identical right now, but this may change in the future, so best to
assume shell-like rules.
touch "/file with spaces"
End-of-line escapes with `\` are not supported.
### Supported commands
watch path [ops] # Watch the path, reporting events for it. Nothing is
# watched by default. Optionally a list of ops can be
# given, as with AddWith(path, WithOps(...)).
unwatch path # Stop watching the path.
watchlist n # Assert watchlist length.
stop # Stop running the script; for debugging.
debug [yes/no] # Enable/disable FSNOTIFY_DEBUG (tests are run in
parallel by default, so -parallel=1 is probably a good
idea).
touch path
mkdir [-p] dir
ln -s target link # Only ln -s supported.
mkfifo path
mknod dev path
mv src dst
rm [-r] path
chmod mode path # Octal only
sleep time-in-ms
cat path # Read path (does nothing with the data; just reads it).
echo str >>path # Append "str" to "path".
echo str >path # Truncate "path" and write "str".
require reason # Skip the test if "reason" is true; "skip" and
skip reason # "require" behave identical; it supports both for
# readability. Possible reasons are:
#
# always Always skip this test.
# symlink Symlinks are supported (requires admin
# permissions on Windows).
# mkfifo Platform doesn't support FIFO named sockets.
# mknod Platform doesn't support device nodes.
output
------
After `Output:` the desired output is given; this is indented by convention, but
that's not required.
The format of that is:
# Comment
event path # Comment
system:
event path
system2:
event path
Every event is one line, and any whitespace between the event and path are
ignored. The path can optionally be surrounded in ". Anything after a "#" is
ignored.
Platform-specific tests can be added after GOOS; for example:
watch /
touch /file
Output:
# Tested if nothing else matches
create /file
# Windows-specific test.
windows:
write /file
You can specify multiple platforms with a comma (e.g. "windows, linux:").
"kqueue" is a shortcut for all kqueue systems (BSD, macOS).
[goon]: https://github.com/arp242/goon
[Vagrant]: https://www.vagrantup.com/

View file

@ -1,8 +1,8 @@
//go:build solaris
// +build solaris
// Note: the documentation on the Watcher type and methods is generated from
// mkdoc.zsh
// FEN backend for illumos (supported) and Solaris (untested, but should work).
//
// See port_create(3c) etc. for docs. https://www.illumos.org/man/3C/port_create
package fsnotify
@ -12,150 +12,33 @@ import (
"os"
"path/filepath"
"sync"
"time"
"github.com/fsnotify/fsnotify/internal"
"golang.org/x/sys/unix"
)
// Watcher watches a set of paths, delivering events on a channel.
//
// A watcher should not be copied (e.g. pass it by pointer, rather than by
// value).
//
// # Linux notes
//
// When a file is removed a Remove event won't be emitted until all file
// descriptors are closed, and deletes will always emit a Chmod. For example:
//
// fp := os.Open("file")
// os.Remove("file") // Triggers Chmod
// fp.Close() // Triggers Remove
//
// This is the event that inotify sends, so not much can be changed about this.
//
// The fs.inotify.max_user_watches sysctl variable specifies the upper limit
// for the number of watches per user, and fs.inotify.max_user_instances
// specifies the maximum number of inotify instances per user. Every Watcher you
// create is an "instance", and every path you add is a "watch".
//
// These are also exposed in /proc as /proc/sys/fs/inotify/max_user_watches and
// /proc/sys/fs/inotify/max_user_instances
//
// To increase them you can use sysctl or write the value to the /proc file:
//
// # Default values on Linux 5.18
// sysctl fs.inotify.max_user_watches=124983
// sysctl fs.inotify.max_user_instances=128
//
// To make the changes persist on reboot edit /etc/sysctl.conf or
// /usr/lib/sysctl.d/50-default.conf (details differ per Linux distro; check
// your distro's documentation):
//
// fs.inotify.max_user_watches=124983
// fs.inotify.max_user_instances=128
//
// Reaching the limit will result in a "no space left on device" or "too many open
// files" error.
//
// # kqueue notes (macOS, BSD)
//
// kqueue requires opening a file descriptor for every file that's being watched;
// so if you're watching a directory with five files then that's six file
// descriptors. You will run in to your system's "max open files" limit faster on
// these platforms.
//
// The sysctl variables kern.maxfiles and kern.maxfilesperproc can be used to
// control the maximum number of open files, as well as /etc/login.conf on BSD
// systems.
//
// # Windows notes
//
// Paths can be added as "C:\path\to\dir", but forward slashes
// ("C:/path/to/dir") will also work.
//
// When a watched directory is removed it will always send an event for the
// directory itself, but may not send events for all files in that directory.
// Sometimes it will send events for all times, sometimes it will send no
// events, and often only for some files.
//
// The default ReadDirectoryChangesW() buffer size is 64K, which is the largest
// value that is guaranteed to work with SMB filesystems. If you have many
// events in quick succession this may not be enough, and you will have to use
// [WithBufferSize] to increase the value.
type Watcher struct {
// Events sends the filesystem change events.
//
// fsnotify can send the following events; a "path" here can refer to a
// file, directory, symbolic link, or special file like a FIFO.
//
// fsnotify.Create A new path was created; this may be followed by one
// or more Write events if data also gets written to a
// file.
//
// fsnotify.Remove A path was removed.
//
// fsnotify.Rename A path was renamed. A rename is always sent with the
// old path as Event.Name, and a Create event will be
// sent with the new name. Renames are only sent for
// paths that are currently watched; e.g. moving an
// unmonitored file into a monitored directory will
// show up as just a Create. Similarly, renaming a file
// to outside a monitored directory will show up as
// only a Rename.
//
// fsnotify.Write A file or named pipe was written to. A Truncate will
// also trigger a Write. A single "write action"
// initiated by the user may show up as one or multiple
// writes, depending on when the system syncs things to
// disk. For example when compiling a large Go program
// you may get hundreds of Write events, and you may
// want to wait until you've stopped receiving them
// (see the dedup example in cmd/fsnotify).
//
// Some systems may send Write event for directories
// when the directory content changes.
//
// fsnotify.Chmod Attributes were changed. On Linux this is also sent
// when a file is removed (or more accurately, when a
// link to an inode is removed). On kqueue it's sent
// when a file is truncated. On Windows it's never
// sent.
type fen struct {
Events chan Event
// Errors sends any errors.
//
// ErrEventOverflow is used to indicate there are too many events:
//
// - inotify: There are too many queued events (fs.inotify.max_queued_events sysctl)
// - windows: The buffer size is too small; WithBufferSize() can be used to increase it.
// - kqueue, fen: Not used.
Errors chan error
mu sync.Mutex
port *unix.EventPort
done chan struct{} // Channel for sending a "quit message" to the reader goroutine
dirs map[string]struct{} // Explicitly watched directories
watches map[string]struct{} // Explicitly watched non-directories
done chan struct{} // Channel for sending a "quit message" to the reader goroutine
dirs map[string]Op // Explicitly watched directories
watches map[string]Op // Explicitly watched non-directories
}
// NewWatcher creates a new Watcher.
func NewWatcher() (*Watcher, error) {
return NewBufferedWatcher(0)
func newBackend(ev chan Event, errs chan error) (backend, error) {
return newBufferedBackend(0, ev, errs)
}
// NewBufferedWatcher creates a new Watcher with a buffered Watcher.Events
// channel.
//
// The main use case for this is situations with a very large number of events
// where the kernel buffer size can't be increased (e.g. due to lack of
// permissions). An unbuffered Watcher will perform better for almost all use
// cases, and whenever possible you will be better off increasing the kernel
// buffers instead of adding a large userspace buffer.
func NewBufferedWatcher(sz uint) (*Watcher, error) {
w := &Watcher{
Events: make(chan Event, sz),
Errors: make(chan error),
dirs: make(map[string]struct{}),
watches: make(map[string]struct{}),
func newBufferedBackend(sz uint, ev chan Event, errs chan error) (backend, error) {
w := &fen{
Events: ev,
Errors: errs,
dirs: make(map[string]Op),
watches: make(map[string]Op),
done: make(chan struct{}),
}
@ -171,27 +54,30 @@ func NewBufferedWatcher(sz uint) (*Watcher, error) {
// sendEvent attempts to send an event to the user, returning true if the event
// was put in the channel successfully and false if the watcher has been closed.
func (w *Watcher) sendEvent(name string, op Op) (sent bool) {
func (w *fen) sendEvent(name string, op Op) (sent bool) {
select {
case w.Events <- Event{Name: name, Op: op}:
return true
case <-w.done:
return false
case w.Events <- Event{Name: name, Op: op}:
return true
}
}
// sendError attempts to send an error to the user, returning true if the error
// was put in the channel successfully and false if the watcher has been closed.
func (w *Watcher) sendError(err error) (sent bool) {
select {
case w.Errors <- err:
func (w *fen) sendError(err error) (sent bool) {
if err == nil {
return true
}
select {
case <-w.done:
return false
case w.Errors <- err:
return true
}
}
func (w *Watcher) isClosed() bool {
func (w *fen) isClosed() bool {
select {
case <-w.done:
return true
@ -200,8 +86,7 @@ func (w *Watcher) isClosed() bool {
}
}
// Close removes all watches and closes the Events channel.
func (w *Watcher) Close() error {
func (w *fen) Close() error {
// Take the lock used by associateFile to prevent lingering events from
// being processed after the close
w.mu.Lock()
@ -213,60 +98,21 @@ func (w *Watcher) Close() error {
return w.port.Close()
}
// Add starts monitoring the path for changes.
//
// A path can only be watched once; watching it more than once is a no-op and will
// not return an error. Paths that do not yet exist on the filesystem cannot be
// watched.
//
// A watch will be automatically removed if the watched path is deleted or
// renamed. The exception is the Windows backend, which doesn't remove the
// watcher on renames.
//
// Notifications on network filesystems (NFS, SMB, FUSE, etc.) or special
// filesystems (/proc, /sys, etc.) generally don't work.
//
// Returns [ErrClosed] if [Watcher.Close] was called.
//
// See [Watcher.AddWith] for a version that allows adding options.
//
// # Watching directories
//
// All files in a directory are monitored, including new files that are created
// after the watcher is started. Subdirectories are not watched (i.e. it's
// non-recursive).
//
// # Watching files
//
// Watching individual files (rather than directories) is generally not
// recommended as many programs (especially editors) update files atomically: it
// will write to a temporary file which is then moved to to destination,
// overwriting the original (or some variant thereof). The watcher on the
// original file is now lost, as that no longer exists.
//
// The upshot of this is that a power failure or crash won't leave a
// half-written file.
//
// Watch the parent directory and use Event.Name to filter out files you're not
// interested in. There is an example of this in cmd/fsnotify/file.go.
func (w *Watcher) Add(name string) error { return w.AddWith(name) }
func (w *fen) Add(name string) error { return w.AddWith(name) }
// AddWith is like [Watcher.Add], but allows adding options. When using Add()
// the defaults described below are used.
//
// Possible options are:
//
// - [WithBufferSize] sets the buffer size for the Windows backend; no-op on
// other platforms. The default is 64K (65536 bytes).
func (w *Watcher) AddWith(name string, opts ...addOpt) error {
func (w *fen) AddWith(name string, opts ...addOpt) error {
if w.isClosed() {
return ErrClosed
}
if w.port.PathIsWatched(name) {
return nil
if debug {
fmt.Fprintf(os.Stderr, "FSNOTIFY_DEBUG: %s AddWith(%q)\n",
time.Now().Format("15:04:05.000000000"), name)
}
_ = getOptions(opts...)
with := getOptions(opts...)
if !w.xSupports(with.op) {
return fmt.Errorf("%w: %s", xErrUnsupported, with.op)
}
// Currently we resolve symlinks that were explicitly requested to be
// watched. Otherwise we would use LStat here.
@ -283,7 +129,7 @@ func (w *Watcher) AddWith(name string, opts ...addOpt) error {
}
w.mu.Lock()
w.dirs[name] = struct{}{}
w.dirs[name] = with.op
w.mu.Unlock()
return nil
}
@ -294,26 +140,22 @@ func (w *Watcher) AddWith(name string, opts ...addOpt) error {
}
w.mu.Lock()
w.watches[name] = struct{}{}
w.watches[name] = with.op
w.mu.Unlock()
return nil
}
// Remove stops monitoring the path for changes.
//
// Directories are always removed non-recursively. For example, if you added
// /tmp/dir and /tmp/dir/subdir then you will need to remove both.
//
// Removing a path that has not yet been added returns [ErrNonExistentWatch].
//
// Returns nil if [Watcher.Close] was called.
func (w *Watcher) Remove(name string) error {
func (w *fen) Remove(name string) error {
if w.isClosed() {
return nil
}
if !w.port.PathIsWatched(name) {
return fmt.Errorf("%w: %s", ErrNonExistentWatch, name)
}
if debug {
fmt.Fprintf(os.Stderr, "FSNOTIFY_DEBUG: %s Remove(%q)\n",
time.Now().Format("15:04:05.000000000"), name)
}
// The user has expressed an intent. Immediately remove this name from
// whichever watch list it might be in. If it's not in there the delete
@ -346,7 +188,7 @@ func (w *Watcher) Remove(name string) error {
}
// readEvents contains the main loop that runs in a goroutine watching for events.
func (w *Watcher) readEvents() {
func (w *fen) readEvents() {
// If this function returns, the watcher has been closed and we can close
// these channels
defer func() {
@ -382,17 +224,19 @@ func (w *Watcher) readEvents() {
continue
}
if debug {
internal.Debug(pevent.Path, pevent.Events)
}
err = w.handleEvent(&pevent)
if err != nil {
if !w.sendError(err) {
return
}
if !w.sendError(err) {
return
}
}
}
}
func (w *Watcher) handleDirectory(path string, stat os.FileInfo, follow bool, handler func(string, os.FileInfo, bool) error) error {
func (w *fen) handleDirectory(path string, stat os.FileInfo, follow bool, handler func(string, os.FileInfo, bool) error) error {
files, err := os.ReadDir(path)
if err != nil {
return err
@ -418,7 +262,7 @@ func (w *Watcher) handleDirectory(path string, stat os.FileInfo, follow bool, ha
// bitmap matches more than one event type (e.g. the file was both modified and
// had the attributes changed between when the association was created and the
// when event was returned)
func (w *Watcher) handleEvent(event *unix.PortEvent) error {
func (w *fen) handleEvent(event *unix.PortEvent) error {
var (
events = event.Events
path = event.Path
@ -510,15 +354,9 @@ func (w *Watcher) handleEvent(event *unix.PortEvent) error {
}
if events&unix.FILE_MODIFIED != 0 {
if fmode.IsDir() {
if watchedDir {
if err := w.updateDirectory(path); err != nil {
return err
}
} else {
if !w.sendEvent(path, Write) {
return nil
}
if fmode.IsDir() && watchedDir {
if err := w.updateDirectory(path); err != nil {
return err
}
} else {
if !w.sendEvent(path, Write) {
@ -543,7 +381,7 @@ func (w *Watcher) handleEvent(event *unix.PortEvent) error {
return nil
}
func (w *Watcher) updateDirectory(path string) error {
func (w *fen) updateDirectory(path string) error {
// The directory was modified, so we must find unwatched entities and watch
// them. If something was removed from the directory, nothing will happen,
// as everything else should still be watched.
@ -563,10 +401,8 @@ func (w *Watcher) updateDirectory(path string) error {
return err
}
err = w.associateFile(path, finfo, false)
if err != nil {
if !w.sendError(err) {
return nil
}
if !w.sendError(err) {
return nil
}
if !w.sendEvent(path, Create) {
return nil
@ -575,7 +411,7 @@ func (w *Watcher) updateDirectory(path string) error {
return nil
}
func (w *Watcher) associateFile(path string, stat os.FileInfo, follow bool) error {
func (w *fen) associateFile(path string, stat os.FileInfo, follow bool) error {
if w.isClosed() {
return ErrClosed
}
@ -593,34 +429,34 @@ func (w *Watcher) associateFile(path string, stat os.FileInfo, follow bool) erro
// cleared up that discrepancy. The most likely cause is that the event
// has fired but we haven't processed it yet.
err := w.port.DissociatePath(path)
if err != nil && err != unix.ENOENT {
if err != nil && !errors.Is(err, unix.ENOENT) {
return err
}
}
// FILE_NOFOLLOW means we watch symlinks themselves rather than their
// targets.
events := unix.FILE_MODIFIED | unix.FILE_ATTRIB | unix.FILE_NOFOLLOW
if follow {
// We *DO* follow symlinks for explicitly watched entries.
events = unix.FILE_MODIFIED | unix.FILE_ATTRIB
var events int
if !follow {
// Watch symlinks themselves rather than their targets unless this entry
// is explicitly watched.
events |= unix.FILE_NOFOLLOW
}
return w.port.AssociatePath(path, stat,
events,
stat.Mode())
if true { // TODO: implement withOps()
events |= unix.FILE_MODIFIED
}
if true {
events |= unix.FILE_ATTRIB
}
return w.port.AssociatePath(path, stat, events, stat.Mode())
}
func (w *Watcher) dissociateFile(path string, stat os.FileInfo, unused bool) error {
func (w *fen) dissociateFile(path string, stat os.FileInfo, unused bool) error {
if !w.port.PathIsWatched(path) {
return nil
}
return w.port.DissociatePath(path)
}
// WatchList returns all paths explicitly added with [Watcher.Add] (and are not
// yet removed).
//
// Returns nil if [Watcher.Close] was called.
func (w *Watcher) WatchList() []string {
func (w *fen) WatchList() []string {
if w.isClosed() {
return nil
}
@ -638,3 +474,11 @@ func (w *Watcher) WatchList() []string {
return entries
}
func (w *fen) xSupports(op Op) bool {
if op.Has(xUnportableOpen) || op.Has(xUnportableRead) ||
op.Has(xUnportableCloseWrite) || op.Has(xUnportableCloseRead) {
return false
}
return true
}

View file

@ -1,8 +1,4 @@
//go:build linux && !appengine
// +build linux,!appengine
// Note: the documentation on the Watcher type and methods is generated from
// mkdoc.zsh
package fsnotify
@ -10,127 +6,20 @@ import (
"errors"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"strings"
"sync"
"time"
"unsafe"
"github.com/fsnotify/fsnotify/internal"
"golang.org/x/sys/unix"
)
// Watcher watches a set of paths, delivering events on a channel.
//
// A watcher should not be copied (e.g. pass it by pointer, rather than by
// value).
//
// # Linux notes
//
// When a file is removed a Remove event won't be emitted until all file
// descriptors are closed, and deletes will always emit a Chmod. For example:
//
// fp := os.Open("file")
// os.Remove("file") // Triggers Chmod
// fp.Close() // Triggers Remove
//
// This is the event that inotify sends, so not much can be changed about this.
//
// The fs.inotify.max_user_watches sysctl variable specifies the upper limit
// for the number of watches per user, and fs.inotify.max_user_instances
// specifies the maximum number of inotify instances per user. Every Watcher you
// create is an "instance", and every path you add is a "watch".
//
// These are also exposed in /proc as /proc/sys/fs/inotify/max_user_watches and
// /proc/sys/fs/inotify/max_user_instances
//
// To increase them you can use sysctl or write the value to the /proc file:
//
// # Default values on Linux 5.18
// sysctl fs.inotify.max_user_watches=124983
// sysctl fs.inotify.max_user_instances=128
//
// To make the changes persist on reboot edit /etc/sysctl.conf or
// /usr/lib/sysctl.d/50-default.conf (details differ per Linux distro; check
// your distro's documentation):
//
// fs.inotify.max_user_watches=124983
// fs.inotify.max_user_instances=128
//
// Reaching the limit will result in a "no space left on device" or "too many open
// files" error.
//
// # kqueue notes (macOS, BSD)
//
// kqueue requires opening a file descriptor for every file that's being watched;
// so if you're watching a directory with five files then that's six file
// descriptors. You will run in to your system's "max open files" limit faster on
// these platforms.
//
// The sysctl variables kern.maxfiles and kern.maxfilesperproc can be used to
// control the maximum number of open files, as well as /etc/login.conf on BSD
// systems.
//
// # Windows notes
//
// Paths can be added as "C:\path\to\dir", but forward slashes
// ("C:/path/to/dir") will also work.
//
// When a watched directory is removed it will always send an event for the
// directory itself, but may not send events for all files in that directory.
// Sometimes it will send events for all times, sometimes it will send no
// events, and often only for some files.
//
// The default ReadDirectoryChangesW() buffer size is 64K, which is the largest
// value that is guaranteed to work with SMB filesystems. If you have many
// events in quick succession this may not be enough, and you will have to use
// [WithBufferSize] to increase the value.
type Watcher struct {
// Events sends the filesystem change events.
//
// fsnotify can send the following events; a "path" here can refer to a
// file, directory, symbolic link, or special file like a FIFO.
//
// fsnotify.Create A new path was created; this may be followed by one
// or more Write events if data also gets written to a
// file.
//
// fsnotify.Remove A path was removed.
//
// fsnotify.Rename A path was renamed. A rename is always sent with the
// old path as Event.Name, and a Create event will be
// sent with the new name. Renames are only sent for
// paths that are currently watched; e.g. moving an
// unmonitored file into a monitored directory will
// show up as just a Create. Similarly, renaming a file
// to outside a monitored directory will show up as
// only a Rename.
//
// fsnotify.Write A file or named pipe was written to. A Truncate will
// also trigger a Write. A single "write action"
// initiated by the user may show up as one or multiple
// writes, depending on when the system syncs things to
// disk. For example when compiling a large Go program
// you may get hundreds of Write events, and you may
// want to wait until you've stopped receiving them
// (see the dedup example in cmd/fsnotify).
//
// Some systems may send Write event for directories
// when the directory content changes.
//
// fsnotify.Chmod Attributes were changed. On Linux this is also sent
// when a file is removed (or more accurately, when a
// link to an inode is removed). On kqueue it's sent
// when a file is truncated. On Windows it's never
// sent.
type inotify struct {
Events chan Event
// Errors sends any errors.
//
// ErrEventOverflow is used to indicate there are too many events:
//
// - inotify: There are too many queued events (fs.inotify.max_queued_events sysctl)
// - windows: The buffer size is too small; WithBufferSize() can be used to increase it.
// - kqueue, fen: Not used.
Errors chan error
// Store fd here as os.File.Read() will no longer return on close after
@ -139,8 +28,26 @@ type Watcher struct {
inotifyFile *os.File
watches *watches
done chan struct{} // Channel for sending a "quit message" to the reader goroutine
closeMu sync.Mutex
doneMu sync.Mutex
doneResp chan struct{} // Channel to respond to Close
// Store rename cookies in an array, with the index wrapping to 0. Almost
// all of the time what we get is a MOVED_FROM to set the cookie and the
// next event inotify sends will be MOVED_TO to read it. However, this is
// not guaranteed as described in inotify(7) and we may get other events
// between the two MOVED_* events (including other MOVED_* ones).
//
// A second issue is that moving a file outside the watched directory will
// trigger a MOVED_FROM to set the cookie, but we never see the MOVED_TO to
// read and delete it. So just storing it in a map would slowly leak memory.
//
// Doing it like this gives us a simple fast LRU-cache that won't allocate.
// Ten items should be more than enough for our purpose, and a loop over
// such a short array is faster than a map access anyway (not that it hugely
// matters since we're talking about hundreds of ns at the most, but still).
cookies [10]koekje
cookieIndex uint8
cookiesMu sync.Mutex
}
type (
@ -150,9 +57,14 @@ type (
path map[string]uint32 // pathname → wd
}
watch struct {
wd uint32 // Watch descriptor (as returned by the inotify_add_watch() syscall)
flags uint32 // inotify flags of this watch (see inotify(7) for the list of valid flags)
path string // Watch path.
wd uint32 // Watch descriptor (as returned by the inotify_add_watch() syscall)
flags uint32 // inotify flags of this watch (see inotify(7) for the list of valid flags)
path string // Watch path.
recurse bool // Recursion with ./...?
}
koekje struct {
cookie uint32
path string
}
)
@ -179,23 +91,45 @@ func (w *watches) add(ww *watch) {
func (w *watches) remove(wd uint32) {
w.mu.Lock()
defer w.mu.Unlock()
delete(w.path, w.wd[wd].path)
watch := w.wd[wd] // Could have had Remove() called. See #616.
if watch == nil {
return
}
delete(w.path, watch.path)
delete(w.wd, wd)
}
func (w *watches) removePath(path string) (uint32, bool) {
func (w *watches) removePath(path string) ([]uint32, error) {
w.mu.Lock()
defer w.mu.Unlock()
path, recurse := recursivePath(path)
wd, ok := w.path[path]
if !ok {
return 0, false
return nil, fmt.Errorf("%w: %s", ErrNonExistentWatch, path)
}
watch := w.wd[wd]
if recurse && !watch.recurse {
return nil, fmt.Errorf("can't use /... with non-recursive watch %q", path)
}
delete(w.path, path)
delete(w.wd, wd)
if !watch.recurse {
return []uint32{wd}, nil
}
return wd, true
wds := make([]uint32, 0, 8)
wds = append(wds, wd)
for p, rwd := range w.path {
if filepath.HasPrefix(p, path) {
delete(w.path, p)
delete(w.wd, rwd)
wds = append(wds, rwd)
}
}
return wds, nil
}
func (w *watches) byPath(path string) *watch {
@ -236,20 +170,11 @@ func (w *watches) updatePath(path string, f func(*watch) (*watch, error)) error
return nil
}
// NewWatcher creates a new Watcher.
func NewWatcher() (*Watcher, error) {
return NewBufferedWatcher(0)
func newBackend(ev chan Event, errs chan error) (backend, error) {
return newBufferedBackend(0, ev, errs)
}
// NewBufferedWatcher creates a new Watcher with a buffered Watcher.Events
// channel.
//
// The main use case for this is situations with a very large number of events
// where the kernel buffer size can't be increased (e.g. due to lack of
// permissions). An unbuffered Watcher will perform better for almost all use
// cases, and whenever possible you will be better off increasing the kernel
// buffers instead of adding a large userspace buffer.
func NewBufferedWatcher(sz uint) (*Watcher, error) {
func newBufferedBackend(sz uint, ev chan Event, errs chan error) (backend, error) {
// Need to set nonblocking mode for SetDeadline to work, otherwise blocking
// I/O operations won't terminate on close.
fd, errno := unix.InotifyInit1(unix.IN_CLOEXEC | unix.IN_NONBLOCK)
@ -257,12 +182,12 @@ func NewBufferedWatcher(sz uint) (*Watcher, error) {
return nil, errno
}
w := &Watcher{
w := &inotify{
Events: ev,
Errors: errs,
fd: fd,
inotifyFile: os.NewFile(uintptr(fd), ""),
watches: newWatches(),
Events: make(chan Event, sz),
Errors: make(chan error),
done: make(chan struct{}),
doneResp: make(chan struct{}),
}
@ -272,26 +197,29 @@ func NewBufferedWatcher(sz uint) (*Watcher, error) {
}
// Returns true if the event was sent, or false if watcher is closed.
func (w *Watcher) sendEvent(e Event) bool {
func (w *inotify) sendEvent(e Event) bool {
select {
case w.Events <- e:
return true
case <-w.done:
return false
case w.Events <- e:
return true
}
}
// Returns true if the error was sent, or false if watcher is closed.
func (w *Watcher) sendError(err error) bool {
select {
case w.Errors <- err:
func (w *inotify) sendError(err error) bool {
if err == nil {
return true
}
select {
case <-w.done:
return false
case w.Errors <- err:
return true
}
}
func (w *Watcher) isClosed() bool {
func (w *inotify) isClosed() bool {
select {
case <-w.done:
return true
@ -300,15 +228,14 @@ func (w *Watcher) isClosed() bool {
}
}
// Close removes all watches and closes the Events channel.
func (w *Watcher) Close() error {
w.closeMu.Lock()
func (w *inotify) Close() error {
w.doneMu.Lock()
if w.isClosed() {
w.closeMu.Unlock()
w.doneMu.Unlock()
return nil
}
close(w.done)
w.closeMu.Unlock()
w.doneMu.Unlock()
// Causes any blocking reads to return with an error, provided the file
// still supports deadline operations.
@ -323,78 +250,104 @@ func (w *Watcher) Close() error {
return nil
}
// Add starts monitoring the path for changes.
//
// A path can only be watched once; watching it more than once is a no-op and will
// not return an error. Paths that do not yet exist on the filesystem cannot be
// watched.
//
// A watch will be automatically removed if the watched path is deleted or
// renamed. The exception is the Windows backend, which doesn't remove the
// watcher on renames.
//
// Notifications on network filesystems (NFS, SMB, FUSE, etc.) or special
// filesystems (/proc, /sys, etc.) generally don't work.
//
// Returns [ErrClosed] if [Watcher.Close] was called.
//
// See [Watcher.AddWith] for a version that allows adding options.
//
// # Watching directories
//
// All files in a directory are monitored, including new files that are created
// after the watcher is started. Subdirectories are not watched (i.e. it's
// non-recursive).
//
// # Watching files
//
// Watching individual files (rather than directories) is generally not
// recommended as many programs (especially editors) update files atomically: it
// will write to a temporary file which is then moved to to destination,
// overwriting the original (or some variant thereof). The watcher on the
// original file is now lost, as that no longer exists.
//
// The upshot of this is that a power failure or crash won't leave a
// half-written file.
//
// Watch the parent directory and use Event.Name to filter out files you're not
// interested in. There is an example of this in cmd/fsnotify/file.go.
func (w *Watcher) Add(name string) error { return w.AddWith(name) }
func (w *inotify) Add(name string) error { return w.AddWith(name) }
// AddWith is like [Watcher.Add], but allows adding options. When using Add()
// the defaults described below are used.
//
// Possible options are:
//
// - [WithBufferSize] sets the buffer size for the Windows backend; no-op on
// other platforms. The default is 64K (65536 bytes).
func (w *Watcher) AddWith(name string, opts ...addOpt) error {
func (w *inotify) AddWith(path string, opts ...addOpt) error {
if w.isClosed() {
return ErrClosed
}
if debug {
fmt.Fprintf(os.Stderr, "FSNOTIFY_DEBUG: %s AddWith(%q)\n",
time.Now().Format("15:04:05.000000000"), path)
}
name = filepath.Clean(name)
_ = getOptions(opts...)
with := getOptions(opts...)
if !w.xSupports(with.op) {
return fmt.Errorf("%w: %s", xErrUnsupported, with.op)
}
var flags uint32 = unix.IN_MOVED_TO | unix.IN_MOVED_FROM |
unix.IN_CREATE | unix.IN_ATTRIB | unix.IN_MODIFY |
unix.IN_MOVE_SELF | unix.IN_DELETE | unix.IN_DELETE_SELF
path, recurse := recursivePath(path)
if recurse {
return filepath.WalkDir(path, func(root string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if !d.IsDir() {
if root == path {
return fmt.Errorf("fsnotify: not a directory: %q", path)
}
return nil
}
return w.watches.updatePath(name, func(existing *watch) (*watch, error) {
// Send a Create event when adding new directory from a recursive
// watch; this is for "mkdir -p one/two/three". Usually all those
// directories will be created before we can set up watchers on the
// subdirectories, so only "one" would be sent as a Create event and
// not "one/two" and "one/two/three" (inotifywait -r has the same
// problem).
if with.sendCreate && root != path {
w.sendEvent(Event{Name: root, Op: Create})
}
return w.add(root, with, true)
})
}
return w.add(path, with, false)
}
func (w *inotify) add(path string, with withOpts, recurse bool) error {
var flags uint32
if with.noFollow {
flags |= unix.IN_DONT_FOLLOW
}
if with.op.Has(Create) {
flags |= unix.IN_CREATE
}
if with.op.Has(Write) {
flags |= unix.IN_MODIFY
}
if with.op.Has(Remove) {
flags |= unix.IN_DELETE | unix.IN_DELETE_SELF
}
if with.op.Has(Rename) {
flags |= unix.IN_MOVED_TO | unix.IN_MOVED_FROM | unix.IN_MOVE_SELF
}
if with.op.Has(Chmod) {
flags |= unix.IN_ATTRIB
}
if with.op.Has(xUnportableOpen) {
flags |= unix.IN_OPEN
}
if with.op.Has(xUnportableRead) {
flags |= unix.IN_ACCESS
}
if with.op.Has(xUnportableCloseWrite) {
flags |= unix.IN_CLOSE_WRITE
}
if with.op.Has(xUnportableCloseRead) {
flags |= unix.IN_CLOSE_NOWRITE
}
return w.register(path, flags, recurse)
}
func (w *inotify) register(path string, flags uint32, recurse bool) error {
return w.watches.updatePath(path, func(existing *watch) (*watch, error) {
if existing != nil {
flags |= existing.flags | unix.IN_MASK_ADD
}
wd, err := unix.InotifyAddWatch(w.fd, name, flags)
wd, err := unix.InotifyAddWatch(w.fd, path, flags)
if wd == -1 {
return nil, err
}
if existing == nil {
return &watch{
wd: uint32(wd),
path: name,
flags: flags,
wd: uint32(wd),
path: path,
flags: flags,
recurse: recurse,
}, nil
}
@ -404,49 +357,44 @@ func (w *Watcher) AddWith(name string, opts ...addOpt) error {
})
}
// Remove stops monitoring the path for changes.
//
// Directories are always removed non-recursively. For example, if you added
// /tmp/dir and /tmp/dir/subdir then you will need to remove both.
//
// Removing a path that has not yet been added returns [ErrNonExistentWatch].
//
// Returns nil if [Watcher.Close] was called.
func (w *Watcher) Remove(name string) error {
func (w *inotify) Remove(name string) error {
if w.isClosed() {
return nil
}
if debug {
fmt.Fprintf(os.Stderr, "FSNOTIFY_DEBUG: %s Remove(%q)\n",
time.Now().Format("15:04:05.000000000"), name)
}
return w.remove(filepath.Clean(name))
}
func (w *Watcher) remove(name string) error {
wd, ok := w.watches.removePath(name)
if !ok {
return fmt.Errorf("%w: %s", ErrNonExistentWatch, name)
func (w *inotify) remove(name string) error {
wds, err := w.watches.removePath(name)
if err != nil {
return err
}
success, errno := unix.InotifyRmWatch(w.fd, wd)
if success == -1 {
// TODO: Perhaps it's not helpful to return an error here in every case;
// The only two possible errors are:
//
// - EBADF, which happens when w.fd is not a valid file descriptor
// of any kind.
// - EINVAL, which is when fd is not an inotify descriptor or wd
// is not a valid watch descriptor. Watch descriptors are
// invalidated when they are removed explicitly or implicitly;
// explicitly by inotify_rm_watch, implicitly when the file they
// are watching is deleted.
return errno
for _, wd := range wds {
_, err := unix.InotifyRmWatch(w.fd, wd)
if err != nil {
// TODO: Perhaps it's not helpful to return an error here in every
// case; the only two possible errors are:
//
// EBADF, which happens when w.fd is not a valid file descriptor of
// any kind.
//
// EINVAL, which is when fd is not an inotify descriptor or wd is
// not a valid watch descriptor. Watch descriptors are invalidated
// when they are removed explicitly or implicitly; explicitly by
// inotify_rm_watch, implicitly when the file they are watching is
// deleted.
return err
}
}
return nil
}
// WatchList returns all paths explicitly added with [Watcher.Add] (and are not
// yet removed).
//
// Returns nil if [Watcher.Close] was called.
func (w *Watcher) WatchList() []string {
func (w *inotify) WatchList() []string {
if w.isClosed() {
return nil
}
@ -463,7 +411,7 @@ func (w *Watcher) WatchList() []string {
// readEvents reads from the inotify file descriptor, converts the
// received events into Event objects and sends them via the Events channel
func (w *Watcher) readEvents() {
func (w *inotify) readEvents() {
defer func() {
close(w.doneResp)
close(w.Errors)
@ -506,15 +454,17 @@ func (w *Watcher) readEvents() {
continue
}
var offset uint32
// We don't know how many events we just read into the buffer
// While the offset points to at least one whole event...
var offset uint32
for offset <= uint32(n-unix.SizeofInotifyEvent) {
var (
// Point "raw" to the event in the buffer
raw = (*unix.InotifyEvent)(unsafe.Pointer(&buf[offset]))
mask = uint32(raw.Mask)
nameLen = uint32(raw.Len)
// Move to the next event in the buffer
next = func() { offset += unix.SizeofInotifyEvent + nameLen }
)
if mask&unix.IN_Q_OVERFLOW != 0 {
@ -523,21 +473,53 @@ func (w *Watcher) readEvents() {
}
}
// If the event happened to the watched directory or the watched file, the kernel
// doesn't append the filename to the event, but we would like to always fill the
// the "Name" field with a valid filename. We retrieve the path of the watch from
// the "paths" map.
/// If the event happened to the watched directory or the watched
/// file, the kernel doesn't append the filename to the event, but
/// we would like to always fill the the "Name" field with a valid
/// filename. We retrieve the path of the watch from the "paths"
/// map.
watch := w.watches.byWd(uint32(raw.Wd))
/// Can be nil if Remove() was called in another goroutine for this
/// path inbetween reading the events from the kernel and reading
/// the internal state. Not much we can do about it, so just skip.
/// See #616.
if watch == nil {
next()
continue
}
name := watch.path
if nameLen > 0 {
/// Point "bytes" at the first byte of the filename
bytes := (*[unix.PathMax]byte)(unsafe.Pointer(&buf[offset+unix.SizeofInotifyEvent]))[:nameLen:nameLen]
/// The filename is padded with NULL bytes. TrimRight() gets rid of those.
name += "/" + strings.TrimRight(string(bytes[0:nameLen]), "\000")
}
if debug {
internal.Debug(name, raw.Mask, raw.Cookie)
}
if mask&unix.IN_IGNORED != 0 { //&& event.Op != 0
next()
continue
}
// inotify will automatically remove the watch on deletes; just need
// to clean our state here.
if watch != nil && mask&unix.IN_DELETE_SELF == unix.IN_DELETE_SELF {
if mask&unix.IN_DELETE_SELF == unix.IN_DELETE_SELF {
w.watches.remove(watch.wd)
}
// We can't really update the state when a watched path is moved;
// only IN_MOVE_SELF is sent and not IN_MOVED_{FROM,TO}. So remove
// the watch.
if watch != nil && mask&unix.IN_MOVE_SELF == unix.IN_MOVE_SELF {
if mask&unix.IN_MOVE_SELF == unix.IN_MOVE_SELF {
if watch.recurse {
next() // Do nothing
continue
}
err := w.remove(watch.path)
if err != nil && !errors.Is(err, ErrNonExistentWatch) {
if !w.sendError(err) {
@ -546,34 +528,69 @@ func (w *Watcher) readEvents() {
}
}
var name string
if watch != nil {
name = watch.path
}
if nameLen > 0 {
// Point "bytes" at the first byte of the filename
bytes := (*[unix.PathMax]byte)(unsafe.Pointer(&buf[offset+unix.SizeofInotifyEvent]))[:nameLen:nameLen]
// The filename is padded with NULL bytes. TrimRight() gets rid of those.
name += "/" + strings.TrimRight(string(bytes[0:nameLen]), "\000")
}
event := w.newEvent(name, mask)
// Send the events that are not ignored on the events channel
if mask&unix.IN_IGNORED == 0 {
if !w.sendEvent(event) {
return
/// Skip if we're watching both this path and the parent; the parent
/// will already send a delete so no need to do it twice.
if mask&unix.IN_DELETE_SELF != 0 {
if _, ok := w.watches.path[filepath.Dir(watch.path)]; ok {
next()
continue
}
}
// Move to the next event in the buffer
offset += unix.SizeofInotifyEvent + nameLen
ev := w.newEvent(name, mask, raw.Cookie)
// Need to update watch path for recurse.
if watch.recurse {
isDir := mask&unix.IN_ISDIR == unix.IN_ISDIR
/// New directory created: set up watch on it.
if isDir && ev.Has(Create) {
err := w.register(ev.Name, watch.flags, true)
if !w.sendError(err) {
return
}
// This was a directory rename, so we need to update all
// the children.
//
// TODO: this is of course pretty slow; we should use a
// better data structure for storing all of this, e.g. store
// children in the watch. I have some code for this in my
// kqueue refactor we can use in the future. For now I'm
// okay with this as it's not publicly available.
// Correctness first, performance second.
if ev.renamedFrom != "" {
w.watches.mu.Lock()
for k, ww := range w.watches.wd {
if k == watch.wd || ww.path == ev.Name {
continue
}
if strings.HasPrefix(ww.path, ev.renamedFrom) {
ww.path = strings.Replace(ww.path, ev.renamedFrom, ev.Name, 1)
w.watches.wd[k] = ww
}
}
w.watches.mu.Unlock()
}
}
}
/// Send the events that are not ignored on the events channel
if !w.sendEvent(ev) {
return
}
next()
}
}
}
// newEvent returns an platform-independent Event based on an inotify mask.
func (w *Watcher) newEvent(name string, mask uint32) Event {
func (w *inotify) isRecursive(path string) bool {
ww := w.watches.byPath(path)
if ww == nil { // path could be a file, so also check the Dir.
ww = w.watches.byPath(filepath.Dir(path))
}
return ww != nil && ww.recurse
}
func (w *inotify) newEvent(name string, mask, cookie uint32) Event {
e := Event{Name: name}
if mask&unix.IN_CREATE == unix.IN_CREATE || mask&unix.IN_MOVED_TO == unix.IN_MOVED_TO {
e.Op |= Create
@ -584,11 +601,58 @@ func (w *Watcher) newEvent(name string, mask uint32) Event {
if mask&unix.IN_MODIFY == unix.IN_MODIFY {
e.Op |= Write
}
if mask&unix.IN_OPEN == unix.IN_OPEN {
e.Op |= xUnportableOpen
}
if mask&unix.IN_ACCESS == unix.IN_ACCESS {
e.Op |= xUnportableRead
}
if mask&unix.IN_CLOSE_WRITE == unix.IN_CLOSE_WRITE {
e.Op |= xUnportableCloseWrite
}
if mask&unix.IN_CLOSE_NOWRITE == unix.IN_CLOSE_NOWRITE {
e.Op |= xUnportableCloseRead
}
if mask&unix.IN_MOVE_SELF == unix.IN_MOVE_SELF || mask&unix.IN_MOVED_FROM == unix.IN_MOVED_FROM {
e.Op |= Rename
}
if mask&unix.IN_ATTRIB == unix.IN_ATTRIB {
e.Op |= Chmod
}
if cookie != 0 {
if mask&unix.IN_MOVED_FROM == unix.IN_MOVED_FROM {
w.cookiesMu.Lock()
w.cookies[w.cookieIndex] = koekje{cookie: cookie, path: e.Name}
w.cookieIndex++
if w.cookieIndex > 9 {
w.cookieIndex = 0
}
w.cookiesMu.Unlock()
} else if mask&unix.IN_MOVED_TO == unix.IN_MOVED_TO {
w.cookiesMu.Lock()
var prev string
for _, c := range w.cookies {
if c.cookie == cookie {
prev = c.path
break
}
}
w.cookiesMu.Unlock()
e.renamedFrom = prev
}
}
return e
}
func (w *inotify) xSupports(op Op) bool {
return true // Supports everything.
}
func (w *inotify) state() {
w.watches.mu.Lock()
defer w.watches.mu.Unlock()
for wd, ww := range w.watches.wd {
fmt.Fprintf(os.Stderr, "%4d: recurse=%t %q\n", wd, ww.recurse, ww.path)
}
}

File diff suppressed because it is too large Load diff

View file

@ -1,205 +1,23 @@
//go:build appengine || (!darwin && !dragonfly && !freebsd && !openbsd && !linux && !netbsd && !solaris && !windows)
// +build appengine !darwin,!dragonfly,!freebsd,!openbsd,!linux,!netbsd,!solaris,!windows
// Note: the documentation on the Watcher type and methods is generated from
// mkdoc.zsh
package fsnotify
import "errors"
// Watcher watches a set of paths, delivering events on a channel.
//
// A watcher should not be copied (e.g. pass it by pointer, rather than by
// value).
//
// # Linux notes
//
// When a file is removed a Remove event won't be emitted until all file
// descriptors are closed, and deletes will always emit a Chmod. For example:
//
// fp := os.Open("file")
// os.Remove("file") // Triggers Chmod
// fp.Close() // Triggers Remove
//
// This is the event that inotify sends, so not much can be changed about this.
//
// The fs.inotify.max_user_watches sysctl variable specifies the upper limit
// for the number of watches per user, and fs.inotify.max_user_instances
// specifies the maximum number of inotify instances per user. Every Watcher you
// create is an "instance", and every path you add is a "watch".
//
// These are also exposed in /proc as /proc/sys/fs/inotify/max_user_watches and
// /proc/sys/fs/inotify/max_user_instances
//
// To increase them you can use sysctl or write the value to the /proc file:
//
// # Default values on Linux 5.18
// sysctl fs.inotify.max_user_watches=124983
// sysctl fs.inotify.max_user_instances=128
//
// To make the changes persist on reboot edit /etc/sysctl.conf or
// /usr/lib/sysctl.d/50-default.conf (details differ per Linux distro; check
// your distro's documentation):
//
// fs.inotify.max_user_watches=124983
// fs.inotify.max_user_instances=128
//
// Reaching the limit will result in a "no space left on device" or "too many open
// files" error.
//
// # kqueue notes (macOS, BSD)
//
// kqueue requires opening a file descriptor for every file that's being watched;
// so if you're watching a directory with five files then that's six file
// descriptors. You will run in to your system's "max open files" limit faster on
// these platforms.
//
// The sysctl variables kern.maxfiles and kern.maxfilesperproc can be used to
// control the maximum number of open files, as well as /etc/login.conf on BSD
// systems.
//
// # Windows notes
//
// Paths can be added as "C:\path\to\dir", but forward slashes
// ("C:/path/to/dir") will also work.
//
// When a watched directory is removed it will always send an event for the
// directory itself, but may not send events for all files in that directory.
// Sometimes it will send events for all times, sometimes it will send no
// events, and often only for some files.
//
// The default ReadDirectoryChangesW() buffer size is 64K, which is the largest
// value that is guaranteed to work with SMB filesystems. If you have many
// events in quick succession this may not be enough, and you will have to use
// [WithBufferSize] to increase the value.
type Watcher struct {
// Events sends the filesystem change events.
//
// fsnotify can send the following events; a "path" here can refer to a
// file, directory, symbolic link, or special file like a FIFO.
//
// fsnotify.Create A new path was created; this may be followed by one
// or more Write events if data also gets written to a
// file.
//
// fsnotify.Remove A path was removed.
//
// fsnotify.Rename A path was renamed. A rename is always sent with the
// old path as Event.Name, and a Create event will be
// sent with the new name. Renames are only sent for
// paths that are currently watched; e.g. moving an
// unmonitored file into a monitored directory will
// show up as just a Create. Similarly, renaming a file
// to outside a monitored directory will show up as
// only a Rename.
//
// fsnotify.Write A file or named pipe was written to. A Truncate will
// also trigger a Write. A single "write action"
// initiated by the user may show up as one or multiple
// writes, depending on when the system syncs things to
// disk. For example when compiling a large Go program
// you may get hundreds of Write events, and you may
// want to wait until you've stopped receiving them
// (see the dedup example in cmd/fsnotify).
//
// Some systems may send Write event for directories
// when the directory content changes.
//
// fsnotify.Chmod Attributes were changed. On Linux this is also sent
// when a file is removed (or more accurately, when a
// link to an inode is removed). On kqueue it's sent
// when a file is truncated. On Windows it's never
// sent.
type other struct {
Events chan Event
// Errors sends any errors.
//
// ErrEventOverflow is used to indicate there are too many events:
//
// - inotify: There are too many queued events (fs.inotify.max_queued_events sysctl)
// - windows: The buffer size is too small; WithBufferSize() can be used to increase it.
// - kqueue, fen: Not used.
Errors chan error
}
// NewWatcher creates a new Watcher.
func NewWatcher() (*Watcher, error) {
func newBackend(ev chan Event, errs chan error) (backend, error) {
return nil, errors.New("fsnotify not supported on the current platform")
}
// NewBufferedWatcher creates a new Watcher with a buffered Watcher.Events
// channel.
//
// The main use case for this is situations with a very large number of events
// where the kernel buffer size can't be increased (e.g. due to lack of
// permissions). An unbuffered Watcher will perform better for almost all use
// cases, and whenever possible you will be better off increasing the kernel
// buffers instead of adding a large userspace buffer.
func NewBufferedWatcher(sz uint) (*Watcher, error) { return NewWatcher() }
// Close removes all watches and closes the Events channel.
func (w *Watcher) Close() error { return nil }
// WatchList returns all paths explicitly added with [Watcher.Add] (and are not
// yet removed).
//
// Returns nil if [Watcher.Close] was called.
func (w *Watcher) WatchList() []string { return nil }
// Add starts monitoring the path for changes.
//
// A path can only be watched once; watching it more than once is a no-op and will
// not return an error. Paths that do not yet exist on the filesystem cannot be
// watched.
//
// A watch will be automatically removed if the watched path is deleted or
// renamed. The exception is the Windows backend, which doesn't remove the
// watcher on renames.
//
// Notifications on network filesystems (NFS, SMB, FUSE, etc.) or special
// filesystems (/proc, /sys, etc.) generally don't work.
//
// Returns [ErrClosed] if [Watcher.Close] was called.
//
// See [Watcher.AddWith] for a version that allows adding options.
//
// # Watching directories
//
// All files in a directory are monitored, including new files that are created
// after the watcher is started. Subdirectories are not watched (i.e. it's
// non-recursive).
//
// # Watching files
//
// Watching individual files (rather than directories) is generally not
// recommended as many programs (especially editors) update files atomically: it
// will write to a temporary file which is then moved to to destination,
// overwriting the original (or some variant thereof). The watcher on the
// original file is now lost, as that no longer exists.
//
// The upshot of this is that a power failure or crash won't leave a
// half-written file.
//
// Watch the parent directory and use Event.Name to filter out files you're not
// interested in. There is an example of this in cmd/fsnotify/file.go.
func (w *Watcher) Add(name string) error { return nil }
// AddWith is like [Watcher.Add], but allows adding options. When using Add()
// the defaults described below are used.
//
// Possible options are:
//
// - [WithBufferSize] sets the buffer size for the Windows backend; no-op on
// other platforms. The default is 64K (65536 bytes).
func (w *Watcher) AddWith(name string, opts ...addOpt) error { return nil }
// Remove stops monitoring the path for changes.
//
// Directories are always removed non-recursively. For example, if you added
// /tmp/dir and /tmp/dir/subdir then you will need to remove both.
//
// Removing a path that has not yet been added returns [ErrNonExistentWatch].
//
// Returns nil if [Watcher.Close] was called.
func (w *Watcher) Remove(name string) error { return nil }
func newBufferedBackend(sz uint, ev chan Event, errs chan error) (backend, error) {
return newBackend(ev, errs)
}
func (w *other) Close() error { return nil }
func (w *other) WatchList() []string { return nil }
func (w *other) Add(name string) error { return nil }
func (w *other) AddWith(name string, opts ...addOpt) error { return nil }
func (w *other) Remove(name string) error { return nil }
func (w *other) xSupports(op Op) bool { return false }

View file

@ -1,12 +1,8 @@
//go:build windows
// +build windows
// Windows backend based on ReadDirectoryChangesW()
//
// https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-readdirectorychangesw
//
// Note: the documentation on the Watcher type and methods is generated from
// mkdoc.zsh
package fsnotify
@ -19,123 +15,15 @@ import (
"runtime"
"strings"
"sync"
"time"
"unsafe"
"github.com/fsnotify/fsnotify/internal"
"golang.org/x/sys/windows"
)
// Watcher watches a set of paths, delivering events on a channel.
//
// A watcher should not be copied (e.g. pass it by pointer, rather than by
// value).
//
// # Linux notes
//
// When a file is removed a Remove event won't be emitted until all file
// descriptors are closed, and deletes will always emit a Chmod. For example:
//
// fp := os.Open("file")
// os.Remove("file") // Triggers Chmod
// fp.Close() // Triggers Remove
//
// This is the event that inotify sends, so not much can be changed about this.
//
// The fs.inotify.max_user_watches sysctl variable specifies the upper limit
// for the number of watches per user, and fs.inotify.max_user_instances
// specifies the maximum number of inotify instances per user. Every Watcher you
// create is an "instance", and every path you add is a "watch".
//
// These are also exposed in /proc as /proc/sys/fs/inotify/max_user_watches and
// /proc/sys/fs/inotify/max_user_instances
//
// To increase them you can use sysctl or write the value to the /proc file:
//
// # Default values on Linux 5.18
// sysctl fs.inotify.max_user_watches=124983
// sysctl fs.inotify.max_user_instances=128
//
// To make the changes persist on reboot edit /etc/sysctl.conf or
// /usr/lib/sysctl.d/50-default.conf (details differ per Linux distro; check
// your distro's documentation):
//
// fs.inotify.max_user_watches=124983
// fs.inotify.max_user_instances=128
//
// Reaching the limit will result in a "no space left on device" or "too many open
// files" error.
//
// # kqueue notes (macOS, BSD)
//
// kqueue requires opening a file descriptor for every file that's being watched;
// so if you're watching a directory with five files then that's six file
// descriptors. You will run in to your system's "max open files" limit faster on
// these platforms.
//
// The sysctl variables kern.maxfiles and kern.maxfilesperproc can be used to
// control the maximum number of open files, as well as /etc/login.conf on BSD
// systems.
//
// # Windows notes
//
// Paths can be added as "C:\path\to\dir", but forward slashes
// ("C:/path/to/dir") will also work.
//
// When a watched directory is removed it will always send an event for the
// directory itself, but may not send events for all files in that directory.
// Sometimes it will send events for all times, sometimes it will send no
// events, and often only for some files.
//
// The default ReadDirectoryChangesW() buffer size is 64K, which is the largest
// value that is guaranteed to work with SMB filesystems. If you have many
// events in quick succession this may not be enough, and you will have to use
// [WithBufferSize] to increase the value.
type Watcher struct {
// Events sends the filesystem change events.
//
// fsnotify can send the following events; a "path" here can refer to a
// file, directory, symbolic link, or special file like a FIFO.
//
// fsnotify.Create A new path was created; this may be followed by one
// or more Write events if data also gets written to a
// file.
//
// fsnotify.Remove A path was removed.
//
// fsnotify.Rename A path was renamed. A rename is always sent with the
// old path as Event.Name, and a Create event will be
// sent with the new name. Renames are only sent for
// paths that are currently watched; e.g. moving an
// unmonitored file into a monitored directory will
// show up as just a Create. Similarly, renaming a file
// to outside a monitored directory will show up as
// only a Rename.
//
// fsnotify.Write A file or named pipe was written to. A Truncate will
// also trigger a Write. A single "write action"
// initiated by the user may show up as one or multiple
// writes, depending on when the system syncs things to
// disk. For example when compiling a large Go program
// you may get hundreds of Write events, and you may
// want to wait until you've stopped receiving them
// (see the dedup example in cmd/fsnotify).
//
// Some systems may send Write event for directories
// when the directory content changes.
//
// fsnotify.Chmod Attributes were changed. On Linux this is also sent
// when a file is removed (or more accurately, when a
// link to an inode is removed). On kqueue it's sent
// when a file is truncated. On Windows it's never
// sent.
type readDirChangesW struct {
Events chan Event
// Errors sends any errors.
//
// ErrEventOverflow is used to indicate there are too many events:
//
// - inotify: There are too many queued events (fs.inotify.max_queued_events sysctl)
// - windows: The buffer size is too small; WithBufferSize() can be used to increase it.
// - kqueue, fen: Not used.
Errors chan error
port windows.Handle // Handle to completion port
@ -147,48 +35,40 @@ type Watcher struct {
closed bool // Set to true when Close() is first called
}
// NewWatcher creates a new Watcher.
func NewWatcher() (*Watcher, error) {
return NewBufferedWatcher(50)
func newBackend(ev chan Event, errs chan error) (backend, error) {
return newBufferedBackend(50, ev, errs)
}
// NewBufferedWatcher creates a new Watcher with a buffered Watcher.Events
// channel.
//
// The main use case for this is situations with a very large number of events
// where the kernel buffer size can't be increased (e.g. due to lack of
// permissions). An unbuffered Watcher will perform better for almost all use
// cases, and whenever possible you will be better off increasing the kernel
// buffers instead of adding a large userspace buffer.
func NewBufferedWatcher(sz uint) (*Watcher, error) {
func newBufferedBackend(sz uint, ev chan Event, errs chan error) (backend, error) {
port, err := windows.CreateIoCompletionPort(windows.InvalidHandle, 0, 0, 0)
if err != nil {
return nil, os.NewSyscallError("CreateIoCompletionPort", err)
}
w := &Watcher{
w := &readDirChangesW{
Events: ev,
Errors: errs,
port: port,
watches: make(watchMap),
input: make(chan *input, 1),
Events: make(chan Event, sz),
Errors: make(chan error),
quit: make(chan chan<- error, 1),
}
go w.readEvents()
return w, nil
}
func (w *Watcher) isClosed() bool {
func (w *readDirChangesW) isClosed() bool {
w.mu.Lock()
defer w.mu.Unlock()
return w.closed
}
func (w *Watcher) sendEvent(name string, mask uint64) bool {
func (w *readDirChangesW) sendEvent(name, renamedFrom string, mask uint64) bool {
if mask == 0 {
return false
}
event := w.newEvent(name, uint32(mask))
event.renamedFrom = renamedFrom
select {
case ch := <-w.quit:
w.quit <- ch
@ -198,17 +78,19 @@ func (w *Watcher) sendEvent(name string, mask uint64) bool {
}
// Returns true if the error was sent, or false if watcher is closed.
func (w *Watcher) sendError(err error) bool {
func (w *readDirChangesW) sendError(err error) bool {
if err == nil {
return true
}
select {
case w.Errors <- err:
return true
case <-w.quit:
return false
}
return false
}
// Close removes all watches and closes the Events channel.
func (w *Watcher) Close() error {
func (w *readDirChangesW) Close() error {
if w.isClosed() {
return nil
}
@ -226,57 +108,21 @@ func (w *Watcher) Close() error {
return <-ch
}
// Add starts monitoring the path for changes.
//
// A path can only be watched once; watching it more than once is a no-op and will
// not return an error. Paths that do not yet exist on the filesystem cannot be
// watched.
//
// A watch will be automatically removed if the watched path is deleted or
// renamed. The exception is the Windows backend, which doesn't remove the
// watcher on renames.
//
// Notifications on network filesystems (NFS, SMB, FUSE, etc.) or special
// filesystems (/proc, /sys, etc.) generally don't work.
//
// Returns [ErrClosed] if [Watcher.Close] was called.
//
// See [Watcher.AddWith] for a version that allows adding options.
//
// # Watching directories
//
// All files in a directory are monitored, including new files that are created
// after the watcher is started. Subdirectories are not watched (i.e. it's
// non-recursive).
//
// # Watching files
//
// Watching individual files (rather than directories) is generally not
// recommended as many programs (especially editors) update files atomically: it
// will write to a temporary file which is then moved to to destination,
// overwriting the original (or some variant thereof). The watcher on the
// original file is now lost, as that no longer exists.
//
// The upshot of this is that a power failure or crash won't leave a
// half-written file.
//
// Watch the parent directory and use Event.Name to filter out files you're not
// interested in. There is an example of this in cmd/fsnotify/file.go.
func (w *Watcher) Add(name string) error { return w.AddWith(name) }
func (w *readDirChangesW) Add(name string) error { return w.AddWith(name) }
// AddWith is like [Watcher.Add], but allows adding options. When using Add()
// the defaults described below are used.
//
// Possible options are:
//
// - [WithBufferSize] sets the buffer size for the Windows backend; no-op on
// other platforms. The default is 64K (65536 bytes).
func (w *Watcher) AddWith(name string, opts ...addOpt) error {
func (w *readDirChangesW) AddWith(name string, opts ...addOpt) error {
if w.isClosed() {
return ErrClosed
}
if debug {
fmt.Fprintf(os.Stderr, "FSNOTIFY_DEBUG: %s AddWith(%q)\n",
time.Now().Format("15:04:05.000000000"), filepath.ToSlash(name))
}
with := getOptions(opts...)
if !w.xSupports(with.op) {
return fmt.Errorf("%w: %s", xErrUnsupported, with.op)
}
if with.bufsize < 4096 {
return fmt.Errorf("fsnotify.WithBufferSize: buffer size cannot be smaller than 4096 bytes")
}
@ -295,18 +141,14 @@ func (w *Watcher) AddWith(name string, opts ...addOpt) error {
return <-in.reply
}
// Remove stops monitoring the path for changes.
//
// Directories are always removed non-recursively. For example, if you added
// /tmp/dir and /tmp/dir/subdir then you will need to remove both.
//
// Removing a path that has not yet been added returns [ErrNonExistentWatch].
//
// Returns nil if [Watcher.Close] was called.
func (w *Watcher) Remove(name string) error {
func (w *readDirChangesW) Remove(name string) error {
if w.isClosed() {
return nil
}
if debug {
fmt.Fprintf(os.Stderr, "FSNOTIFY_DEBUG: %s Remove(%q)\n",
time.Now().Format("15:04:05.000000000"), filepath.ToSlash(name))
}
in := &input{
op: opRemoveWatch,
@ -320,11 +162,7 @@ func (w *Watcher) Remove(name string) error {
return <-in.reply
}
// WatchList returns all paths explicitly added with [Watcher.Add] (and are not
// yet removed).
//
// Returns nil if [Watcher.Close] was called.
func (w *Watcher) WatchList() []string {
func (w *readDirChangesW) WatchList() []string {
if w.isClosed() {
return nil
}
@ -335,7 +173,13 @@ func (w *Watcher) WatchList() []string {
entries := make([]string, 0, len(w.watches))
for _, entry := range w.watches {
for _, watchEntry := range entry {
entries = append(entries, watchEntry.path)
for name := range watchEntry.names {
entries = append(entries, filepath.Join(watchEntry.path, name))
}
// the directory itself is being watched
if watchEntry.mask != 0 {
entries = append(entries, watchEntry.path)
}
}
}
@ -361,7 +205,7 @@ const (
sysFSIGNORED = 0x8000
)
func (w *Watcher) newEvent(name string, mask uint32) Event {
func (w *readDirChangesW) newEvent(name string, mask uint32) Event {
e := Event{Name: name}
if mask&sysFSCREATE == sysFSCREATE || mask&sysFSMOVEDTO == sysFSMOVEDTO {
e.Op |= Create
@ -417,7 +261,7 @@ type (
watchMap map[uint32]indexMap
)
func (w *Watcher) wakeupReader() error {
func (w *readDirChangesW) wakeupReader() error {
err := windows.PostQueuedCompletionStatus(w.port, 0, 0, nil)
if err != nil {
return os.NewSyscallError("PostQueuedCompletionStatus", err)
@ -425,7 +269,7 @@ func (w *Watcher) wakeupReader() error {
return nil
}
func (w *Watcher) getDir(pathname string) (dir string, err error) {
func (w *readDirChangesW) getDir(pathname string) (dir string, err error) {
attr, err := windows.GetFileAttributes(windows.StringToUTF16Ptr(pathname))
if err != nil {
return "", os.NewSyscallError("GetFileAttributes", err)
@ -439,7 +283,7 @@ func (w *Watcher) getDir(pathname string) (dir string, err error) {
return
}
func (w *Watcher) getIno(path string) (ino *inode, err error) {
func (w *readDirChangesW) getIno(path string) (ino *inode, err error) {
h, err := windows.CreateFile(windows.StringToUTF16Ptr(path),
windows.FILE_LIST_DIRECTORY,
windows.FILE_SHARE_READ|windows.FILE_SHARE_WRITE|windows.FILE_SHARE_DELETE,
@ -482,9 +326,8 @@ func (m watchMap) set(ino *inode, watch *watch) {
}
// Must run within the I/O thread.
func (w *Watcher) addWatch(pathname string, flags uint64, bufsize int) error {
//pathname, recurse := recursivePath(pathname)
recurse := false
func (w *readDirChangesW) addWatch(pathname string, flags uint64, bufsize int) error {
pathname, recurse := recursivePath(pathname)
dir, err := w.getDir(pathname)
if err != nil {
@ -538,7 +381,7 @@ func (w *Watcher) addWatch(pathname string, flags uint64, bufsize int) error {
}
// Must run within the I/O thread.
func (w *Watcher) remWatch(pathname string) error {
func (w *readDirChangesW) remWatch(pathname string) error {
pathname, recurse := recursivePath(pathname)
dir, err := w.getDir(pathname)
@ -566,11 +409,11 @@ func (w *Watcher) remWatch(pathname string) error {
return fmt.Errorf("%w: %s", ErrNonExistentWatch, pathname)
}
if pathname == dir {
w.sendEvent(watch.path, watch.mask&sysFSIGNORED)
w.sendEvent(watch.path, "", watch.mask&sysFSIGNORED)
watch.mask = 0
} else {
name := filepath.Base(pathname)
w.sendEvent(filepath.Join(watch.path, name), watch.names[name]&sysFSIGNORED)
w.sendEvent(filepath.Join(watch.path, name), "", watch.names[name]&sysFSIGNORED)
delete(watch.names, name)
}
@ -578,23 +421,23 @@ func (w *Watcher) remWatch(pathname string) error {
}
// Must run within the I/O thread.
func (w *Watcher) deleteWatch(watch *watch) {
func (w *readDirChangesW) deleteWatch(watch *watch) {
for name, mask := range watch.names {
if mask&provisional == 0 {
w.sendEvent(filepath.Join(watch.path, name), mask&sysFSIGNORED)
w.sendEvent(filepath.Join(watch.path, name), "", mask&sysFSIGNORED)
}
delete(watch.names, name)
}
if watch.mask != 0 {
if watch.mask&provisional == 0 {
w.sendEvent(watch.path, watch.mask&sysFSIGNORED)
w.sendEvent(watch.path, "", watch.mask&sysFSIGNORED)
}
watch.mask = 0
}
}
// Must run within the I/O thread.
func (w *Watcher) startRead(watch *watch) error {
func (w *readDirChangesW) startRead(watch *watch) error {
err := windows.CancelIo(watch.ino.handle)
if err != nil {
w.sendError(os.NewSyscallError("CancelIo", err))
@ -624,7 +467,7 @@ func (w *Watcher) startRead(watch *watch) error {
err := os.NewSyscallError("ReadDirectoryChanges", rdErr)
if rdErr == windows.ERROR_ACCESS_DENIED && watch.mask&provisional == 0 {
// Watched directory was probably removed
w.sendEvent(watch.path, watch.mask&sysFSDELETESELF)
w.sendEvent(watch.path, "", watch.mask&sysFSDELETESELF)
err = nil
}
w.deleteWatch(watch)
@ -637,7 +480,7 @@ func (w *Watcher) startRead(watch *watch) error {
// readEvents reads from the I/O completion port, converts the
// received events into Event objects and sends them via the Events channel.
// Entry point to the I/O thread.
func (w *Watcher) readEvents() {
func (w *readDirChangesW) readEvents() {
var (
n uint32
key uintptr
@ -700,7 +543,7 @@ func (w *Watcher) readEvents() {
}
case windows.ERROR_ACCESS_DENIED:
// Watched directory was probably removed
w.sendEvent(watch.path, watch.mask&sysFSDELETESELF)
w.sendEvent(watch.path, "", watch.mask&sysFSDELETESELF)
w.deleteWatch(watch)
w.startRead(watch)
continue
@ -733,6 +576,10 @@ func (w *Watcher) readEvents() {
name := windows.UTF16ToString(buf)
fullname := filepath.Join(watch.path, name)
if debug {
internal.Debug(fullname, raw.Action)
}
var mask uint64
switch raw.Action {
case windows.FILE_ACTION_REMOVED:
@ -761,21 +608,22 @@ func (w *Watcher) readEvents() {
}
}
sendNameEvent := func() {
w.sendEvent(fullname, watch.names[name]&mask)
}
if raw.Action != windows.FILE_ACTION_RENAMED_NEW_NAME {
sendNameEvent()
w.sendEvent(fullname, "", watch.names[name]&mask)
}
if raw.Action == windows.FILE_ACTION_REMOVED {
w.sendEvent(fullname, watch.names[name]&sysFSIGNORED)
w.sendEvent(fullname, "", watch.names[name]&sysFSIGNORED)
delete(watch.names, name)
}
w.sendEvent(fullname, watch.mask&w.toFSnotifyFlags(raw.Action))
if watch.rename != "" && raw.Action == windows.FILE_ACTION_RENAMED_NEW_NAME {
w.sendEvent(fullname, filepath.Join(watch.path, watch.rename), watch.mask&w.toFSnotifyFlags(raw.Action))
} else {
w.sendEvent(fullname, "", watch.mask&w.toFSnotifyFlags(raw.Action))
}
if raw.Action == windows.FILE_ACTION_RENAMED_NEW_NAME {
fullname = filepath.Join(watch.path, watch.rename)
sendNameEvent()
w.sendEvent(filepath.Join(watch.path, watch.rename), "", watch.names[name]&mask)
}
// Move to the next event in the buffer
@ -787,8 +635,7 @@ func (w *Watcher) readEvents() {
// Error!
if offset >= n {
//lint:ignore ST1005 Windows should be capitalized
w.sendError(errors.New(
"Windows system assumed buffer larger than it is, events have likely been missed"))
w.sendError(errors.New("Windows system assumed buffer larger than it is, events have likely been missed"))
break
}
}
@ -799,7 +646,7 @@ func (w *Watcher) readEvents() {
}
}
func (w *Watcher) toWindowsFlags(mask uint64) uint32 {
func (w *readDirChangesW) toWindowsFlags(mask uint64) uint32 {
var m uint32
if mask&sysFSMODIFY != 0 {
m |= windows.FILE_NOTIFY_CHANGE_LAST_WRITE
@ -810,7 +657,7 @@ func (w *Watcher) toWindowsFlags(mask uint64) uint32 {
return m
}
func (w *Watcher) toFSnotifyFlags(action uint32) uint64 {
func (w *readDirChangesW) toFSnotifyFlags(action uint32) uint64 {
switch action {
case windows.FILE_ACTION_ADDED:
return sysFSCREATE
@ -825,3 +672,11 @@ func (w *Watcher) toFSnotifyFlags(action uint32) uint64 {
}
return 0
}
func (w *readDirChangesW) xSupports(op Op) bool {
if op.Has(xUnportableOpen) || op.Has(xUnportableRead) ||
op.Has(xUnportableCloseWrite) || op.Has(xUnportableCloseRead) {
return false
}
return true
}

View file

@ -3,19 +3,146 @@
//
// Currently supported systems:
//
// Linux 2.6.32+ via inotify
// BSD, macOS via kqueue
// Windows via ReadDirectoryChangesW
// illumos via FEN
// - Linux via inotify
// - BSD, macOS via kqueue
// - Windows via ReadDirectoryChangesW
// - illumos via FEN
//
// # FSNOTIFY_DEBUG
//
// Set the FSNOTIFY_DEBUG environment variable to "1" to print debug messages to
// stderr. This can be useful to track down some problems, especially in cases
// where fsnotify is used as an indirect dependency.
//
// Every event will be printed as soon as there's something useful to print,
// with as little processing from fsnotify.
//
// Example output:
//
// FSNOTIFY_DEBUG: 11:34:23.633087586 256:IN_CREATE → "/tmp/file-1"
// FSNOTIFY_DEBUG: 11:34:23.633202319 4:IN_ATTRIB → "/tmp/file-1"
// FSNOTIFY_DEBUG: 11:34:28.989728764 512:IN_DELETE → "/tmp/file-1"
package fsnotify
import (
"errors"
"fmt"
"os"
"path/filepath"
"strings"
)
// Watcher watches a set of paths, delivering events on a channel.
//
// A watcher should not be copied (e.g. pass it by pointer, rather than by
// value).
//
// # Linux notes
//
// When a file is removed a Remove event won't be emitted until all file
// descriptors are closed, and deletes will always emit a Chmod. For example:
//
// fp := os.Open("file")
// os.Remove("file") // Triggers Chmod
// fp.Close() // Triggers Remove
//
// This is the event that inotify sends, so not much can be changed about this.
//
// The fs.inotify.max_user_watches sysctl variable specifies the upper limit
// for the number of watches per user, and fs.inotify.max_user_instances
// specifies the maximum number of inotify instances per user. Every Watcher you
// create is an "instance", and every path you add is a "watch".
//
// These are also exposed in /proc as /proc/sys/fs/inotify/max_user_watches and
// /proc/sys/fs/inotify/max_user_instances
//
// To increase them you can use sysctl or write the value to the /proc file:
//
// # Default values on Linux 5.18
// sysctl fs.inotify.max_user_watches=124983
// sysctl fs.inotify.max_user_instances=128
//
// To make the changes persist on reboot edit /etc/sysctl.conf or
// /usr/lib/sysctl.d/50-default.conf (details differ per Linux distro; check
// your distro's documentation):
//
// fs.inotify.max_user_watches=124983
// fs.inotify.max_user_instances=128
//
// Reaching the limit will result in a "no space left on device" or "too many open
// files" error.
//
// # kqueue notes (macOS, BSD)
//
// kqueue requires opening a file descriptor for every file that's being watched;
// so if you're watching a directory with five files then that's six file
// descriptors. You will run in to your system's "max open files" limit faster on
// these platforms.
//
// The sysctl variables kern.maxfiles and kern.maxfilesperproc can be used to
// control the maximum number of open files, as well as /etc/login.conf on BSD
// systems.
//
// # Windows notes
//
// Paths can be added as "C:\\path\\to\\dir", but forward slashes
// ("C:/path/to/dir") will also work.
//
// When a watched directory is removed it will always send an event for the
// directory itself, but may not send events for all files in that directory.
// Sometimes it will send events for all files, sometimes it will send no
// events, and often only for some files.
//
// The default ReadDirectoryChangesW() buffer size is 64K, which is the largest
// value that is guaranteed to work with SMB filesystems. If you have many
// events in quick succession this may not be enough, and you will have to use
// [WithBufferSize] to increase the value.
type Watcher struct {
b backend
// Events sends the filesystem change events.
//
// fsnotify can send the following events; a "path" here can refer to a
// file, directory, symbolic link, or special file like a FIFO.
//
// fsnotify.Create A new path was created; this may be followed by one
// or more Write events if data also gets written to a
// file.
//
// fsnotify.Remove A path was removed.
//
// fsnotify.Rename A path was renamed. A rename is always sent with the
// old path as Event.Name, and a Create event will be
// sent with the new name. Renames are only sent for
// paths that are currently watched; e.g. moving an
// unmonitored file into a monitored directory will
// show up as just a Create. Similarly, renaming a file
// to outside a monitored directory will show up as
// only a Rename.
//
// fsnotify.Write A file or named pipe was written to. A Truncate will
// also trigger a Write. A single "write action"
// initiated by the user may show up as one or multiple
// writes, depending on when the system syncs things to
// disk. For example when compiling a large Go program
// you may get hundreds of Write events, and you may
// want to wait until you've stopped receiving them
// (see the dedup example in cmd/fsnotify).
//
// Some systems may send Write event for directories
// when the directory content changes.
//
// fsnotify.Chmod Attributes were changed. On Linux this is also sent
// when a file is removed (or more accurately, when a
// link to an inode is removed). On kqueue it's sent
// when a file is truncated. On Windows it's never
// sent.
Events chan Event
// Errors sends any errors.
Errors chan error
}
// Event represents a file system notification.
type Event struct {
// Path to the file or directory.
@ -30,6 +157,16 @@ type Event struct {
// This is a bitmask and some systems may send multiple operations at once.
// Use the Event.Has() method instead of comparing with ==.
Op Op
// Create events will have this set to the old path if it's a rename. This
// only works when both the source and destination are watched. It's not
// reliable when watching individual files, only directories.
//
// For example "mv /tmp/file /tmp/rename" will emit:
//
// Event{Op: Rename, Name: "/tmp/file"}
// Event{Op: Create, Name: "/tmp/rename", RenamedFrom: "/tmp/file"}
renamedFrom string
}
// Op describes a set of file operations.
@ -50,7 +187,7 @@ const (
// example "remove to trash" is often a rename).
Remove
// The path was renamed to something else; any watched on it will be
// The path was renamed to something else; any watches on it will be
// removed.
Rename
@ -60,15 +197,155 @@ const (
// get triggered very frequently by some software. For example, Spotlight
// indexing on macOS, anti-virus software, backup software, etc.
Chmod
// File descriptor was opened.
//
// Only works on Linux and FreeBSD.
xUnportableOpen
// File was read from.
//
// Only works on Linux and FreeBSD.
xUnportableRead
// File opened for writing was closed.
//
// Only works on Linux and FreeBSD.
//
// The advantage of using this over Write is that it's more reliable than
// waiting for Write events to stop. It's also faster (if you're not
// listening to Write events): copying a file of a few GB can easily
// generate tens of thousands of Write events in a short span of time.
xUnportableCloseWrite
// File opened for reading was closed.
//
// Only works on Linux and FreeBSD.
xUnportableCloseRead
)
// Common errors that can be reported.
var (
// ErrNonExistentWatch is used when Remove() is called on a path that's not
// added.
ErrNonExistentWatch = errors.New("fsnotify: can't remove non-existent watch")
ErrEventOverflow = errors.New("fsnotify: queue or buffer overflow")
ErrClosed = errors.New("fsnotify: watcher already closed")
// ErrClosed is used when trying to operate on a closed Watcher.
ErrClosed = errors.New("fsnotify: watcher already closed")
// ErrEventOverflow is reported from the Errors channel when there are too
// many events:
//
// - inotify: inotify returns IN_Q_OVERFLOW because there are too
// many queued events (the fs.inotify.max_queued_events
// sysctl can be used to increase this).
// - windows: The buffer size is too small; WithBufferSize() can be used to increase it.
// - kqueue, fen: Not used.
ErrEventOverflow = errors.New("fsnotify: queue or buffer overflow")
// ErrUnsupported is returned by AddWith() when WithOps() specified an
// Unportable event that's not supported on this platform.
xErrUnsupported = errors.New("fsnotify: not supported with this backend")
)
// NewWatcher creates a new Watcher.
func NewWatcher() (*Watcher, error) {
ev, errs := make(chan Event), make(chan error)
b, err := newBackend(ev, errs)
if err != nil {
return nil, err
}
return &Watcher{b: b, Events: ev, Errors: errs}, nil
}
// NewBufferedWatcher creates a new Watcher with a buffered Watcher.Events
// channel.
//
// The main use case for this is situations with a very large number of events
// where the kernel buffer size can't be increased (e.g. due to lack of
// permissions). An unbuffered Watcher will perform better for almost all use
// cases, and whenever possible you will be better off increasing the kernel
// buffers instead of adding a large userspace buffer.
func NewBufferedWatcher(sz uint) (*Watcher, error) {
ev, errs := make(chan Event), make(chan error)
b, err := newBufferedBackend(sz, ev, errs)
if err != nil {
return nil, err
}
return &Watcher{b: b, Events: ev, Errors: errs}, nil
}
// Add starts monitoring the path for changes.
//
// A path can only be watched once; watching it more than once is a no-op and will
// not return an error. Paths that do not yet exist on the filesystem cannot be
// watched.
//
// A watch will be automatically removed if the watched path is deleted or
// renamed. The exception is the Windows backend, which doesn't remove the
// watcher on renames.
//
// Notifications on network filesystems (NFS, SMB, FUSE, etc.) or special
// filesystems (/proc, /sys, etc.) generally don't work.
//
// Returns [ErrClosed] if [Watcher.Close] was called.
//
// See [Watcher.AddWith] for a version that allows adding options.
//
// # Watching directories
//
// All files in a directory are monitored, including new files that are created
// after the watcher is started. Subdirectories are not watched (i.e. it's
// non-recursive).
//
// # Watching files
//
// Watching individual files (rather than directories) is generally not
// recommended as many programs (especially editors) update files atomically: it
// will write to a temporary file which is then moved to destination,
// overwriting the original (or some variant thereof). The watcher on the
// original file is now lost, as that no longer exists.
//
// The upshot of this is that a power failure or crash won't leave a
// half-written file.
//
// Watch the parent directory and use Event.Name to filter out files you're not
// interested in. There is an example of this in cmd/fsnotify/file.go.
func (w *Watcher) Add(path string) error { return w.b.Add(path) }
// AddWith is like [Watcher.Add], but allows adding options. When using Add()
// the defaults described below are used.
//
// Possible options are:
//
// - [WithBufferSize] sets the buffer size for the Windows backend; no-op on
// other platforms. The default is 64K (65536 bytes).
func (w *Watcher) AddWith(path string, opts ...addOpt) error { return w.b.AddWith(path, opts...) }
// Remove stops monitoring the path for changes.
//
// Directories are always removed non-recursively. For example, if you added
// /tmp/dir and /tmp/dir/subdir then you will need to remove both.
//
// Removing a path that has not yet been added returns [ErrNonExistentWatch].
//
// Returns nil if [Watcher.Close] was called.
func (w *Watcher) Remove(path string) error { return w.b.Remove(path) }
// Close removes all watches and closes the Events channel.
func (w *Watcher) Close() error { return w.b.Close() }
// WatchList returns all paths explicitly added with [Watcher.Add] (and are not
// yet removed).
//
// Returns nil if [Watcher.Close] was called.
func (w *Watcher) WatchList() []string { return w.b.WatchList() }
// Supports reports if all the listed operations are supported by this platform.
//
// Create, Write, Remove, Rename, and Chmod are always supported. It can only
// return false for an Op starting with Unportable.
func (w *Watcher) xSupports(op Op) bool { return w.b.xSupports(op) }
func (o Op) String() string {
var b strings.Builder
if o.Has(Create) {
@ -80,6 +357,18 @@ func (o Op) String() string {
if o.Has(Write) {
b.WriteString("|WRITE")
}
if o.Has(xUnportableOpen) {
b.WriteString("|OPEN")
}
if o.Has(xUnportableRead) {
b.WriteString("|READ")
}
if o.Has(xUnportableCloseWrite) {
b.WriteString("|CLOSE_WRITE")
}
if o.Has(xUnportableCloseRead) {
b.WriteString("|CLOSE_READ")
}
if o.Has(Rename) {
b.WriteString("|RENAME")
}
@ -100,24 +389,48 @@ func (e Event) Has(op Op) bool { return e.Op.Has(op) }
// String returns a string representation of the event with their path.
func (e Event) String() string {
if e.renamedFrom != "" {
return fmt.Sprintf("%-13s %q ← %q", e.Op.String(), e.Name, e.renamedFrom)
}
return fmt.Sprintf("%-13s %q", e.Op.String(), e.Name)
}
type (
backend interface {
Add(string) error
AddWith(string, ...addOpt) error
Remove(string) error
WatchList() []string
Close() error
xSupports(Op) bool
}
addOpt func(opt *withOpts)
withOpts struct {
bufsize int
bufsize int
op Op
noFollow bool
sendCreate bool
}
)
var debug = func() bool {
// Check for exactly "1" (rather than mere existence) so we can add
// options/flags in the future. I don't know if we ever want that, but it's
// nice to leave the option open.
return os.Getenv("FSNOTIFY_DEBUG") == "1"
}()
var defaultOpts = withOpts{
bufsize: 65536, // 64K
op: Create | Write | Remove | Rename | Chmod,
}
func getOptions(opts ...addOpt) withOpts {
with := defaultOpts
for _, o := range opts {
o(&with)
if o != nil {
o(&with)
}
}
return with
}
@ -136,9 +449,44 @@ func WithBufferSize(bytes int) addOpt {
return func(opt *withOpts) { opt.bufsize = bytes }
}
// WithOps sets which operations to listen for. The default is [Create],
// [Write], [Remove], [Rename], and [Chmod].
//
// Excluding operations you're not interested in can save quite a bit of CPU
// time; in some use cases there may be hundreds of thousands of useless Write
// or Chmod operations per second.
//
// This can also be used to add unportable operations not supported by all
// platforms; unportable operations all start with "Unportable":
// [UnportableOpen], [UnportableRead], [UnportableCloseWrite], and
// [UnportableCloseRead].
//
// AddWith returns an error when using an unportable operation that's not
// supported. Use [Watcher.Support] to check for support.
func withOps(op Op) addOpt {
return func(opt *withOpts) { opt.op = op }
}
// WithNoFollow disables following symlinks, so the symlinks themselves are
// watched.
func withNoFollow() addOpt {
return func(opt *withOpts) { opt.noFollow = true }
}
// "Internal" option for recursive watches on inotify.
func withCreate() addOpt {
return func(opt *withOpts) { opt.sendCreate = true }
}
var enableRecurse = false
// Check if this path is recursive (ends with "/..." or "\..."), and return the
// path with the /... stripped.
func recursivePath(path string) (string, bool) {
path = filepath.Clean(path)
if !enableRecurse { // Only enabled in tests for now.
return path, false
}
if filepath.Base(path) == "..." {
return filepath.Dir(path), true
}

39
vendor/github.com/fsnotify/fsnotify/internal/darwin.go generated vendored Normal file
View file

@ -0,0 +1,39 @@
//go:build darwin
package internal
import (
"syscall"
"golang.org/x/sys/unix"
)
var (
SyscallEACCES = syscall.EACCES
UnixEACCES = unix.EACCES
)
var maxfiles uint64
// Go 1.19 will do this automatically: https://go-review.googlesource.com/c/go/+/393354/
func SetRlimit() {
var l syscall.Rlimit
err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &l)
if err == nil && l.Cur != l.Max {
l.Cur = l.Max
syscall.Setrlimit(syscall.RLIMIT_NOFILE, &l)
}
maxfiles = l.Cur
if n, err := syscall.SysctlUint32("kern.maxfiles"); err == nil && uint64(n) < maxfiles {
maxfiles = uint64(n)
}
if n, err := syscall.SysctlUint32("kern.maxfilesperproc"); err == nil && uint64(n) < maxfiles {
maxfiles = uint64(n)
}
}
func Maxfiles() uint64 { return maxfiles }
func Mkfifo(path string, mode uint32) error { return unix.Mkfifo(path, mode) }
func Mknod(path string, mode uint32, dev int) error { return unix.Mknod(path, mode, dev) }

View file

@ -0,0 +1,57 @@
package internal
import "golang.org/x/sys/unix"
var names = []struct {
n string
m uint32
}{
{"NOTE_ABSOLUTE", unix.NOTE_ABSOLUTE},
{"NOTE_ATTRIB", unix.NOTE_ATTRIB},
{"NOTE_BACKGROUND", unix.NOTE_BACKGROUND},
{"NOTE_CHILD", unix.NOTE_CHILD},
{"NOTE_CRITICAL", unix.NOTE_CRITICAL},
{"NOTE_DELETE", unix.NOTE_DELETE},
{"NOTE_EXEC", unix.NOTE_EXEC},
{"NOTE_EXIT", unix.NOTE_EXIT},
{"NOTE_EXITSTATUS", unix.NOTE_EXITSTATUS},
{"NOTE_EXIT_CSERROR", unix.NOTE_EXIT_CSERROR},
{"NOTE_EXIT_DECRYPTFAIL", unix.NOTE_EXIT_DECRYPTFAIL},
{"NOTE_EXIT_DETAIL", unix.NOTE_EXIT_DETAIL},
{"NOTE_EXIT_DETAIL_MASK", unix.NOTE_EXIT_DETAIL_MASK},
{"NOTE_EXIT_MEMORY", unix.NOTE_EXIT_MEMORY},
{"NOTE_EXIT_REPARENTED", unix.NOTE_EXIT_REPARENTED},
{"NOTE_EXTEND", unix.NOTE_EXTEND},
{"NOTE_FFAND", unix.NOTE_FFAND},
{"NOTE_FFCOPY", unix.NOTE_FFCOPY},
{"NOTE_FFCTRLMASK", unix.NOTE_FFCTRLMASK},
{"NOTE_FFLAGSMASK", unix.NOTE_FFLAGSMASK},
{"NOTE_FFNOP", unix.NOTE_FFNOP},
{"NOTE_FFOR", unix.NOTE_FFOR},
{"NOTE_FORK", unix.NOTE_FORK},
{"NOTE_FUNLOCK", unix.NOTE_FUNLOCK},
{"NOTE_LEEWAY", unix.NOTE_LEEWAY},
{"NOTE_LINK", unix.NOTE_LINK},
{"NOTE_LOWAT", unix.NOTE_LOWAT},
{"NOTE_MACHTIME", unix.NOTE_MACHTIME},
{"NOTE_MACH_CONTINUOUS_TIME", unix.NOTE_MACH_CONTINUOUS_TIME},
{"NOTE_NONE", unix.NOTE_NONE},
{"NOTE_NSECONDS", unix.NOTE_NSECONDS},
{"NOTE_OOB", unix.NOTE_OOB},
//{"NOTE_PCTRLMASK", unix.NOTE_PCTRLMASK}, -0x100000 (?!)
{"NOTE_PDATAMASK", unix.NOTE_PDATAMASK},
{"NOTE_REAP", unix.NOTE_REAP},
{"NOTE_RENAME", unix.NOTE_RENAME},
{"NOTE_REVOKE", unix.NOTE_REVOKE},
{"NOTE_SECONDS", unix.NOTE_SECONDS},
{"NOTE_SIGNAL", unix.NOTE_SIGNAL},
{"NOTE_TRACK", unix.NOTE_TRACK},
{"NOTE_TRACKERR", unix.NOTE_TRACKERR},
{"NOTE_TRIGGER", unix.NOTE_TRIGGER},
{"NOTE_USECONDS", unix.NOTE_USECONDS},
{"NOTE_VM_ERROR", unix.NOTE_VM_ERROR},
{"NOTE_VM_PRESSURE", unix.NOTE_VM_PRESSURE},
{"NOTE_VM_PRESSURE_SUDDEN_TERMINATE", unix.NOTE_VM_PRESSURE_SUDDEN_TERMINATE},
{"NOTE_VM_PRESSURE_TERMINATE", unix.NOTE_VM_PRESSURE_TERMINATE},
{"NOTE_WRITE", unix.NOTE_WRITE},
}

View file

@ -0,0 +1,33 @@
package internal
import "golang.org/x/sys/unix"
var names = []struct {
n string
m uint32
}{
{"NOTE_ATTRIB", unix.NOTE_ATTRIB},
{"NOTE_CHILD", unix.NOTE_CHILD},
{"NOTE_DELETE", unix.NOTE_DELETE},
{"NOTE_EXEC", unix.NOTE_EXEC},
{"NOTE_EXIT", unix.NOTE_EXIT},
{"NOTE_EXTEND", unix.NOTE_EXTEND},
{"NOTE_FFAND", unix.NOTE_FFAND},
{"NOTE_FFCOPY", unix.NOTE_FFCOPY},
{"NOTE_FFCTRLMASK", unix.NOTE_FFCTRLMASK},
{"NOTE_FFLAGSMASK", unix.NOTE_FFLAGSMASK},
{"NOTE_FFNOP", unix.NOTE_FFNOP},
{"NOTE_FFOR", unix.NOTE_FFOR},
{"NOTE_FORK", unix.NOTE_FORK},
{"NOTE_LINK", unix.NOTE_LINK},
{"NOTE_LOWAT", unix.NOTE_LOWAT},
{"NOTE_OOB", unix.NOTE_OOB},
{"NOTE_PCTRLMASK", unix.NOTE_PCTRLMASK},
{"NOTE_PDATAMASK", unix.NOTE_PDATAMASK},
{"NOTE_RENAME", unix.NOTE_RENAME},
{"NOTE_REVOKE", unix.NOTE_REVOKE},
{"NOTE_TRACK", unix.NOTE_TRACK},
{"NOTE_TRACKERR", unix.NOTE_TRACKERR},
{"NOTE_TRIGGER", unix.NOTE_TRIGGER},
{"NOTE_WRITE", unix.NOTE_WRITE},
}

View file

@ -0,0 +1,42 @@
package internal
import "golang.org/x/sys/unix"
var names = []struct {
n string
m uint32
}{
{"NOTE_ABSTIME", unix.NOTE_ABSTIME},
{"NOTE_ATTRIB", unix.NOTE_ATTRIB},
{"NOTE_CHILD", unix.NOTE_CHILD},
{"NOTE_CLOSE", unix.NOTE_CLOSE},
{"NOTE_CLOSE_WRITE", unix.NOTE_CLOSE_WRITE},
{"NOTE_DELETE", unix.NOTE_DELETE},
{"NOTE_EXEC", unix.NOTE_EXEC},
{"NOTE_EXIT", unix.NOTE_EXIT},
{"NOTE_EXTEND", unix.NOTE_EXTEND},
{"NOTE_FFAND", unix.NOTE_FFAND},
{"NOTE_FFCOPY", unix.NOTE_FFCOPY},
{"NOTE_FFCTRLMASK", unix.NOTE_FFCTRLMASK},
{"NOTE_FFLAGSMASK", unix.NOTE_FFLAGSMASK},
{"NOTE_FFNOP", unix.NOTE_FFNOP},
{"NOTE_FFOR", unix.NOTE_FFOR},
{"NOTE_FILE_POLL", unix.NOTE_FILE_POLL},
{"NOTE_FORK", unix.NOTE_FORK},
{"NOTE_LINK", unix.NOTE_LINK},
{"NOTE_LOWAT", unix.NOTE_LOWAT},
{"NOTE_MSECONDS", unix.NOTE_MSECONDS},
{"NOTE_NSECONDS", unix.NOTE_NSECONDS},
{"NOTE_OPEN", unix.NOTE_OPEN},
{"NOTE_PCTRLMASK", unix.NOTE_PCTRLMASK},
{"NOTE_PDATAMASK", unix.NOTE_PDATAMASK},
{"NOTE_READ", unix.NOTE_READ},
{"NOTE_RENAME", unix.NOTE_RENAME},
{"NOTE_REVOKE", unix.NOTE_REVOKE},
{"NOTE_SECONDS", unix.NOTE_SECONDS},
{"NOTE_TRACK", unix.NOTE_TRACK},
{"NOTE_TRACKERR", unix.NOTE_TRACKERR},
{"NOTE_TRIGGER", unix.NOTE_TRIGGER},
{"NOTE_USECONDS", unix.NOTE_USECONDS},
{"NOTE_WRITE", unix.NOTE_WRITE},
}

View file

@ -0,0 +1,32 @@
//go:build freebsd || openbsd || netbsd || dragonfly || darwin
package internal
import (
"fmt"
"os"
"strings"
"time"
"golang.org/x/sys/unix"
)
func Debug(name string, kevent *unix.Kevent_t) {
mask := uint32(kevent.Fflags)
var (
l []string
unknown = mask
)
for _, n := range names {
if mask&n.m == n.m {
l = append(l, n.n)
unknown ^= n.m
}
}
if unknown > 0 {
l = append(l, fmt.Sprintf("0x%x", unknown))
}
fmt.Fprintf(os.Stderr, "FSNOTIFY_DEBUG: %s %10d:%-60s → %q\n",
time.Now().Format("15:04:05.000000000"), mask, strings.Join(l, " | "), name)
}

View file

@ -0,0 +1,56 @@
package internal
import (
"fmt"
"os"
"strings"
"time"
"golang.org/x/sys/unix"
)
func Debug(name string, mask, cookie uint32) {
names := []struct {
n string
m uint32
}{
{"IN_ACCESS", unix.IN_ACCESS},
{"IN_ATTRIB", unix.IN_ATTRIB},
{"IN_CLOSE", unix.IN_CLOSE},
{"IN_CLOSE_NOWRITE", unix.IN_CLOSE_NOWRITE},
{"IN_CLOSE_WRITE", unix.IN_CLOSE_WRITE},
{"IN_CREATE", unix.IN_CREATE},
{"IN_DELETE", unix.IN_DELETE},
{"IN_DELETE_SELF", unix.IN_DELETE_SELF},
{"IN_IGNORED", unix.IN_IGNORED},
{"IN_ISDIR", unix.IN_ISDIR},
{"IN_MODIFY", unix.IN_MODIFY},
{"IN_MOVE", unix.IN_MOVE},
{"IN_MOVED_FROM", unix.IN_MOVED_FROM},
{"IN_MOVED_TO", unix.IN_MOVED_TO},
{"IN_MOVE_SELF", unix.IN_MOVE_SELF},
{"IN_OPEN", unix.IN_OPEN},
{"IN_Q_OVERFLOW", unix.IN_Q_OVERFLOW},
{"IN_UNMOUNT", unix.IN_UNMOUNT},
}
var (
l []string
unknown = mask
)
for _, n := range names {
if mask&n.m == n.m {
l = append(l, n.n)
unknown ^= n.m
}
}
if unknown > 0 {
l = append(l, fmt.Sprintf("0x%x", unknown))
}
var c string
if cookie > 0 {
c = fmt.Sprintf("(cookie: %d) ", cookie)
}
fmt.Fprintf(os.Stderr, "FSNOTIFY_DEBUG: %s %-30s → %s%q\n",
time.Now().Format("15:04:05.000000000"), strings.Join(l, "|"), c, name)
}

View file

@ -0,0 +1,25 @@
package internal
import "golang.org/x/sys/unix"
var names = []struct {
n string
m uint32
}{
{"NOTE_ATTRIB", unix.NOTE_ATTRIB},
{"NOTE_CHILD", unix.NOTE_CHILD},
{"NOTE_DELETE", unix.NOTE_DELETE},
{"NOTE_EXEC", unix.NOTE_EXEC},
{"NOTE_EXIT", unix.NOTE_EXIT},
{"NOTE_EXTEND", unix.NOTE_EXTEND},
{"NOTE_FORK", unix.NOTE_FORK},
{"NOTE_LINK", unix.NOTE_LINK},
{"NOTE_LOWAT", unix.NOTE_LOWAT},
{"NOTE_PCTRLMASK", unix.NOTE_PCTRLMASK},
{"NOTE_PDATAMASK", unix.NOTE_PDATAMASK},
{"NOTE_RENAME", unix.NOTE_RENAME},
{"NOTE_REVOKE", unix.NOTE_REVOKE},
{"NOTE_TRACK", unix.NOTE_TRACK},
{"NOTE_TRACKERR", unix.NOTE_TRACKERR},
{"NOTE_WRITE", unix.NOTE_WRITE},
}

View file

@ -0,0 +1,28 @@
package internal
import "golang.org/x/sys/unix"
var names = []struct {
n string
m uint32
}{
{"NOTE_ATTRIB", unix.NOTE_ATTRIB},
// {"NOTE_CHANGE", unix.NOTE_CHANGE}, // Not on 386?
{"NOTE_CHILD", unix.NOTE_CHILD},
{"NOTE_DELETE", unix.NOTE_DELETE},
{"NOTE_EOF", unix.NOTE_EOF},
{"NOTE_EXEC", unix.NOTE_EXEC},
{"NOTE_EXIT", unix.NOTE_EXIT},
{"NOTE_EXTEND", unix.NOTE_EXTEND},
{"NOTE_FORK", unix.NOTE_FORK},
{"NOTE_LINK", unix.NOTE_LINK},
{"NOTE_LOWAT", unix.NOTE_LOWAT},
{"NOTE_PCTRLMASK", unix.NOTE_PCTRLMASK},
{"NOTE_PDATAMASK", unix.NOTE_PDATAMASK},
{"NOTE_RENAME", unix.NOTE_RENAME},
{"NOTE_REVOKE", unix.NOTE_REVOKE},
{"NOTE_TRACK", unix.NOTE_TRACK},
{"NOTE_TRACKERR", unix.NOTE_TRACKERR},
{"NOTE_TRUNCATE", unix.NOTE_TRUNCATE},
{"NOTE_WRITE", unix.NOTE_WRITE},
}

View file

@ -0,0 +1,45 @@
package internal
import (
"fmt"
"os"
"strings"
"time"
"golang.org/x/sys/unix"
)
func Debug(name string, mask int32) {
names := []struct {
n string
m int32
}{
{"FILE_ACCESS", unix.FILE_ACCESS},
{"FILE_MODIFIED", unix.FILE_MODIFIED},
{"FILE_ATTRIB", unix.FILE_ATTRIB},
{"FILE_TRUNC", unix.FILE_TRUNC},
{"FILE_NOFOLLOW", unix.FILE_NOFOLLOW},
{"FILE_DELETE", unix.FILE_DELETE},
{"FILE_RENAME_TO", unix.FILE_RENAME_TO},
{"FILE_RENAME_FROM", unix.FILE_RENAME_FROM},
{"UNMOUNTED", unix.UNMOUNTED},
{"MOUNTEDOVER", unix.MOUNTEDOVER},
{"FILE_EXCEPTION", unix.FILE_EXCEPTION},
}
var (
l []string
unknown = mask
)
for _, n := range names {
if mask&n.m == n.m {
l = append(l, n.n)
unknown ^= n.m
}
}
if unknown > 0 {
l = append(l, fmt.Sprintf("0x%x", unknown))
}
fmt.Fprintf(os.Stderr, "FSNOTIFY_DEBUG: %s %10d:%-30s → %q\n",
time.Now().Format("15:04:05.000000000"), mask, strings.Join(l, " | "), name)
}

View file

@ -0,0 +1,40 @@
package internal
import (
"fmt"
"os"
"path/filepath"
"strings"
"time"
"golang.org/x/sys/windows"
)
func Debug(name string, mask uint32) {
names := []struct {
n string
m uint32
}{
{"FILE_ACTION_ADDED", windows.FILE_ACTION_ADDED},
{"FILE_ACTION_REMOVED", windows.FILE_ACTION_REMOVED},
{"FILE_ACTION_MODIFIED", windows.FILE_ACTION_MODIFIED},
{"FILE_ACTION_RENAMED_OLD_NAME", windows.FILE_ACTION_RENAMED_OLD_NAME},
{"FILE_ACTION_RENAMED_NEW_NAME", windows.FILE_ACTION_RENAMED_NEW_NAME},
}
var (
l []string
unknown = mask
)
for _, n := range names {
if mask&n.m == n.m {
l = append(l, n.n)
unknown ^= n.m
}
}
if unknown > 0 {
l = append(l, fmt.Sprintf("0x%x", unknown))
}
fmt.Fprintf(os.Stderr, "FSNOTIFY_DEBUG: %s %-65s → %q\n",
time.Now().Format("15:04:05.000000000"), strings.Join(l, " | "), filepath.ToSlash(name))
}

View file

@ -0,0 +1,31 @@
//go:build freebsd
package internal
import (
"syscall"
"golang.org/x/sys/unix"
)
var (
SyscallEACCES = syscall.EACCES
UnixEACCES = unix.EACCES
)
var maxfiles uint64
func SetRlimit() {
// Go 1.19 will do this automatically: https://go-review.googlesource.com/c/go/+/393354/
var l syscall.Rlimit
err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &l)
if err == nil && l.Cur != l.Max {
l.Cur = l.Max
syscall.Setrlimit(syscall.RLIMIT_NOFILE, &l)
}
maxfiles = uint64(l.Cur)
}
func Maxfiles() uint64 { return maxfiles }
func Mkfifo(path string, mode uint32) error { return unix.Mkfifo(path, mode) }
func Mknod(path string, mode uint32, dev int) error { return unix.Mknod(path, mode, uint64(dev)) }

View file

@ -0,0 +1,2 @@
// Package internal contains some helpers.
package internal

31
vendor/github.com/fsnotify/fsnotify/internal/unix.go generated vendored Normal file
View file

@ -0,0 +1,31 @@
//go:build !windows && !darwin && !freebsd
package internal
import (
"syscall"
"golang.org/x/sys/unix"
)
var (
SyscallEACCES = syscall.EACCES
UnixEACCES = unix.EACCES
)
var maxfiles uint64
func SetRlimit() {
// Go 1.19 will do this automatically: https://go-review.googlesource.com/c/go/+/393354/
var l syscall.Rlimit
err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &l)
if err == nil && l.Cur != l.Max {
l.Cur = l.Max
syscall.Setrlimit(syscall.RLIMIT_NOFILE, &l)
}
maxfiles = uint64(l.Cur)
}
func Maxfiles() uint64 { return maxfiles }
func Mkfifo(path string, mode uint32) error { return unix.Mkfifo(path, mode) }
func Mknod(path string, mode uint32, dev int) error { return unix.Mknod(path, mode, dev) }

View file

@ -0,0 +1,7 @@
//go:build !windows
package internal
func HasPrivilegesForSymlink() bool {
return true
}

View file

@ -0,0 +1,41 @@
//go:build windows
package internal
import (
"errors"
"golang.org/x/sys/windows"
)
// Just a dummy.
var (
SyscallEACCES = errors.New("dummy")
UnixEACCES = errors.New("dummy")
)
func SetRlimit() {}
func Maxfiles() uint64 { return 1<<64 - 1 }
func Mkfifo(path string, mode uint32) error { return errors.New("no FIFOs on Windows") }
func Mknod(path string, mode uint32, dev int) error { return errors.New("no device nodes on Windows") }
func HasPrivilegesForSymlink() bool {
var sid *windows.SID
err := windows.AllocateAndInitializeSid(
&windows.SECURITY_NT_AUTHORITY,
2,
windows.SECURITY_BUILTIN_DOMAIN_RID,
windows.DOMAIN_ALIAS_RID_ADMINS,
0, 0, 0, 0, 0, 0,
&sid)
if err != nil {
return false
}
defer windows.FreeSid(sid)
token := windows.Token(0)
member, err := token.IsMember(sid)
if err != nil {
return false
}
return member || token.IsElevated()
}

View file

@ -1,259 +0,0 @@
#!/usr/bin/env zsh
[ "${ZSH_VERSION:-}" = "" ] && echo >&2 "Only works with zsh" && exit 1
setopt err_exit no_unset pipefail extended_glob
# Simple script to update the godoc comments on all watchers so you don't need
# to update the same comment 5 times.
watcher=$(<<EOF
// Watcher watches a set of paths, delivering events on a channel.
//
// A watcher should not be copied (e.g. pass it by pointer, rather than by
// value).
//
// # Linux notes
//
// When a file is removed a Remove event won't be emitted until all file
// descriptors are closed, and deletes will always emit a Chmod. For example:
//
// fp := os.Open("file")
// os.Remove("file") // Triggers Chmod
// fp.Close() // Triggers Remove
//
// This is the event that inotify sends, so not much can be changed about this.
//
// The fs.inotify.max_user_watches sysctl variable specifies the upper limit
// for the number of watches per user, and fs.inotify.max_user_instances
// specifies the maximum number of inotify instances per user. Every Watcher you
// create is an "instance", and every path you add is a "watch".
//
// These are also exposed in /proc as /proc/sys/fs/inotify/max_user_watches and
// /proc/sys/fs/inotify/max_user_instances
//
// To increase them you can use sysctl or write the value to the /proc file:
//
// # Default values on Linux 5.18
// sysctl fs.inotify.max_user_watches=124983
// sysctl fs.inotify.max_user_instances=128
//
// To make the changes persist on reboot edit /etc/sysctl.conf or
// /usr/lib/sysctl.d/50-default.conf (details differ per Linux distro; check
// your distro's documentation):
//
// fs.inotify.max_user_watches=124983
// fs.inotify.max_user_instances=128
//
// Reaching the limit will result in a "no space left on device" or "too many open
// files" error.
//
// # kqueue notes (macOS, BSD)
//
// kqueue requires opening a file descriptor for every file that's being watched;
// so if you're watching a directory with five files then that's six file
// descriptors. You will run in to your system's "max open files" limit faster on
// these platforms.
//
// The sysctl variables kern.maxfiles and kern.maxfilesperproc can be used to
// control the maximum number of open files, as well as /etc/login.conf on BSD
// systems.
//
// # Windows notes
//
// Paths can be added as "C:\\path\\to\\dir", but forward slashes
// ("C:/path/to/dir") will also work.
//
// When a watched directory is removed it will always send an event for the
// directory itself, but may not send events for all files in that directory.
// Sometimes it will send events for all times, sometimes it will send no
// events, and often only for some files.
//
// The default ReadDirectoryChangesW() buffer size is 64K, which is the largest
// value that is guaranteed to work with SMB filesystems. If you have many
// events in quick succession this may not be enough, and you will have to use
// [WithBufferSize] to increase the value.
EOF
)
new=$(<<EOF
// NewWatcher creates a new Watcher.
EOF
)
newbuffered=$(<<EOF
// NewBufferedWatcher creates a new Watcher with a buffered Watcher.Events
// channel.
//
// The main use case for this is situations with a very large number of events
// where the kernel buffer size can't be increased (e.g. due to lack of
// permissions). An unbuffered Watcher will perform better for almost all use
// cases, and whenever possible you will be better off increasing the kernel
// buffers instead of adding a large userspace buffer.
EOF
)
add=$(<<EOF
// Add starts monitoring the path for changes.
//
// A path can only be watched once; watching it more than once is a no-op and will
// not return an error. Paths that do not yet exist on the filesystem cannot be
// watched.
//
// A watch will be automatically removed if the watched path is deleted or
// renamed. The exception is the Windows backend, which doesn't remove the
// watcher on renames.
//
// Notifications on network filesystems (NFS, SMB, FUSE, etc.) or special
// filesystems (/proc, /sys, etc.) generally don't work.
//
// Returns [ErrClosed] if [Watcher.Close] was called.
//
// See [Watcher.AddWith] for a version that allows adding options.
//
// # Watching directories
//
// All files in a directory are monitored, including new files that are created
// after the watcher is started. Subdirectories are not watched (i.e. it's
// non-recursive).
//
// # Watching files
//
// Watching individual files (rather than directories) is generally not
// recommended as many programs (especially editors) update files atomically: it
// will write to a temporary file which is then moved to to destination,
// overwriting the original (or some variant thereof). The watcher on the
// original file is now lost, as that no longer exists.
//
// The upshot of this is that a power failure or crash won't leave a
// half-written file.
//
// Watch the parent directory and use Event.Name to filter out files you're not
// interested in. There is an example of this in cmd/fsnotify/file.go.
EOF
)
addwith=$(<<EOF
// AddWith is like [Watcher.Add], but allows adding options. When using Add()
// the defaults described below are used.
//
// Possible options are:
//
// - [WithBufferSize] sets the buffer size for the Windows backend; no-op on
// other platforms. The default is 64K (65536 bytes).
EOF
)
remove=$(<<EOF
// Remove stops monitoring the path for changes.
//
// Directories are always removed non-recursively. For example, if you added
// /tmp/dir and /tmp/dir/subdir then you will need to remove both.
//
// Removing a path that has not yet been added returns [ErrNonExistentWatch].
//
// Returns nil if [Watcher.Close] was called.
EOF
)
close=$(<<EOF
// Close removes all watches and closes the Events channel.
EOF
)
watchlist=$(<<EOF
// WatchList returns all paths explicitly added with [Watcher.Add] (and are not
// yet removed).
//
// Returns nil if [Watcher.Close] was called.
EOF
)
events=$(<<EOF
// Events sends the filesystem change events.
//
// fsnotify can send the following events; a "path" here can refer to a
// file, directory, symbolic link, or special file like a FIFO.
//
// fsnotify.Create A new path was created; this may be followed by one
// or more Write events if data also gets written to a
// file.
//
// fsnotify.Remove A path was removed.
//
// fsnotify.Rename A path was renamed. A rename is always sent with the
// old path as Event.Name, and a Create event will be
// sent with the new name. Renames are only sent for
// paths that are currently watched; e.g. moving an
// unmonitored file into a monitored directory will
// show up as just a Create. Similarly, renaming a file
// to outside a monitored directory will show up as
// only a Rename.
//
// fsnotify.Write A file or named pipe was written to. A Truncate will
// also trigger a Write. A single "write action"
// initiated by the user may show up as one or multiple
// writes, depending on when the system syncs things to
// disk. For example when compiling a large Go program
// you may get hundreds of Write events, and you may
// want to wait until you've stopped receiving them
// (see the dedup example in cmd/fsnotify).
//
// Some systems may send Write event for directories
// when the directory content changes.
//
// fsnotify.Chmod Attributes were changed. On Linux this is also sent
// when a file is removed (or more accurately, when a
// link to an inode is removed). On kqueue it's sent
// when a file is truncated. On Windows it's never
// sent.
EOF
)
errors=$(<<EOF
// Errors sends any errors.
//
// ErrEventOverflow is used to indicate there are too many events:
//
// - inotify: There are too many queued events (fs.inotify.max_queued_events sysctl)
// - windows: The buffer size is too small; WithBufferSize() can be used to increase it.
// - kqueue, fen: Not used.
EOF
)
set-cmt() {
local pat=$1
local cmt=$2
IFS=$'\n' local files=($(grep -n $pat backend_*~*_test.go))
for f in $files; do
IFS=':' local fields=($=f)
local file=$fields[1]
local end=$(( $fields[2] - 1 ))
# Find start of comment.
local start=0
IFS=$'\n' local lines=($(head -n$end $file))
for (( i = 1; i <= $#lines; i++ )); do
local line=$lines[-$i]
if ! grep -q '^[[:space:]]*//' <<<$line; then
start=$(( end - (i - 2) ))
break
fi
done
head -n $(( start - 1 )) $file >/tmp/x
print -r -- $cmt >>/tmp/x
tail -n+$(( end + 1 )) $file >>/tmp/x
mv /tmp/x $file
done
}
set-cmt '^type Watcher struct ' $watcher
set-cmt '^func NewWatcher(' $new
set-cmt '^func NewBufferedWatcher(' $newbuffered
set-cmt '^func (w \*Watcher) Add(' $add
set-cmt '^func (w \*Watcher) AddWith(' $addwith
set-cmt '^func (w \*Watcher) Remove(' $remove
set-cmt '^func (w \*Watcher) Close(' $close
set-cmt '^func (w \*Watcher) WatchList(' $watchlist
set-cmt '^[[:space:]]*Events *chan Event$' $events
set-cmt '^[[:space:]]*Errors *chan error$' $errors

View file

@ -1,5 +1,4 @@
//go:build freebsd || openbsd || netbsd || dragonfly
// +build freebsd openbsd netbsd dragonfly
package fsnotify

View file

@ -1,5 +1,4 @@
//go:build darwin
// +build darwin
package fsnotify

View file

@ -1,205 +0,0 @@
## Changelog
### [1.8.7](https://github.com/magiconair/properties/tree/v1.8.7) - 08 Dec 2022
* [PR #65](https://github.com/magiconair/properties/pull/65): Speedup Merge
Thanks to [@AdityaVallabh](https://github.com/AdityaVallabh) for the patch.
* [PR #66](https://github.com/magiconair/properties/pull/66): use github actions
### [1.8.6](https://github.com/magiconair/properties/tree/v1.8.6) - 23 Feb 2022
* [PR #57](https://github.com/magiconair/properties/pull/57):Fix "unreachable code" lint error
Thanks to [@ellie](https://github.com/ellie) for the patch.
* [PR #63](https://github.com/magiconair/properties/pull/63): Make TestMustGetParsedDuration backwards compatible
This patch ensures that the `TestMustGetParsedDuration` still works with `go1.3` to make the
author happy until it affects real users.
Thanks to [@maage](https://github.com/maage) for the patch.
### [1.8.5](https://github.com/magiconair/properties/tree/v1.8.5) - 24 Mar 2021
* [PR #55](https://github.com/magiconair/properties/pull/55): Fix: Encoding Bug in Comments
When reading comments \ are loaded correctly, but when writing they are then
replaced by \\. This leads to wrong comments when writing and reading multiple times.
Thanks to [@doxsch](https://github.com/doxsch) for the patch.
### [1.8.4](https://github.com/magiconair/properties/tree/v1.8.4) - 23 Sep 2020
* [PR #50](https://github.com/magiconair/properties/pull/50): enhance error message for circular references
Thanks to [@sriv](https://github.com/sriv) for the patch.
### [1.8.3](https://github.com/magiconair/properties/tree/v1.8.3) - 14 Sep 2020
* [PR #49](https://github.com/magiconair/properties/pull/49): Include the key in error message causing the circular reference
The change is include the key in the error message which is causing the circular
reference when parsing/loading the properties files.
Thanks to [@haroon-sheikh](https://github.com/haroon-sheikh) for the patch.
### [1.8.2](https://github.com/magiconair/properties/tree/v1.8.2) - 25 Aug 2020
* [PR #36](https://github.com/magiconair/properties/pull/36): Escape backslash on write
This patch ensures that backslashes are escaped on write. Existing applications which
rely on the old behavior may need to be updated.
Thanks to [@apesternikov](https://github.com/apesternikov) for the patch.
* [PR #42](https://github.com/magiconair/properties/pull/42): Made Content-Type check whitespace agnostic in LoadURL()
Thanks to [@aliras1](https://github.com/aliras1) for the patch.
* [PR #41](https://github.com/magiconair/properties/pull/41): Make key/value separator configurable on Write()
Thanks to [@mkjor](https://github.com/mkjor) for the patch.
* [PR #40](https://github.com/magiconair/properties/pull/40): Add method to return a sorted list of keys
Thanks to [@mkjor](https://github.com/mkjor) for the patch.
### [1.8.1](https://github.com/magiconair/properties/tree/v1.8.1) - 10 May 2019
* [PR #35](https://github.com/magiconair/properties/pull/35): Close body always after request
This patch ensures that in `LoadURL` the response body is always closed.
Thanks to [@liubog2008](https://github.com/liubog2008) for the patch.
### [1.8](https://github.com/magiconair/properties/tree/v1.8) - 15 May 2018
* [PR #26](https://github.com/magiconair/properties/pull/26): Disable expansion during loading
This adds the option to disable property expansion during loading.
Thanks to [@kmala](https://github.com/kmala) for the patch.
### [1.7.6](https://github.com/magiconair/properties/tree/v1.7.6) - 14 Feb 2018
* [PR #29](https://github.com/magiconair/properties/pull/29): Reworked expansion logic to handle more complex cases.
See PR for an example.
Thanks to [@yobert](https://github.com/yobert) for the fix.
### [1.7.5](https://github.com/magiconair/properties/tree/v1.7.5) - 13 Feb 2018
* [PR #28](https://github.com/magiconair/properties/pull/28): Support duplicate expansions in the same value
Values which expand the same key multiple times (e.g. `key=${a} ${a}`) will no longer fail
with a `circular reference error`.
Thanks to [@yobert](https://github.com/yobert) for the fix.
### [1.7.4](https://github.com/magiconair/properties/tree/v1.7.4) - 31 Oct 2017
* [Issue #23](https://github.com/magiconair/properties/issues/23): Ignore blank lines with whitespaces
* [PR #24](https://github.com/magiconair/properties/pull/24): Update keys when DisableExpansion is enabled
Thanks to [@mgurov](https://github.com/mgurov) for the fix.
### [1.7.3](https://github.com/magiconair/properties/tree/v1.7.3) - 10 Jul 2017
* [Issue #17](https://github.com/magiconair/properties/issues/17): Add [SetValue()](http://godoc.org/github.com/magiconair/properties#Properties.SetValue) method to set values generically
* [Issue #22](https://github.com/magiconair/properties/issues/22): Add [LoadMap()](http://godoc.org/github.com/magiconair/properties#LoadMap) function to load properties from a string map
### [1.7.2](https://github.com/magiconair/properties/tree/v1.7.2) - 20 Mar 2017
* [Issue #15](https://github.com/magiconair/properties/issues/15): Drop gocheck dependency
* [PR #21](https://github.com/magiconair/properties/pull/21): Add [Map()](http://godoc.org/github.com/magiconair/properties#Properties.Map) and [FilterFunc()](http://godoc.org/github.com/magiconair/properties#Properties.FilterFunc)
### [1.7.1](https://github.com/magiconair/properties/tree/v1.7.1) - 13 Jan 2017
* [Issue #14](https://github.com/magiconair/properties/issues/14): Decouple TestLoadExpandedFile from `$USER`
* [PR #12](https://github.com/magiconair/properties/pull/12): Load from files and URLs
* [PR #16](https://github.com/magiconair/properties/pull/16): Keep gofmt happy
* [PR #18](https://github.com/magiconair/properties/pull/18): Fix Delete() function
### [1.7.0](https://github.com/magiconair/properties/tree/v1.7.0) - 20 Mar 2016
* [Issue #10](https://github.com/magiconair/properties/issues/10): Add [LoadURL,LoadURLs,MustLoadURL,MustLoadURLs](http://godoc.org/github.com/magiconair/properties#LoadURL) method to load properties from a URL.
* [Issue #11](https://github.com/magiconair/properties/issues/11): Add [LoadString,MustLoadString](http://godoc.org/github.com/magiconair/properties#LoadString) method to load properties from an UTF8 string.
* [PR #8](https://github.com/magiconair/properties/pull/8): Add [MustFlag](http://godoc.org/github.com/magiconair/properties#Properties.MustFlag) method to provide overrides via command line flags. (@pascaldekloe)
### [1.6.0](https://github.com/magiconair/properties/tree/v1.6.0) - 11 Dec 2015
* Add [Decode](http://godoc.org/github.com/magiconair/properties#Properties.Decode) method to populate struct from properties via tags.
### [1.5.6](https://github.com/magiconair/properties/tree/v1.5.6) - 18 Oct 2015
* Vendored in gopkg.in/check.v1
### [1.5.5](https://github.com/magiconair/properties/tree/v1.5.5) - 31 Jul 2015
* [PR #6](https://github.com/magiconair/properties/pull/6): Add [Delete](http://godoc.org/github.com/magiconair/properties#Properties.Delete) method to remove keys including comments. (@gerbenjacobs)
### [1.5.4](https://github.com/magiconair/properties/tree/v1.5.4) - 23 Jun 2015
* [Issue #5](https://github.com/magiconair/properties/issues/5): Allow disabling of property expansion [DisableExpansion](http://godoc.org/github.com/magiconair/properties#Properties.DisableExpansion). When property expansion is disabled Properties become a simple key/value store and don't check for circular references.
### [1.5.3](https://github.com/magiconair/properties/tree/v1.5.3) - 02 Jun 2015
* [Issue #4](https://github.com/magiconair/properties/issues/4): Maintain key order in [Filter()](http://godoc.org/github.com/magiconair/properties#Properties.Filter), [FilterPrefix()](http://godoc.org/github.com/magiconair/properties#Properties.FilterPrefix) and [FilterRegexp()](http://godoc.org/github.com/magiconair/properties#Properties.FilterRegexp)
### [1.5.2](https://github.com/magiconair/properties/tree/v1.5.2) - 10 Apr 2015
* [Issue #3](https://github.com/magiconair/properties/issues/3): Don't print comments in [WriteComment()](http://godoc.org/github.com/magiconair/properties#Properties.WriteComment) if they are all empty
* Add clickable links to README
### [1.5.1](https://github.com/magiconair/properties/tree/v1.5.1) - 08 Dec 2014
* Added [GetParsedDuration()](http://godoc.org/github.com/magiconair/properties#Properties.GetParsedDuration) and [MustGetParsedDuration()](http://godoc.org/github.com/magiconair/properties#Properties.MustGetParsedDuration) for values specified compatible with
[time.ParseDuration()](http://golang.org/pkg/time/#ParseDuration).
### [1.5.0](https://github.com/magiconair/properties/tree/v1.5.0) - 18 Nov 2014
* Added support for single and multi-line comments (reading, writing and updating)
* The order of keys is now preserved
* Calling [Set()](http://godoc.org/github.com/magiconair/properties#Properties.Set) with an empty key now silently ignores the call and does not create a new entry
* Added a [MustSet()](http://godoc.org/github.com/magiconair/properties#Properties.MustSet) method
* Migrated test library from launchpad.net/gocheck to [gopkg.in/check.v1](http://gopkg.in/check.v1)
### [1.4.2](https://github.com/magiconair/properties/tree/v1.4.2) - 15 Nov 2014
* [Issue #2](https://github.com/magiconair/properties/issues/2): Fixed goroutine leak in parser which created two lexers but cleaned up only one
### [1.4.1](https://github.com/magiconair/properties/tree/v1.4.1) - 13 Nov 2014
* [Issue #1](https://github.com/magiconair/properties/issues/1): Fixed bug in Keys() method which returned an empty string
### [1.4.0](https://github.com/magiconair/properties/tree/v1.4.0) - 23 Sep 2014
* Added [Keys()](http://godoc.org/github.com/magiconair/properties#Properties.Keys) to get the keys
* Added [Filter()](http://godoc.org/github.com/magiconair/properties#Properties.Filter), [FilterRegexp()](http://godoc.org/github.com/magiconair/properties#Properties.FilterRegexp) and [FilterPrefix()](http://godoc.org/github.com/magiconair/properties#Properties.FilterPrefix) to get a subset of the properties
### [1.3.0](https://github.com/magiconair/properties/tree/v1.3.0) - 18 Mar 2014
* Added support for time.Duration
* Made MustXXX() failure beha[ior configurable (log.Fatal, panic](https://github.com/magiconair/properties/tree/vior configurable (log.Fatal, panic) - custom)
* Changed default of MustXXX() failure from panic to log.Fatal
### [1.2.0](https://github.com/magiconair/properties/tree/v1.2.0) - 05 Mar 2014
* Added MustGet... functions
* Added support for int and uint with range checks on 32 bit platforms
### [1.1.0](https://github.com/magiconair/properties/tree/v1.1.0) - 20 Jan 2014
* Renamed from goproperties to properties
* Added support for expansion of environment vars in
filenames and value expressions
* Fixed bug where value expressions were not at the
start of the string
### [1.0.0](https://github.com/magiconair/properties/tree/v1.0.0) - 7 Jan 2014
* Initial release

View file

@ -1,12 +1,9 @@
[![](https://img.shields.io/github/tag/magiconair/properties.svg?style=flat-square&label=release)](https://github.com/magiconair/properties/releases)
[![Travis CI Status](https://img.shields.io/travis/magiconair/properties.svg?branch=master&style=flat-square&label=travis)](https://travis-ci.org/magiconair/properties)
[![License](https://img.shields.io/badge/License-BSD%202--Clause-orange.svg?style=flat-square)](https://raw.githubusercontent.com/magiconair/properties/master/LICENSE)
[![GoDoc](http://img.shields.io/badge/godoc-reference-5272B4.svg?style=flat-square)](http://godoc.org/github.com/magiconair/properties)
# Overview
#### Please run `git pull --tags` to update the tags. See [below](#updated-git-tags) why.
properties is a Go library for reading and writing properties files.
It supports reading from multiple files or URLs and Spring style recursive
@ -99,30 +96,3 @@ $ go get -u github.com/magiconair/properties
## ToDo
* Dump contents with passwords and secrets obscured
## Updated Git tags
#### 13 Feb 2018
I realized that all of the git tags I had pushed before v1.7.5 were lightweight tags
and I've only recently learned that this doesn't play well with `git describe` 😞
I have replaced all lightweight tags with signed tags using this script which should
retain the commit date, name and email address. Please run `git pull --tags` to update them.
Worst case you have to reclone the repo.
```shell
#!/bin/bash
tag=$1
echo "Updating $tag"
date=$(git show ${tag}^0 --format=%aD | head -1)
email=$(git show ${tag}^0 --format=%aE | head -1)
name=$(git show ${tag}^0 --format=%aN | head -1)
GIT_COMMITTER_DATE="$date" GIT_COMMITTER_NAME="$name" GIT_COMMITTER_EMAIL="$email" git tag -s -f ${tag} ${tag}^0 -m ${tag}
```
I apologize for the inconvenience.
Frank

View file

@ -189,12 +189,12 @@ func dec(p *Properties, key string, def *string, opts map[string]string, v refle
for i := 0; i < v.NumField(); i++ {
fv := v.Field(i)
fk, def, opts := keydef(t.Field(i))
if !fv.CanSet() {
return fmt.Errorf("cannot set %s", t.Field(i).Name)
}
if fk == "-" {
continue
}
if !fv.CanSet() {
return fmt.Errorf("cannot set %s", t.Field(i).Name)
}
if key != "" {
fk = key + "." + fk
}

View file

@ -6,7 +6,7 @@ package properties
import (
"fmt"
"io/ioutil"
"io"
"net/http"
"os"
"strings"
@ -52,6 +52,15 @@ func (l *Loader) LoadBytes(buf []byte) (*Properties, error) {
return l.loadBytes(buf, l.Encoding)
}
// LoadReader reads an io.Reader into a Properties struct.
func (l *Loader) LoadReader(r io.Reader) (*Properties, error) {
if buf, err := io.ReadAll(r); err != nil {
return nil, err
} else {
return l.loadBytes(buf, l.Encoding)
}
}
// LoadAll reads the content of multiple URLs or files in the given order into
// a Properties struct. If IgnoreMissing is true then a 404 status code or
// missing file will not be reported as error. Encoding sets the encoding for
@ -91,7 +100,7 @@ func (l *Loader) LoadAll(names []string) (*Properties, error) {
// If IgnoreMissing is true then a missing file will not be
// reported as error.
func (l *Loader) LoadFile(filename string) (*Properties, error) {
data, err := ioutil.ReadFile(filename)
data, err := os.ReadFile(filename)
if err != nil {
if l.IgnoreMissing && os.IsNotExist(err) {
LogPrintf("properties: %s not found. skipping", filename)
@ -126,7 +135,7 @@ func (l *Loader) LoadURL(url string) (*Properties, error) {
return nil, fmt.Errorf("properties: %s returned %d", url, resp.StatusCode)
}
body, err := ioutil.ReadAll(resp.Body)
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("properties: %s error reading response. %s", url, err)
}
@ -185,6 +194,12 @@ func LoadFile(filename string, enc Encoding) (*Properties, error) {
return l.LoadAll([]string{filename})
}
// LoadReader reads an io.Reader into a Properties struct.
func LoadReader(r io.Reader, enc Encoding) (*Properties, error) {
l := &Loader{Encoding: enc}
return l.LoadReader(r)
}
// LoadFiles reads multiple files in the given order into
// a Properties struct. If 'ignoreMissing' is true then
// non-existent files will not be reported as error.
@ -224,6 +239,12 @@ func MustLoadString(s string) *Properties {
return must(LoadString(s))
}
// MustLoadSReader reads an io.Reader into a Properties struct and
// panics on error.
func MustLoadReader(r io.Reader, enc Encoding) *Properties {
return must(LoadReader(r, enc))
}
// MustLoadFile reads a file into a Properties struct and
// panics on error.
func MustLoadFile(filename string, enc Encoding) *Properties {

View file

@ -1,3 +1,4 @@
version: 2
before:
hooks:
- go mod tidy

View file

@ -565,7 +565,7 @@ complete solutions exist out there.
## Versioning
Expect for parts explicitely marked otherwise, go-toml follows [Semantic
Expect for parts explicitly marked otherwise, go-toml follows [Semantic
Versioning](https://semver.org). The supported version of
[TOML](https://github.com/toml-lang/toml) is indicated at the beginning of this
document. The last two major versions of Go are supported (see [Go Release

View file

@ -8,7 +8,7 @@ import (
"io"
"math"
"reflect"
"sort"
"slices"
"strconv"
"strings"
"time"
@ -280,7 +280,7 @@ func (enc *Encoder) encode(b []byte, ctx encoderCtx, v reflect.Value) ([]byte, e
}
hasTextMarshaler := v.Type().Implements(textMarshalerType)
if hasTextMarshaler || (v.CanAddr() && reflect.PtrTo(v.Type()).Implements(textMarshalerType)) {
if hasTextMarshaler || (v.CanAddr() && reflect.PointerTo(v.Type()).Implements(textMarshalerType)) {
if !hasTextMarshaler {
v = v.Addr()
}
@ -631,6 +631,18 @@ func (enc *Encoder) keyToString(k reflect.Value) (string, error) {
return "", fmt.Errorf("toml: error marshalling key %v from text: %w", k, err)
}
return string(keyB), nil
case keyType.Kind() == reflect.Int || keyType.Kind() == reflect.Int8 || keyType.Kind() == reflect.Int16 || keyType.Kind() == reflect.Int32 || keyType.Kind() == reflect.Int64:
return strconv.FormatInt(k.Int(), 10), nil
case keyType.Kind() == reflect.Uint || keyType.Kind() == reflect.Uint8 || keyType.Kind() == reflect.Uint16 || keyType.Kind() == reflect.Uint32 || keyType.Kind() == reflect.Uint64:
return strconv.FormatUint(k.Uint(), 10), nil
case keyType.Kind() == reflect.Float32:
return strconv.FormatFloat(k.Float(), 'f', -1, 32), nil
case keyType.Kind() == reflect.Float64:
return strconv.FormatFloat(k.Float(), 'f', -1, 64), nil
}
return "", fmt.Errorf("toml: type %s is not supported as a map key", keyType.Kind())
}
@ -668,8 +680,8 @@ func (enc *Encoder) encodeMap(b []byte, ctx encoderCtx, v reflect.Value) ([]byte
}
func sortEntriesByKey(e []entry) {
sort.Slice(e, func(i, j int) bool {
return e[i].Key < e[j].Key
slices.SortFunc(e, func(a, b entry) int {
return strings.Compare(a.Key, b.Key)
})
}
@ -732,7 +744,7 @@ func walkStruct(ctx encoderCtx, t *table, v reflect.Value) {
if fieldType.Anonymous {
if fieldType.Type.Kind() == reflect.Struct {
walkStruct(ctx, t, f)
} else if fieldType.Type.Kind() == reflect.Pointer && !f.IsNil() && f.Elem().Kind() == reflect.Struct {
} else if fieldType.Type.Kind() == reflect.Ptr && !f.IsNil() && f.Elem().Kind() == reflect.Struct {
walkStruct(ctx, t, f.Elem())
}
continue
@ -951,7 +963,7 @@ func willConvertToTable(ctx encoderCtx, v reflect.Value) bool {
if !v.IsValid() {
return false
}
if v.Type() == timeType || v.Type().Implements(textMarshalerType) || (v.Kind() != reflect.Ptr && v.CanAddr() && reflect.PtrTo(v.Type()).Implements(textMarshalerType)) {
if v.Type() == timeType || v.Type().Implements(textMarshalerType) || (v.Kind() != reflect.Ptr && v.CanAddr() && reflect.PointerTo(v.Type()).Implements(textMarshalerType)) {
return false
}

View file

@ -5,9 +5,9 @@ import (
"errors"
"fmt"
"io"
"io/ioutil"
"math"
"reflect"
"strconv"
"strings"
"sync/atomic"
"time"
@ -21,10 +21,8 @@ import (
//
// It is a shortcut for Decoder.Decode() with the default options.
func Unmarshal(data []byte, v interface{}) error {
p := unstable.Parser{}
p.Reset(data)
d := decoder{p: &p}
d := decoder{}
d.p.Reset(data)
return d.FromParser(v)
}
@ -117,27 +115,25 @@ func (d *Decoder) EnableUnmarshalerInterface() *Decoder {
// Inline Table -> same as Table
// Array of Tables -> same as Array and Table
func (d *Decoder) Decode(v interface{}) error {
b, err := ioutil.ReadAll(d.r)
b, err := io.ReadAll(d.r)
if err != nil {
return fmt.Errorf("toml: %w", err)
}
p := unstable.Parser{}
p.Reset(b)
dec := decoder{
p: &p,
strict: strict{
Enabled: d.strict,
},
unmarshalerInterface: d.unmarshalerInterface,
}
dec.p.Reset(b)
return dec.FromParser(v)
}
type decoder struct {
// Which parser instance in use for this decoding session.
p *unstable.Parser
p unstable.Parser
// Flag indicating that the current expression is stashed.
// If set to true, calling nextExpr will not actually pull a new expression
@ -1078,12 +1074,39 @@ func (d *decoder) keyFromData(keyType reflect.Type, data []byte) (reflect.Value,
}
return mk, nil
case reflect.PtrTo(keyType).Implements(textUnmarshalerType):
case reflect.PointerTo(keyType).Implements(textUnmarshalerType):
mk := reflect.New(keyType)
if err := mk.Interface().(encoding.TextUnmarshaler).UnmarshalText(data); err != nil {
return reflect.Value{}, fmt.Errorf("toml: error unmarshalling key type %s from text: %w", stringType, err)
}
return mk.Elem(), nil
case keyType.Kind() == reflect.Int || keyType.Kind() == reflect.Int8 || keyType.Kind() == reflect.Int16 || keyType.Kind() == reflect.Int32 || keyType.Kind() == reflect.Int64:
key, err := strconv.ParseInt(string(data), 10, 64)
if err != nil {
return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from integer: %w", stringType, err)
}
return reflect.ValueOf(key).Convert(keyType), nil
case keyType.Kind() == reflect.Uint || keyType.Kind() == reflect.Uint8 || keyType.Kind() == reflect.Uint16 || keyType.Kind() == reflect.Uint32 || keyType.Kind() == reflect.Uint64:
key, err := strconv.ParseUint(string(data), 10, 64)
if err != nil {
return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from unsigned integer: %w", stringType, err)
}
return reflect.ValueOf(key).Convert(keyType), nil
case keyType.Kind() == reflect.Float32:
key, err := strconv.ParseFloat(string(data), 32)
if err != nil {
return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from float: %w", stringType, err)
}
return reflect.ValueOf(float32(key)), nil
case keyType.Kind() == reflect.Float64:
key, err := strconv.ParseFloat(string(data), 64)
if err != nil {
return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from float: %w", stringType, err)
}
return reflect.ValueOf(float64(key)), nil
}
return reflect.Value{}, fmt.Errorf("toml: cannot convert map key of type %s to expected type %s", stringType, keyType)
}

View file

@ -1,4 +1,4 @@
if ! has nix_direnv_version || ! nix_direnv_version 2.3.0; then
source_url "https://raw.githubusercontent.com/nix-community/nix-direnv/2.3.0/direnvrc" "sha256-Dmd+j63L84wuzgyjITIfSxSD57Tx7v51DMxVZOsiUD8="
if ! has nix_direnv_version || ! nix_direnv_version 3.0.4; then
source_url "https://raw.githubusercontent.com/nix-community/nix-direnv/3.0.4/direnvrc" "sha256-DzlYZ33mWF/Gs8DDeyjr8mnVmQGx7ASYqA5WlxwvBG4="
fi
use flake . --impure

View file

@ -27,7 +27,7 @@ type Finder struct {
// It provides the capability to search for entries with depth,
// meaning it can target deeper locations within the directory structure.
//
// It also supports glob syntax (as defined by [filepat.Match]), offering greater flexibility in search patterns.
// It also supports glob syntax (as defined by [filepath.Match]), offering greater flexibility in search patterns.
//
// Examples:
// - config.yaml
@ -63,7 +63,7 @@ func (f Finder) Find(fsys afero.Fs) ([]string, error) {
// pool.Go(func() ([]string, error) {
// // If the name contains any glob character, perform a glob match
// if strings.ContainsAny(searchName, "*?[]\\^") {
// if strings.ContainsAny(searchName, globMatch) {
// return globWalkSearch(fsys, searchPath, searchName, f.Type)
// }
//
@ -79,7 +79,7 @@ func (f Finder) Find(fsys afero.Fs) ([]string, error) {
allResults, err := iter.MapErr(searchItems, func(item *searchItem) ([]string, error) {
// If the name contains any glob character, perform a glob match
if strings.ContainsAny(item.name, "*?[]\\^") {
if strings.ContainsAny(item.name, globMatch) {
return globWalkSearch(fsys, item.path, item.name, f.Type)
}

View file

@ -1,22 +1,84 @@
{
"nodes": {
"devenv": {
"cachix": {
"inputs": {
"flake-compat": "flake-compat",
"nix": "nix",
"nixpkgs": "nixpkgs",
"pre-commit-hooks": "pre-commit-hooks"
"devenv": "devenv_2",
"flake-compat": [
"devenv",
"flake-compat"
],
"nixpkgs": [
"devenv",
"nixpkgs"
],
"pre-commit-hooks": [
"devenv",
"pre-commit-hooks"
]
},
"locked": {
"lastModified": 1694097209,
"narHash": "sha256-gQmBjjxeSyySjbh0yQVBKApo2KWIFqqbRUvG+Fa+QpM=",
"lastModified": 1712055811,
"narHash": "sha256-7FcfMm5A/f02yyzuavJe06zLa9hcMHsagE28ADcmQvk=",
"owner": "cachix",
"repo": "devenv",
"rev": "7a8e6a91510efe89d8dcb8e43233f93e86f6b189",
"repo": "cachix",
"rev": "02e38da89851ec7fec3356a5c04bc8349cae0e30",
"type": "github"
},
"original": {
"owner": "cachix",
"repo": "cachix",
"type": "github"
}
},
"devenv": {
"inputs": {
"cachix": "cachix",
"flake-compat": "flake-compat_2",
"nix": "nix_2",
"nixpkgs": "nixpkgs_2",
"pre-commit-hooks": "pre-commit-hooks"
},
"locked": {
"lastModified": 1725907707,
"narHash": "sha256-s3pbtzZmVPHzc86WQjK7MGZMNvvw6hWnFMljEkllAfM=",
"owner": "cachix",
"repo": "devenv",
"rev": "2bbbbc468fc02257265a79652a8350651cca495a",
"type": "github"
},
"original": {
"owner": "cachix",
"repo": "devenv",
"type": "github"
}
},
"devenv_2": {
"inputs": {
"flake-compat": [
"devenv",
"cachix",
"flake-compat"
],
"nix": "nix",
"nixpkgs": "nixpkgs",
"poetry2nix": "poetry2nix",
"pre-commit-hooks": [
"devenv",
"cachix",
"pre-commit-hooks"
]
},
"locked": {
"lastModified": 1708704632,
"narHash": "sha256-w+dOIW60FKMaHI1q5714CSibk99JfYxm0CzTinYWr+Q=",
"owner": "cachix",
"repo": "devenv",
"rev": "2ee4450b0f4b95a1b90f2eb5ffea98b90e48c196",
"type": "github"
},
"original": {
"owner": "cachix",
"ref": "python-rewrite",
"repo": "devenv",
"type": "github"
}
@ -37,16 +99,32 @@
"type": "github"
}
},
"flake-compat_2": {
"flake": false,
"locked": {
"lastModified": 1696426674,
"narHash": "sha256-kvjfFW7WAETZlt09AgDn1MrtKzP7t90Vf7vypd3OL1U=",
"owner": "edolstra",
"repo": "flake-compat",
"rev": "0f9255e01c2351cc7d116c072cb317785dd33b33",
"type": "github"
},
"original": {
"owner": "edolstra",
"repo": "flake-compat",
"type": "github"
}
},
"flake-parts": {
"inputs": {
"nixpkgs-lib": "nixpkgs-lib"
},
"locked": {
"lastModified": 1693611461,
"narHash": "sha256-aPODl8vAgGQ0ZYFIRisxYG5MOGSkIczvu2Cd8Gb9+1Y=",
"lastModified": 1725234343,
"narHash": "sha256-+ebgonl3NbiKD2UD0x4BszCZQ6sTfL4xioaM49o5B3Y=",
"owner": "hercules-ci",
"repo": "flake-parts",
"rev": "7f53fdb7bdc5bb237da7fefef12d099e4fd611ca",
"rev": "567b938d64d4b4112ee253b9274472dc3a346eb6",
"type": "github"
},
"original": {
@ -60,11 +138,29 @@
"systems": "systems"
},
"locked": {
"lastModified": 1685518550,
"narHash": "sha256-o2d0KcvaXzTrPRIo0kOLV0/QXHhDQ5DTi+OxcjO8xqY=",
"lastModified": 1689068808,
"narHash": "sha256-6ixXo3wt24N/melDWjq70UuHQLxGV8jZvooRanIHXw0=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "a1720a10a6cfe8234c0e93907ffe81be440f4cef",
"rev": "919d646de7be200f3bf08cb76ae1f09402b6f9b4",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
},
"flake-utils_2": {
"inputs": {
"systems": "systems_2"
},
"locked": {
"lastModified": 1710146030,
"narHash": "sha256-SZ5L6eA7HJ/nmkzGG7/ISclqe6oZdOZTNoesiInkXPQ=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "b1d9ab70662946ef0850d488da1c9019f3a9752a",
"type": "github"
},
"original": {
@ -82,11 +178,11 @@
]
},
"locked": {
"lastModified": 1660459072,
"narHash": "sha256-8DFJjXG8zqoONA1vXtgeKXy68KdJL5UaXR8NtVMUbx8=",
"lastModified": 1709087332,
"narHash": "sha256-HG2cCnktfHsKV0s4XW83gU3F57gaTljL9KNSuG6bnQs=",
"owner": "hercules-ci",
"repo": "gitignore.nix",
"rev": "a20de23b925fd8264fd7fad6454652e142fd7f73",
"rev": "637db329424fd7e46cf4185293b9cc8c88c95394",
"type": "github"
},
"original": {
@ -95,53 +191,90 @@
"type": "github"
}
},
"lowdown-src": {
"flake": false,
"locked": {
"lastModified": 1633514407,
"narHash": "sha256-Dw32tiMjdK9t3ETl5fzGrutQTzh2rufgZV4A/BbxuD4=",
"owner": "kristapsdz",
"repo": "lowdown",
"rev": "d2c2b44ff6c27b936ec27358a2653caaef8f73b8",
"type": "github"
},
"original": {
"owner": "kristapsdz",
"repo": "lowdown",
"type": "github"
}
},
"nix": {
"inputs": {
"lowdown-src": "lowdown-src",
"flake-compat": "flake-compat",
"nixpkgs": [
"devenv",
"cachix",
"devenv",
"nixpkgs"
],
"nixpkgs-regression": "nixpkgs-regression"
},
"locked": {
"lastModified": 1676545802,
"narHash": "sha256-EK4rZ+Hd5hsvXnzSzk2ikhStJnD63odF7SzsQ8CuSPU=",
"lastModified": 1712911606,
"narHash": "sha256-BGvBhepCufsjcUkXnEEXhEVjwdJAwPglCC2+bInc794=",
"owner": "domenkozar",
"repo": "nix",
"rev": "7c91803598ffbcfe4a55c44ac6d49b2cf07a527f",
"rev": "b24a9318ea3f3600c1e24b4a00691ee912d4de12",
"type": "github"
},
"original": {
"owner": "domenkozar",
"ref": "relaxed-flakes",
"ref": "devenv-2.21",
"repo": "nix",
"type": "github"
}
},
"nix-github-actions": {
"inputs": {
"nixpkgs": [
"devenv",
"cachix",
"devenv",
"poetry2nix",
"nixpkgs"
]
},
"locked": {
"lastModified": 1688870561,
"narHash": "sha256-4UYkifnPEw1nAzqqPOTL2MvWtm3sNGw1UTYTalkTcGY=",
"owner": "nix-community",
"repo": "nix-github-actions",
"rev": "165b1650b753316aa7f1787f3005a8d2da0f5301",
"type": "github"
},
"original": {
"owner": "nix-community",
"repo": "nix-github-actions",
"type": "github"
}
},
"nix_2": {
"inputs": {
"flake-compat": [
"devenv",
"flake-compat"
],
"nixpkgs": [
"devenv",
"nixpkgs"
],
"nixpkgs-regression": "nixpkgs-regression_2"
},
"locked": {
"lastModified": 1712911606,
"narHash": "sha256-BGvBhepCufsjcUkXnEEXhEVjwdJAwPglCC2+bInc794=",
"owner": "domenkozar",
"repo": "nix",
"rev": "b24a9318ea3f3600c1e24b4a00691ee912d4de12",
"type": "github"
},
"original": {
"owner": "domenkozar",
"ref": "devenv-2.21",
"repo": "nix",
"type": "github"
}
},
"nixpkgs": {
"locked": {
"lastModified": 1678875422,
"narHash": "sha256-T3o6NcQPwXjxJMn2shz86Chch4ljXgZn746c2caGxd8=",
"lastModified": 1692808169,
"narHash": "sha256-x9Opq06rIiwdwGeK2Ykj69dNc2IvUH1fY55Wm7atwrE=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "126f49a01de5b7e35a43fd43f891ecf6d3a51459",
"rev": "9201b5ff357e781bf014d0330d18555695df7ba8",
"type": "github"
},
"original": {
@ -153,20 +286,14 @@
},
"nixpkgs-lib": {
"locked": {
"dir": "lib",
"lastModified": 1693471703,
"narHash": "sha256-0l03ZBL8P1P6z8MaSDS/MvuU8E75rVxe5eE1N6gxeTo=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "3e52e76b70d5508f3cec70b882a29199f4d1ee85",
"type": "github"
"lastModified": 1725233747,
"narHash": "sha256-Ss8QWLXdr2JCBPcYChJhz4xJm+h/xjl4G0c0XlP6a74=",
"type": "tarball",
"url": "https://github.com/NixOS/nixpkgs/archive/356624c12086a18f2ea2825fed34523d60ccc4e3.tar.gz"
},
"original": {
"dir": "lib",
"owner": "NixOS",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
"type": "tarball",
"url": "https://github.com/NixOS/nixpkgs/archive/356624c12086a18f2ea2825fed34523d60ccc4e3.tar.gz"
}
},
"nixpkgs-regression": {
@ -185,29 +312,61 @@
"type": "github"
}
},
"nixpkgs-stable": {
"nixpkgs-regression_2": {
"locked": {
"lastModified": 1685801374,
"narHash": "sha256-otaSUoFEMM+LjBI1XL/xGB5ao6IwnZOXc47qhIgJe8U=",
"lastModified": 1643052045,
"narHash": "sha256-uGJ0VXIhWKGXxkeNnq4TvV3CIOkUJ3PAoLZ3HMzNVMw=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "c37ca420157f4abc31e26f436c1145f8951ff373",
"rev": "215d4d0fd80ca5163643b03a33fde804a29cc1e2",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-23.05",
"repo": "nixpkgs",
"rev": "215d4d0fd80ca5163643b03a33fde804a29cc1e2",
"type": "github"
}
},
"nixpkgs-stable": {
"locked": {
"lastModified": 1710695816,
"narHash": "sha256-3Eh7fhEID17pv9ZxrPwCLfqXnYP006RKzSs0JptsN84=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "614b4613980a522ba49f0d194531beddbb7220d3",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-23.11",
"repo": "nixpkgs",
"type": "github"
}
},
"nixpkgs_2": {
"locked": {
"lastModified": 1694343207,
"narHash": "sha256-jWi7OwFxU5Owi4k2JmiL1sa/OuBCQtpaAesuj5LXC8w=",
"lastModified": 1713361204,
"narHash": "sha256-TA6EDunWTkc5FvDCqU3W2T3SFn0gRZqh6D/hJnM02MM=",
"owner": "cachix",
"repo": "devenv-nixpkgs",
"rev": "285676e87ad9f0ca23d8714a6ab61e7e027020c6",
"type": "github"
},
"original": {
"owner": "cachix",
"ref": "rolling",
"repo": "devenv-nixpkgs",
"type": "github"
}
},
"nixpkgs_3": {
"locked": {
"lastModified": 1725910328,
"narHash": "sha256-n9pCtzGZ0httmTwMuEbi5E78UQ4ZbQMr1pzi5N0LAG8=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "78058d810644f5ed276804ce7ea9e82d92bee293",
"rev": "5775c2583f1801df7b790bf7f7d710a19bac66f4",
"type": "github"
},
"original": {
@ -217,13 +376,38 @@
"type": "github"
}
},
"poetry2nix": {
"inputs": {
"flake-utils": "flake-utils",
"nix-github-actions": "nix-github-actions",
"nixpkgs": [
"devenv",
"cachix",
"devenv",
"nixpkgs"
]
},
"locked": {
"lastModified": 1692876271,
"narHash": "sha256-IXfZEkI0Mal5y1jr6IRWMqK8GW2/f28xJenZIPQqkY0=",
"owner": "nix-community",
"repo": "poetry2nix",
"rev": "d5006be9c2c2417dafb2e2e5034d83fabd207ee3",
"type": "github"
},
"original": {
"owner": "nix-community",
"repo": "poetry2nix",
"type": "github"
}
},
"pre-commit-hooks": {
"inputs": {
"flake-compat": [
"devenv",
"flake-compat"
],
"flake-utils": "flake-utils",
"flake-utils": "flake-utils_2",
"gitignore": "gitignore",
"nixpkgs": [
"devenv",
@ -232,11 +416,11 @@
"nixpkgs-stable": "nixpkgs-stable"
},
"locked": {
"lastModified": 1688056373,
"narHash": "sha256-2+SDlNRTKsgo3LBRiMUcoEUb6sDViRNQhzJquZ4koOI=",
"lastModified": 1713775815,
"narHash": "sha256-Wu9cdYTnGQQwtT20QQMg7jzkANKQjwBD9iccfGKkfls=",
"owner": "cachix",
"repo": "pre-commit-hooks.nix",
"rev": "5843cf069272d92b60c3ed9e55b7a8989c01d4c7",
"rev": "2ac4dcbf55ed43f3be0bae15e181f08a57af24a4",
"type": "github"
},
"original": {
@ -249,7 +433,7 @@
"inputs": {
"devenv": "devenv",
"flake-parts": "flake-parts",
"nixpkgs": "nixpkgs_2"
"nixpkgs": "nixpkgs_3"
}
},
"systems": {
@ -266,6 +450,21 @@
"repo": "default",
"type": "github"
}
},
"systems_2": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
}
},
"root": "root",

View file

@ -20,6 +20,7 @@
default = {
languages = {
go.enable = true;
go.package = pkgs.lib.mkDefault pkgs.go_1_23;
};
packages = with pkgs; [
@ -34,11 +35,27 @@
ci = devenv.shells.default;
ci_1_20 = {
ci_1_21 = {
imports = [ devenv.shells.ci ];
languages = {
go.package = pkgs.go_1_20;
go.package = pkgs.go_1_21;
};
};
ci_1_22 = {
imports = [ devenv.shells.ci ];
languages = {
go.package = pkgs.go_1_22;
};
};
ci_1_23 = {
imports = [ devenv.shells.ci ];
languages = {
go.package = pkgs.go_1_23;
};
};
};

5
vendor/github.com/sagikazarmark/locafero/glob.go generated vendored Normal file
View file

@ -0,0 +1,5 @@
//go:build !windows
package locafero
const globMatch = "*?[]\\^"

View file

@ -0,0 +1,8 @@
//go:build windows
package locafero
// See [filepath.Match]:
//
// On Windows, escaping is disabled. Instead, '\\' is treated as path separator.
const globMatch = "*?[]^"

View file

@ -1,12 +1,12 @@
root = true
[*]
charset = utf-8
end_of_line = lf
indent_size = 4
indent_style = space
insert_final_newline = true
trim_trailing_whitespace = true
[*.go]
indent_style = tab
indent_size = 4
insert_final_newline = true
[*.{yml,yaml}]
indent_style = space
indent_size = 2
insert_final_newline = true
trim_trailing_whitespace = true

18
vendor/github.com/spf13/afero/.golangci.yaml generated vendored Normal file
View file

@ -0,0 +1,18 @@
linters-settings:
gci:
sections:
- standard
- default
- prefix(github.com/spf13/afero)
linters:
disable-all: true
enable:
- gci
- gofmt
- gofumpt
- staticcheck
issues:
exclude-dirs:
- gcsfs/internal/stiface

View file

@ -12,7 +12,7 @@ types and methods. Afero has an exceptionally clean interface and simple design
without needless constructors or initialization methods.
Afero is also a library providing a base set of interoperable backend
filesystems that make it easy to work with afero while retaining all the power
filesystems that make it easy to work with, while retaining all the power
and benefit of the os and ioutil packages.
Afero provides significant improvements over using the os package alone, most

View file

@ -255,7 +255,6 @@ func (f fromIOFSFile) Readdir(count int) ([]os.FileInfo, error) {
ret := make([]os.FileInfo, len(entries))
for i := range entries {
ret[i], err = entries[i].Info()
if err != nil {
return nil, err
}

View file

@ -16,11 +16,9 @@ package afero
import (
"fmt"
"io"
"log"
"os"
"path/filepath"
"sort"
"strings"
"sync"

View file

@ -1,6 +1,6 @@
# cast
[![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/spf13/cast/ci.yaml?branch=master&style=flat-square)](https://github.com/spf13/cast/actions/workflows/ci.yaml)
[![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/spf13/cast/test.yaml?branch=master&style=flat-square)](https://github.com/spf13/cast/actions/workflows/test.yaml)
[![PkgGoDev](https://pkg.go.dev/badge/mod/github.com/spf13/cast)](https://pkg.go.dev/mod/github.com/spf13/cast)
![Go Version](https://img.shields.io/badge/go%20version-%3E=1.16-61CFDD.svg?style=flat-square)
[![Go Report Card](https://goreportcard.com/badge/github.com/spf13/cast?style=flat-square)](https://goreportcard.com/report/github.com/spf13/cast)

View file

@ -18,6 +18,14 @@ import (
var errNegativeNotAllowed = errors.New("unable to cast negative value")
type float64EProvider interface {
Float64() (float64, error)
}
type float64Provider interface {
Float64() float64
}
// ToTimeE casts an interface to a time.Time type.
func ToTimeE(i interface{}) (tim time.Time, err error) {
return ToTimeInDefaultLocationE(i, time.UTC)
@ -77,11 +85,14 @@ func ToDurationE(i interface{}) (d time.Duration, err error) {
d, err = time.ParseDuration(s + "ns")
}
return
case json.Number:
case float64EProvider:
var v float64
v, err = s.Float64()
d = time.Duration(v)
return
case float64Provider:
d = time.Duration(s.Float64())
return
default:
err = fmt.Errorf("unable to cast %#v of type %T to Duration", i, i)
return
@ -174,12 +185,14 @@ func ToFloat64E(i interface{}) (float64, error) {
return v, nil
}
return 0, fmt.Errorf("unable to cast %#v of type %T to float64", i, i)
case json.Number:
case float64EProvider:
v, err := s.Float64()
if err == nil {
return v, nil
}
return 0, fmt.Errorf("unable to cast %#v of type %T to float64", i, i)
case float64Provider:
return s.Float64(), nil
case bool:
if s {
return 1, nil
@ -230,12 +243,14 @@ func ToFloat32E(i interface{}) (float32, error) {
return float32(v), nil
}
return 0, fmt.Errorf("unable to cast %#v of type %T to float32", i, i)
case json.Number:
case float64EProvider:
v, err := s.Float64()
if err == nil {
return float32(v), nil
}
return 0, fmt.Errorf("unable to cast %#v of type %T to float32", i, i)
case float64Provider:
return float32(s.Float64()), nil
case bool:
if s {
return 1, nil
@ -598,12 +613,12 @@ func ToUint64E(i interface{}) (uint64, error) {
switch s := i.(type) {
case string:
v, err := strconv.ParseInt(trimZeroDecimal(s), 0, 0)
v, err := strconv.ParseUint(trimZeroDecimal(s), 0, 0)
if err == nil {
if v < 0 {
return 0, errNegativeNotAllowed
}
return uint64(v), nil
return v, nil
}
return 0, fmt.Errorf("unable to cast %#v of type %T to uint64", i, i)
case json.Number:
@ -917,8 +932,8 @@ func indirectToStringerOrError(a interface{}) interface{} {
return nil
}
var errorType = reflect.TypeOf((*error)(nil)).Elem()
var fmtStringerType = reflect.TypeOf((*fmt.Stringer)(nil)).Elem()
errorType := reflect.TypeOf((*error)(nil)).Elem()
fmtStringerType := reflect.TypeOf((*fmt.Stringer)(nil)).Elem()
v := reflect.ValueOf(a)
for !v.Type().Implements(fmtStringerType) && !v.Type().Implements(errorType) && v.Kind() == reflect.Ptr && !v.IsNil() {
@ -987,7 +1002,7 @@ func ToStringE(i interface{}) (string, error) {
// ToStringMapStringE casts an interface to a map[string]string type.
func ToStringMapStringE(i interface{}) (map[string]string, error) {
var m = map[string]string{}
m := map[string]string{}
switch v := i.(type) {
case map[string]string:
@ -1017,7 +1032,7 @@ func ToStringMapStringE(i interface{}) (map[string]string, error) {
// ToStringMapStringSliceE casts an interface to a map[string][]string type.
func ToStringMapStringSliceE(i interface{}) (map[string][]string, error) {
var m = map[string][]string{}
m := map[string][]string{}
switch v := i.(type) {
case map[string][]string:
@ -1081,7 +1096,7 @@ func ToStringMapStringSliceE(i interface{}) (map[string][]string, error) {
// ToStringMapBoolE casts an interface to a map[string]bool type.
func ToStringMapBoolE(i interface{}) (map[string]bool, error) {
var m = map[string]bool{}
m := map[string]bool{}
switch v := i.(type) {
case map[interface{}]interface{}:
@ -1106,7 +1121,7 @@ func ToStringMapBoolE(i interface{}) (map[string]bool, error) {
// ToStringMapE casts an interface to a map[string]interface{} type.
func ToStringMapE(i interface{}) (map[string]interface{}, error) {
var m = map[string]interface{}{}
m := map[string]interface{}{}
switch v := i.(type) {
case map[interface{}]interface{}:
@ -1126,7 +1141,7 @@ func ToStringMapE(i interface{}) (map[string]interface{}, error) {
// ToStringMapIntE casts an interface to a map[string]int{} type.
func ToStringMapIntE(i interface{}) (map[string]int, error) {
var m = map[string]int{}
m := map[string]int{}
if i == nil {
return m, fmt.Errorf("unable to cast %#v of type %T to map[string]int", i, i)
}
@ -1167,7 +1182,7 @@ func ToStringMapIntE(i interface{}) (map[string]int, error) {
// ToStringMapInt64E casts an interface to a map[string]int64{} type.
func ToStringMapInt64E(i interface{}) (map[string]int64, error) {
var m = map[string]int64{}
m := map[string]int64{}
if i == nil {
return m, fmt.Errorf("unable to cast %#v of type %T to map[string]int64", i, i)
}
@ -1404,38 +1419,35 @@ func (f timeFormat) hasTimezone() bool {
return f.typ >= timeFormatNumericTimezone && f.typ <= timeFormatNumericAndNamedTimezone
}
var (
timeFormats = []timeFormat{
// Keep common formats at the top.
{"2006-01-02", timeFormatNoTimezone},
{time.RFC3339, timeFormatNumericTimezone},
{"2006-01-02T15:04:05", timeFormatNoTimezone}, // iso8601 without timezone
{time.RFC1123Z, timeFormatNumericTimezone},
{time.RFC1123, timeFormatNamedTimezone},
{time.RFC822Z, timeFormatNumericTimezone},
{time.RFC822, timeFormatNamedTimezone},
{time.RFC850, timeFormatNamedTimezone},
{"2006-01-02 15:04:05.999999999 -0700 MST", timeFormatNumericAndNamedTimezone}, // Time.String()
{"2006-01-02T15:04:05-0700", timeFormatNumericTimezone}, // RFC3339 without timezone hh:mm colon
{"2006-01-02 15:04:05Z0700", timeFormatNumericTimezone}, // RFC3339 without T or timezone hh:mm colon
{"2006-01-02 15:04:05", timeFormatNoTimezone},
{time.ANSIC, timeFormatNoTimezone},
{time.UnixDate, timeFormatNamedTimezone},
{time.RubyDate, timeFormatNumericTimezone},
{"2006-01-02 15:04:05Z07:00", timeFormatNumericTimezone},
{"02 Jan 2006", timeFormatNoTimezone},
{"2006-01-02 15:04:05 -07:00", timeFormatNumericTimezone},
{"2006-01-02 15:04:05 -0700", timeFormatNumericTimezone},
{time.Kitchen, timeFormatTimeOnly},
{time.Stamp, timeFormatTimeOnly},
{time.StampMilli, timeFormatTimeOnly},
{time.StampMicro, timeFormatTimeOnly},
{time.StampNano, timeFormatTimeOnly},
}
)
var timeFormats = []timeFormat{
// Keep common formats at the top.
{"2006-01-02", timeFormatNoTimezone},
{time.RFC3339, timeFormatNumericTimezone},
{"2006-01-02T15:04:05", timeFormatNoTimezone}, // iso8601 without timezone
{time.RFC1123Z, timeFormatNumericTimezone},
{time.RFC1123, timeFormatNamedTimezone},
{time.RFC822Z, timeFormatNumericTimezone},
{time.RFC822, timeFormatNamedTimezone},
{time.RFC850, timeFormatNamedTimezone},
{"2006-01-02 15:04:05.999999999 -0700 MST", timeFormatNumericAndNamedTimezone}, // Time.String()
{"2006-01-02T15:04:05-0700", timeFormatNumericTimezone}, // RFC3339 without timezone hh:mm colon
{"2006-01-02 15:04:05Z0700", timeFormatNumericTimezone}, // RFC3339 without T or timezone hh:mm colon
{"2006-01-02 15:04:05", timeFormatNoTimezone},
{time.ANSIC, timeFormatNoTimezone},
{time.UnixDate, timeFormatNamedTimezone},
{time.RubyDate, timeFormatNumericTimezone},
{"2006-01-02 15:04:05Z07:00", timeFormatNumericTimezone},
{"02 Jan 2006", timeFormatNoTimezone},
{"2006-01-02 15:04:05 -07:00", timeFormatNumericTimezone},
{"2006-01-02 15:04:05 -0700", timeFormatNumericTimezone},
{time.Kitchen, timeFormatTimeOnly},
{time.Stamp, timeFormatTimeOnly},
{time.StampMilli, timeFormatTimeOnly},
{time.StampMicro, timeFormatTimeOnly},
{time.StampNano, timeFormatTimeOnly},
}
func parseDateWith(s string, location *time.Location, formats []timeFormat) (d time.Time, e error) {
for _, format := range formats {
if d, e = time.Parse(format.format, s); e == nil {