Skip to content

Latest commit

 

History

History
1197 lines (791 loc) · 29.5 KB

command_reference.md

File metadata and controls

1197 lines (791 loc) · 29.5 KB
sidebar_label sidebar_position slug
Command Reference
1
/command_reference

import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';

Command Reference

There are many commands to help you manage your file system. This page provides a detailed reference for these commands.

Overview

If you run juicefs by itself, it will print all available commands. In addition, you can add -h/--help flag after each command to get more information of it, e.g., juicefs format -h.

NAME:
   juicefs - A POSIX file system built on Redis and object storage.

USAGE:
   juicefs [global options] command [command options] [arguments...]

VERSION:
   1.0.0+2022-08-01.0e7afe2d

COMMANDS:
   ADMIN:
     format   Format a volume
     config   Change configuration of a volume
     destroy  Destroy an existing volume
     gc       Garbage collector of objects in data storage
     fsck     Check consistency of a volume
     dump     Dump metadata into a JSON file
     load     Load metadata from a previously dumped JSON file
     version  Show version
   INSPECTOR:
     status   Show status of a volume
     stats    Show real time performance statistics of JuiceFS
     profile  Show profiling of operations completed in JuiceFS
     info     Show internal information of a path or inode
   SERVICE:
     mount    Mount a volume
     umount   Unmount a volume
     gateway  Start an S3-compatible gateway
     webdav   Start a WebDAV server
   TOOL:
     bench     Run benchmarks on a path
     objbench  Run benchmarks on an object storage
     warmup    Build cache for target directories/files
     rmr       Remove directories recursively
     sync      Sync between two storages

GLOBAL OPTIONS:
   --verbose, --debug, -v  enable debug log (default: false)
   --quiet, -q             show warning and errors only (default: false)
   --trace                 enable trace log (default: false)
   --no-agent              disable pprof (:6060) and gops (:6070) agent (default: false)
   --pyroscope value       pyroscope address
   --no-color              disable colors (default: false)
   --help, -h              show help (default: false)
   --version, -V           print version only (default: false)

COPYRIGHT:
   Apache License 2.0

:::note If juicefs is not placed in your $PATH, you should run the script with the path to the script. For example, if juicefs is in current directory, you should use ./juicefs. It is recommended to place juicefs in your $PATH for convenience. You can refer to "Installation" for more information. :::

:::note If the command option is of boolean type, such as --debug, there is no need to set any value, just add --debug to the command to enable the function; this function is disabled if --debug is not added. :::

Auto Completion

:::note This feature requires JuiceFS >= 0.15.2. It is implemented based on github.com/urfave/cli/v2. You can find more information here. :::

To enable commands completion, simply source the script provided within hack/autocomplete directory. For example:

source hack/autocomplete/bash_autocomplete
source hack/autocomplete/zsh_autocomplete

Please note the auto-completion is only enabled for the current session. If you want to apply it for all new sessions, add the source command to .bashrc or .zshrc:

echo "source path/to/bash_autocomplete" >> ~/.bashrc
echo "source path/to/zsh_autocomplete" >> ~/.zshrc

Alternatively, if you are using bash on a Linux system, you may just copy the script to /etc/bash_completion.d and rename it to juicefs:

sudo cp hack/autocomplete/bash_autocomplete /etc/bash_completion.d/juicefs
source /etc/bash_completion.d/juicefs

Commands

juicefs format

Description

Format a volume. It's the first step for initializing a new file system volume.

Synopsis

juicefs format [command options] META-URL NAME

Options

--block-size value
size of block in KiB (default: 4096)

--capacity value
the limit for space in GiB (0 means unlimited) (default: 0)

--inodes value
the limit for number of inodes (0 means unlimited) (default: 0)

--compress value
compression algorithm (lz4, zstd, none) (default: "none")

--shards value
store the blocks into N buckets by hash of key (default: 0)

--storage value
Object storage type (e.g. s3, gcs, oss, cos) (default: "file", please refer to documentation for all supported object storage types)

--bucket value
A bucket URL to store data (default: "$HOME/.juicefs/local" or "/var/jfs")

--access-key value
Access Key for object storage (can also be set via the environment variable ACCESS_KEY)

--secret-key value
Secret Key for object storage (can also be set via the environment variable SECRET_KEY)

--session-token value
session token for object storage

--encrypt-rsa-key value
A path to RSA private key (PEM)

--trash-days value
number of days after which removed files will be permanently deleted (default: 1)

--force
overwrite existing format (default: false)

--no-update
don't update existing volume (default: false)

Examples

# Create a simple test volume (data will be stored in a local directory)
$ juicefs format sqlite3://myjfs.db myjfs

# Create a volume with Redis and S3
$ juicefs format redis://localhost myjfs --storage s3 --bucket https://mybucket.s3.us-east-2.amazonaws.com

# Create a volume with password protected MySQL
$ juicefs format mysql://jfs:mypassword@(127.0.0.1:3306)/juicefs myjfs
# A safer alternative
$ META_PASSWORD=mypassword juicefs format mysql://jfs:@(127.0.0.1:3306)/juicefs myjfs

# Create a volume with "quota" enabled
$ juicefs format sqlite3://myjfs.db myjfs --inode 1000000 --capacity 102400

# Create a volume with "trash" disabled
$ juicefs format sqlite3://myjfs.db myjfs --trash-days 0

juicefs mount

Description

Mount a volume. The volume shoud be formatted first.

Synopsis

juicefs mount [command options] META-URL MOUNTPOINT

Options

--metrics value
address to export metrics (default: "127.0.0.1:9567")

--consul value
consul address to register (default: "127.0.0.1:8500")

--no-usage-report
do not send usage report (default: false)

-d, --background
run in background (default: false)

--no-syslog
disable syslog (default: false)

--log value
path of log file when running in background (default: $HOME/.juicefs/juicefs.log or /var/log/juicefs.log)

-o value
other FUSE options (see this document for more information)

--attr-cache value
attributes cache timeout in seconds (default: 1)

--entry-cache value
file entry cache timeout in seconds (default: 1)

--dir-entry-cache value
dir entry cache timeout in seconds (default: 1)

--enable-xattr
enable extended attributes (xattr) (default: false)

--bucket value
customized endpoint to access object storage

--get-timeout value
the max number of seconds to download an object (default: 60)

--put-timeout value
the max number of seconds to upload an object (default: 60)

--io-retries value
number of retries after network failure (default: 10)

--max-uploads value
number of connections to upload (default: 20)

--max-deletes value
number of threads to delete objects (default: 2)

--buffer-size value
total read/write buffering in MiB (default: 300)

--upload-limit value
bandwidth limit for upload in Mbps (default: 0)

--download-limit value
bandwidth limit for download in Mbps (default: 0)

--prefetch value
prefetch N blocks in parallel (default: 1)

--writeback
upload objects in background (default: false)

--cache-dir value
directory paths of local cache, use : (Linux, macOS) or ; (Windows) to separate multiple paths (default: "$HOME/.juicefs/cache" or "/var/jfsCache")

--cache-size value
size of cached objects in MiB (default: 102400)

--free-space-ratio value
min free space (ratio) (default: 0.1)

--cache-partial-only
cache random/small read only (default: false)

--read-only
allow lookup/read operations only (default: false)

--open-cache value
open file cache timeout in seconds (0 means disable this feature) (default: 0)

--subdir value
mount a sub-directory as root (default: "")

--backup-meta value
interval (in seconds) to automatically backup metadata in the object storage (0 means disable backup) (default: "3600")

--heartbeat value
interval (in seconds) to send heartbeat; it's recommended that all clients use the same heartbeat value (default: "12")

--upload-delay value
delayed duration for uploading objects ("s", "m", "h") (default: 0s)

--no-bgjob
disable background jobs (clean-up, backup, etc.) (default: false)

Examples

# Mount in foreground
$ juicefs mount redis://localhost /mnt/jfs

# Mount in background with password protected Redis
$ juicefs mount redis://:mypassword@localhost /mnt/jfs -d
# A safer alternative
$ META_PASSWORD=mypassword juicefs mount redis://localhost /mnt/jfs -d

# Mount with a sub-directory as root
$ juicefs mount redis://localhost /mnt/jfs --subdir /dir/in/jfs

# Enable "writeback" mode, which improves performance at the risk of losing objects
$ juicefs mount redis://localhost /mnt/jfs -d --writeback

# Enable "read-only" mode
$ juicefs mount redis://localhost /mnt/jfs -d --read-only

# Disable metadata backup
$ juicefs mount redis://localhost /mnt/jfs --backup-meta 0

juicefs umount

Description

Unmount a volume.

Synopsis

juicefs umount [command options] MOUNTPOINT

Options

-f, --force
force unmount a busy mount point (default: false)

Examples

$ juicefs umount /mnt/jfs

juicefs gateway

Description

Start an S3-compatible gateway.

Synopsis

juicefs gateway [command options] META-URL ADDRESS

Options

--bucket value
customized endpoint to access an object storage

--get-timeout value
the max number of seconds to download an object (default: 60)

--put-timeout value
the max number of seconds to upload an object (default: 60)

--io-retries value
number of retries after network failure (default: 10)

--max-uploads value
number of connections to upload (default: 20)

--max-deletes value
number of threads to delete objects (default: 2)

--buffer-size value
total read/write buffering in MiB (default: 300)

--upload-limit value
bandwidth limit for upload in Mbps (default: 0)

--download-limit value
bandwidth limit for download in Mbps (default: 0)

--prefetch value
prefetch N blocks in parallel (default: 1)

--writeback
upload objects in background (default: false)

--cache-dir value
directory paths of local cache, use : (Linux, macOS) or ; (Windows) to separate multiple paths (default: "$HOME/.juicefs/cache" or /var/jfsCache)

--cache-size value
size of cached objects in MiB (default: 102400)

--free-space-ratio value
min free space (ratio) (default: 0.1)

--cache-partial-only
cache random/small read only (default: false)

--read-only
allow lookup/read operations only (default: false)

--open-cache value
open file cache timeout in seconds (0 means disable this feature) (default: 0)

--subdir value
mount a sub-directory as root (default: "")

--attr-cache value
attributes cache timeout in seconds (default: 1)

--entry-cache value
file entry cache timeout in seconds (default: 0)

--dir-entry-cache value
dir entry cache timeout in seconds (default: 1)

--access-log value
path for JuiceFS access log

--metrics value
address to export metrics (default: "127.0.0.1:9567")

--no-usage-report
do not send usage report (default: false)

--no-banner
disable MinIO startup information (default: false)

--multi-buckets
use top level of directories as buckets (default: false)

--keep-etag
save the ETag for uploaded objects (default: false)

--storage value
Object storage type (e.g. s3, gcs, oss, cos) (default: "file", please refer to documentation for all supported object storage types)

--upload-delay value
delayed duration (in seconds) for uploading objects (default: "0")

--backup-meta value
interval (in seconds) to automatically backup metadata in the object storage (0 means disable backup) (default: "3600")

--heartbeat value
interval (in seconds) to send heartbeat; it's recommended that all clients use the same heartbeat value (default: "12")

--no-bgjob
disable background jobs (clean-up, backup, etc.) (default: false)

--umask value
umask for new file in octal (default: "022")

--consul value
consul address to register (default: "127.0.0.1:8500")

Examples

$ export MINIO_ROOT_USER=admin
$ export MINIO_ROOT_PASSWORD=12345678
$ juicefs gateway redis://localhost localhost:9000

juicefs webdav

Description

Start a WebDAV server.

Synopsis

juicefs webdav [command options] META-URL ADDRESS
  • META-URL: Database URL for metadata storage, see "JuiceFS supported metadata engines" for details.
  • ADDRESS: WebDAV address and listening port, for example: localhost:9007

Options

--bucket value
customized endpoint to access an object storage

--get-timeout value
the max number of seconds to download an object (default: 60)

--put-timeout value
the max number of seconds to upload an object (default: 60)

--io-retries value
number of retries after network failure (default: 10)

--max-uploads value
number of connections to upload (default: 20)

--max-deletes value
number of threads to delete objects (default: 2)

--buffer-size value
total read/write buffering in MiB (default: 300)

--upload-limit value
bandwidth limit for upload in Mbps (default: 0)

--download-limit value
bandwidth limit for download in Mbps (default: 0)

--prefetch value
prefetch N blocks in parallel (default: 1)

--writeback
upload objects in background (default: false)

--upload-delay value
delayed duration for uploading objects ("s", "m", "h") (default: 0s)

--cache-dir value
directory paths of local cache, use : (Linux, macOS) or ; (Windows) to separate multiple paths (default: "$HOME/.juicefs/cache" or /var/jfsCache)

--cache-size value
size of cached objects in MiB (default: 102400)

--free-space-ratio value
min free space (ratio) (default: 0.1)

--cache-partial-only
cache random/small read only (default: false)

--read-only
allow lookup/read operations only (default: false)

--backup-meta
interval to automatically backup metadata in the object storage (0 means disable backup) (default: 1h0m0s)

--no-bgjob
disable background jobs (clean-up, backup, etc.) (default: false)

--open-cache value
open file cache timeout in seconds (0 means disable this feature) (default: 0)

--subdir value
mount a sub-directory as root (default: "")

--attr-cache value
attributes cache timeout in seconds (default: 1)

--entry-cache value
file entry cache timeout in seconds (default: 0)

--dir-entry-cache value
dir entry cache timeout in seconds (default: 1)

--gzip
compress served files via gzip (default: false)

--disallowList
disallow list a directory (default: false)

--access-log value
path for JuiceFS access log

--metrics value
address to export metrics (default: "127.0.0.1:9567")

--consul value
consul address to register (default: "127.0.0.1:8500")

--no-usage-report
do not send usage report (default: false)

--storage value
Object storage type (e.g. s3, gcs, oss, cos) (default: "file", please refer to documentation for all supported object storage types)

--heartbeat value
interval (in seconds) to send heartbeat; it's recommended that all clients use the same heartbeat value (default: "12")

Examples

$ juicefs webdav redis://localhost localhost:9007

juicefs sync

Description

Sync between two storage.

Synopsis

juicefs sync [command options] SRC DST
  • SRC: source path
  • DST: destination path

The format of both source and destination paths is [NAME://][ACCESS_KEY:SECRET_KEY@]BUCKET[.ENDPOINT][/PREFIX], in which:

  • NAME: JuiceFS supported data storage types (e.g. s3, oss) (please refer to this document).
  • ACCESS_KEY and SECRET_KEY: The credential required to access the data storage (please refer to this document).
  • BUCKET[.ENDPOINT]: The access address of the data storage service. The format may be different for different storage types, and please refer to the document.
  • [/PREFIX]: Optional, a prefix for the source and destination paths that can be used to limit synchronization of data only in certain paths.

For a detailed introduction to the sync subcommand, please refer to the documentation.

Options

--start KEY, -s KEY
the first KEY to sync

--end KEY, -e KEY
the last KEY to sync

--threads value, -p value
number of concurrent threads (default: 10)

--http-port PORT
HTTP PORT to listen to (default: 6070)

--update, -u
update existing file if the source is newer (default: false)

--force-update, -f
always update existing file (default: false)

--perms
preserve permissions (default: false)

--dirs
Sync directories or holders (default: false)

--dry
don't copy file (default: false)

--delete-src, --deleteSrc
delete objects from source after synced (default: false)

--delete-dst, --deleteDst
delete extraneous objects from destination (default: false)

--exclude PATTERN
exclude Key matching PATTERN

--include PATTERN
don't exclude Key matching PATTERN, need to be used with --exclude option

--links, -l
copy symlinks as symlinks (default: false)

--limit value
limit the number of objects that will be processed (default: -1)

--manager value
manager address

--worker value
hosts (seperated by comma) to launch worker

--bwlimit value
limit bandwidth in Mbps (0 means unlimited) (default: 0)

--no-https
do not use HTTPS (default: false)

--check-all
verify integrity of all files in source and destination (default: false)

--check-new
verify integrity of newly copied files (default: false)

Examples

# Sync object from OSS to S3
$ juicefs sync oss://mybucket.oss-cn-shanghai.aliyuncs.com s3://mybucket.s3.us-east-2.amazonaws.com

# Sync objects from S3 to JuiceFS
$ juicefs mount -d redis://localhost /mnt/jfs
$ juicefs sync s3://mybucket.s3.us-east-2.amazonaws.com/ /mnt/jfs/

# SRC: a1/b1,a2/b2,aaa/b1   DST: empty   sync result: aaa/b1
$ juicefs sync --exclude='a?/b*' s3://mybucket.s3.us-east-2.amazonaws.com/ /mnt/jfs/

# SRC: a1/b1,a2/b2,aaa/b1   DST: empty   sync result: a1/b1,aaa/b1
$ juicefs sync --include='a1/b1' --exclude='a[1-9]/b*' s3://mybucket.s3.us-east-2.amazonaws.com/ /mnt/jfs/

# SRC: a1/b1,a2/b2,aaa/b1,b1,b2  DST: empty   sync result: a1/b1,b2
$ juicefs sync --include='a1/b1' --exclude='a*' --include='b2' --exclude='b?' s3://mybucket.s3.us-east-2.amazonaws.com/ /mnt/jfs/

juicefs rmr

Description

Remove all files in directories recursively.

Synopsis

juicefs rmr PATH ...

Examples

$ juicefs rmr /mnt/jfs/foo

juicefs info

Description

Show internal information for given paths or inodes.

Synopsis

juicefs info [command options] PATH or INODE

Options

--inode, -i
use inode instead of path (current dir should be inside JuiceFS) (default: false)

--recursive, -r
get summary of directories recursively (NOTE: it may take a long time for huge trees) (default: false)

--raw
show internal raw information (default: false)

Examples

$ Check a path
$ juicefs info /mnt/jfs/foo

# Check an inode
$ cd /mnt/jfs
$ juicefs info -i 100

juicefs bench

Description

Run benchmark, including read/write/stat for big and small files.

Synopsis

juicefs bench [command options] PATH

For a detailed introduction to the bench subcommand, please refer to the documentation.

Options

--block-size value
block size in MiB (default: 1)

--big-file-size value
size of big file in MiB (default: 1024)

--small-file-size value
size of small file in MiB (default: 0.1)

--small-file-count value
number of small files (default: 100)

--threads value, -p value
number of concurrent threads (default: 1)

Examples

# Run benchmarks with 4 threads
$ juicefs bench /mnt/jfs -p 4

# Run benchmarks of only small files
$ juicefs bench /mnt/jfs --big-file-size 0

juicefs objbench

Description

Run basic benchmarks on the target object storage to test if it works as expected.

Synopsis

juicefs objbench [command options] BUCKET

For a detailed introduction to the objbench subcommand, please refer to the documentation.

Options

--storage value
Object storage type (e.g. s3, gcs, oss, cos) (default: "file", please refer to documentation for all supported object storage types)

--access-key value
Access Key for object storage (can also be set via the environment variable ACCESS_KEY)

--secret-key value
Secret Key for object storage (can also be set via the environment variable SECRET_KEY)

--block-size value
size of each IO block in KiB (default: 4096)

--big-object-size value
size of each big object in MiB (default: 1024)

--small-object-size value
size of each small object in KiB (default: 128)

--skip-functional-tests
skip functional tests (default: false)

--threads value, -p value
number of concurrent threads (default: 4)

Examples

# Run benchmarks on S3
$ ACCESS_KEY=myAccessKey SECRET_KEY=mySecretKey juicefs objbench --storage s3  https://mybucket.s3.us-east-2.amazonaws.com -p 6

juicefs gc

Description

Collect leaked objects.

Synopsis

juicefs gc [command options] META-URL

Options

--delete
delete leaked objects (default: false)

--compact
compact all chunks with more than 1 slices (default: false).

--threads value
number of threads to delete leaked objects (default: 10)

Examples

# Check only, no writable change
$ juicefs gc redis://localhost

# Trigger compaction of all slices
$ juicefs gc redis://localhost --compact

# Delete leaked objects
$ juicefs gc redis://localhost --delete

juicefs fsck

Description

Check consistency of file system.

Synopsis

juicefs fsck [command options] META-URL

Examples

$ juicefs fsck redis://localhost

juicefs profile

Description

Analyze access log.

Synopsis

juicefs profile [command options] MOUNTPOINT/LOGFILE

Options

--uid value, -u value
only track specified UIDs (separated by comma ,)

--gid value, -g value
only track specified GIDs(separated by comma ,)

--pid value, -p value
only track specified PIDs(separated by comma ,)

--interval value
flush interval in seconds; set it to 0 when replaying a log file to get an immediate result (default: 2)

Examples

# Monitor real time operations
$ juicefs profile /mnt/jfs

# Replay an access log
$ cat /mnt/jfs/.accesslog > /tmp/jfs.alog
# Press Ctrl-C to stop the "cat" command after some time
$ juicefs profile /tmp/jfs.alog

# Analyze an access log and print the total statistics immediately
$ juicefs profile /tmp/jfs.alog --interval 0

juicefs stats

Description

Show runtime statistics.

Synopsis

juicefs stats [command options] MOUNTPOINT

Options

--schema value
schema string that controls the output sections (u: usage, f: fuse, m: meta, c: blockcache, o: object, g: go) (default: "ufmco")

--interval value
interval in seconds between each update (default: 1)

--verbosity value
verbosity level, 0 or 1 is enough for most cases (default: 0)

Examples

$ juicefs stats /mnt/jfs

# More metrics
$ juicefs stats /mnt/jfs -l 1

juicefs status

Description

Show status of JuiceFS.

Synopsis

juicefs status [command options] META-URL

Options

--session value, -s value
show detailed information (sustained inodes, locks) of the specified session (sid) (default: 0)

Examples

$ juicefs status redis://localhost

juicefs warmup

Description

Build cache for target directories/files.

Synopsis

juicefs warmup [command options] [PATH ...]

Options

--file value, -f value
file containing a list of paths

--threads value, -p value
number of concurrent workers (default: 50)

--background, -b
run in background (default: false)

Examples

# Warm all files in datadir
$ juicefs warmup /mnt/jfs/datadir

# Warm only three files in datadir
$ cat /tmp/filelist
/mnt/jfs/datadir/f1
/mnt/jfs/datadir/f2
/mnt/jfs/datadir/f3
$ juicefs warmup -f /tmp/filelist

juicefs dump

Description

Dump metadata into a JSON file.

Synopsis

juicefs dump [command options] META-URL [FILE]

When the FILE is not provided, STDOUT will be used instead.

Options

--subdir value
only dump a sub-directory.

Examples

$ juicefs dump redis://localhost meta-dump

# Dump only a subtree of the volume
$ juicefs dump redis://localhost sub-meta-dump --subdir /dir/in/jfs

juicefs load

Description

Load metadata from a previously dumped JSON file.

Synopsis

juicefs load [command options] META-URL [FILE]

When the FILE is not provided, STDIN will be used instead.

Examples

$ juicefs load redis://localhost/1 meta-dump

juicefs config

Description

Change config of a volume.

Synopsis

juicefs config [command options] META-URL

Options

--capacity value
limit for space in GiB

--inodes value
limit for number of inodes

--bucket value
a bucket URL to store data

--access-key value
access key for object storage

--secret-key value
secret key for object storage

--session-token value
session token for object storage

--trash-days value
number of days after which removed files will be permanently deleted

--force
skip sanity check and force update the configurations (default: false)

--encrypt-secret
encrypt the secret key if it was previously stored in plain format (default: false)

--min-client-version value
minimum client version allowed to connect

--max-client-version value
maximum client version allowed to connect

Examples

# Show the current configurations
$ juicefs config redis://localhost

# Change volume "quota"
$ juicefs config redis://localhost --inode 10000000 --capacity 1048576

# Change maximum days before files in trash are deleted
$ juicefs config redis://localhost --trash-days 7

# Limit client version that is allowed to connect
$ juicefs config redis://localhost --min-client-version 1.0.0 --max-client-version 1.1.0

juicefs destroy

Description

Destroy an existing volume

Synopsis

juicefs destroy [command options] META-URL UUID

Options

--force
skip sanity check and force destroy the volume (default: false)

Examples

$ juicefs destroy redis://localhost e94d66a8-2339-4abd-b8d8-6812df737892