Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

translate 3 files #13

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions docs/command.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
## Basic build tools

The main build tool is the `ya make` command.

## Command line (format, options)

Build commands are integrated into the universal `ya` utility. The utility offers a wide range of functions and command line parameters that you can use to adapt the build process to different project requirements.
Running `ya` without parameters displays help:
```
Yet another build tool.

Usage: ya [--precise] [--profile] [--error-file ERROR_FILE] [--keep-tmp]
[--no-logs] [--no-report] [--no-tmp-dir] [--print-path] [--version]
[-v]

Options:
--precise show precise timings in log
--profile run python profiler for ya binary
--error-file ERROR_FILE
--keep-tmp
--no-logs
--no-report
--no-tmp-dir
--print-path
--version
-v, --verbose-level

Available subcommands:
dump Repository related information
gc Collect garbage
gen-config Generate default ya config
ide Generate project for IDE
java Java build helpers
make Build and run tests
To see more help use -hh/-hhh
package Build package using json package description in the release build type by default.
style Run styler
test Build and run all tests
ya test is alias for ya make -A
tool Execute specific tool
Examples:
ya test -tt Build and run small and medium tests
ya test -t Build and run small tests only
ya dump debug Show all items
ya test Build and run all tests
ya dump debug last Upload last debug item
```
### General use

ya [options] [subcommands]

Main options:
- --precise: Displays the log's exact timestamps.
- --profile: Runs a Python profiler for the ya binary.
- --error-file ERROR_FILE: Specifies the path to the file where error messages will be sent.
- --keep-tmp: Saves temporary files after the build is completed.
- --no-logs: Disables log output.
- --no-report: Disables reports.
- --no-tmp-dir: Prevents the creation of a temporary directory during execution.
- --print-path: Outputs the path to the ya executable.
- --version: Displays information about the utility version.
- -v, --verbose-level: Adjusts the verbosity level of messages.

Commands:
- dump: Display information associated with the repository.
- gc: Performs garbage collection.
- gen-config: Generates default configuration for ya.
- ide: Generates an integrated development environment project.
- java: Auxiliary functions for building Java projects.
- make: Executes the build process and runs tests.
- package: Creates a package using the JSON package description.
- style: Executes utilities to check code style.
- test: Compiles and runs all tests (this is an alias for "ya make -A").
- tool: Executes a specific tool.

For more information on any command and its parameters, add the -h, -hh, or -hhh options after the command. This will show you an extended description and use cases.

Examples:
- Run only small and medium tests:
```ya test -tt```
- Run only small tests:
```ya test -t```
- Show all elements using "dump":
```ya dump debug```
- Run all tests:
```ya test```
- Load the last debug element:
```ya dump debug last```

`ya` is a platform-agnostic utility that can run on most operating systems, including Linux, macOS, and Windows.
To run `ya` in Windows without specifying the full path, you can add it to the PATH environment variable.
For Linux or Bash, you can use the following command:
```
echo "alias ya='~/yatool/ya'" >> ~/.bashrc
source ~/.bashrc
```
`ya` uses configuration files in `TOML` format, which you need to place in the project root or in a special directory.

You can learn more about configuration files [here](gen-config.md "Configuration for ya").
122 changes: 122 additions & 0 deletions docs/gc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# ya gc cache

The `ya gc cache` command serves to clear temporary files created when using the `ya make` build system.

The build process produces various intermediate and auxiliary files, and the resulting build files are temporarily cached on the disk.

Over time, these files can take up a significant amount of disk space, and deleting them manually becomes an inconvenient and risky process. The `ya gc cache` command automates the cleanup process by deleting outdated or unnecessary files in line with configuration settings.

After a certain period, most files are supposed to either be moved to permanent storage or be deleted. This helps optimize disk space usage.

## Syntax

`ya gc cache [OPTIONS]`

`[OPTIONS]` allows the user to specify additional parameters to fine-tune the cleanup process. But in the basic use case, no additional parameters are needed.

### Options
```
Options:
Ya operation control
-h, --help Print help. Use -hh for additional options, and -hhh for even more options.
Expert options
-B=CUSTOM_BUILD_DIRECTORY, --build-dir=CUSTOM_BUILD_DIRECTORY
Custom build directory (autodetected by default)
Local cache
--cache-stat Show cache statistics
--gc Remove all cache except uids from the current graph
--gc-symlinks Remove all symlink results except files from the current graph
Advanced options
--tools-cache-size=TOOLS_CACHE_SIZE
Max tool cache size (default: 30.0GiB)
--symlinks-ttl=SYMLINKS_TTL
Results cache TTL (default: 0.0h)
--cache-size=CACHE_SIZE
Max cache size (default: 140.04687070846558GiB)
--cache-codec=CACHE_CODEC
Cache codec (default: )
--auto-clean=AUTO_CLEAN_RESULTS_CACHE
Auto clean results cache (default: True)
```

## Auto-clearing

You can specify the cache clearing configuration locally using the configuration file `~/.ya/ya.conf`.

### Basic functions:

1. Full cleanup of auxiliary files — Deleting all files that are stored in the tool cache, as well as other build data that is normally deleted automatically.
2. Deleting temporary files — This excludes files that are links to files used in other builds.
3. Filtered clearing of local build cache — Allows you to fine-tune the data deletion process using parameters.

#### Auxiliary build files
Auxiliary build files include compilers, SDKs, emulators, and other tools that are not directly included in the build process, but on which `ya` relies for building, running tests, and more. Such files are stored in the tool cache.

Automatic deletion is configured in Linux and MacOS.

These files are deleted in line with the LRU policy as soon as the cache reaches its capacity limit. You can override the default limit using the `tools_cache_size` parameter.

Note that tools get deleted only after all processes that use the tool (compiler, emulator, or other) complete, which is why in-the-moment space usage may exceed the limit.

If you run tools using direct paths obtained with, for example, `ya tool cc --print-path`, then such tools won't be cleared automatically as we have no control over this kind of scenario. These tools will be automatically cleared in the usual order after running `ya gc cache`.

#### Temporary build files

Temporary build files include:
1. Temporary files created using functions that are derived from `mktemp/mkdtemp` and so on. These files are created in a temporary directory, which is fully deleted once `ya` is done.
2. Temporary files that can be used in other builds. These include intermediate build files that are explicitly described in the build via `ya.make` files.
You can limit the total disk space allocated for such files using `cache_size`. This category also includes files loaded from a distributed cache.
3. Temporary files used in the current build, or build directories. In most cases, these files are links to files used in other builds. There are several reasons why such files are put into a separate category. Firstly, some intermediate files are explicitly restricted from being used in other builds; secondly, the build mode with the `--rebuild` key uses only files from the current build; and thirdly, the cache size limit may be too small for the current build.
By default, files in this category are deleted either during the current build or during the next build.

You can't limit in-the-moment disk space usage with the files from the first and third category in advance.

The files from the second category are stored in the local build cache.

### Basic settings for automatic file clearing:
1. `cache_size`: [Local cache](#temporary-build-files) size limit.
2. `tools_cache_size`: [Tool cache](#auxiliary-build-files) size limit.
3. `symlinks_ttl`: The limit of the lifetime of [build results](cache.md) cached by `ya`. The lifetime is counted from the time of creation.

You can disable automatic deletion by setting large values for these parameters, and the deleting of build directories is disabled using the build system's `--keep-temps` option.

#### Sample ya.conf settings
```
tools_cache_size = "6GiB"
symlinks_ttl = "1h"
cache_size = "80GiB"
```
In this example, the tool cache limit is set to 6 GiB, the lifetime of links to build results is set to 1 hour, and the local build cache size is set to 80 GiB.

You can specify the size in bytes or time in seconds without quotes (for example, `symlinks_ttl = 3600`).

## Manual cleanup
In addition to automatic cache clearing, you can explicitly run disk cleanup using the `ya gc cache` command. It will do the cleanup and also perform additional checks for errors.

The command deletes:
- All auxiliary files (except those that are cleaned automatically).
- All temporary files (except [local cache](#temporary-build-files) files).

### Deletion based on size and "age"
```
- --object-size-limit=OBJECT_SIZE_LIMIT: Deletes build cache objects that exceed the specified size (in MiB).
- --age-limit=AGE_LIMIT: Deletes build cache objects that are older than the specified "age" (in hours).
```
If a filter is not set, then the cleanup will proceed in line with the `cache_size` setting.

### Managing build symlinks
```
- --gc-symlinks: Clears outdated build symlinks in the repository.
```
### Practical application examples
Deleting large cache objects
```
ya gc cache --object-size-limit=100
```
Clears the cache of objects larger than 100 MiB, freeing up a significant amount of disk space.

Clearing "old" cache objects
```
ya gc cache --age-limit=72
```
Initiates the deletion of data that hasn't been used in the last 72 hours, keeping the cache up to date and optimized.
Loading