Skip to content

Commit

Permalink
chore: fix broken or incorrect links in documentation/files
Browse files Browse the repository at this point in the history
Signed-off-by: Abhishek Gaikwad <[email protected]>
  • Loading branch information
gaikwadabhishek committed Jan 23, 2025
1 parent 8109466 commit 98cf030
Show file tree
Hide file tree
Showing 6 changed files with 9 additions and 9 deletions.
4 changes: 2 additions & 2 deletions docs/_posts/2022-07-11-aisio-pytorch.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ categories: aistore pytorch sdk python
Listing and loading data from AIS buckets (buckets that are not 3rd
party backend-based) and remote cloud buckets (3rd party backend-based
cloud buckets) using
[AISFileLister](https://pytorch.org/data/main/generated/torchdata.datapipes.iter.AISFileLister.html#aisfilelister)
[AISFileLister](https://pytorch.org/data/0.8/generated/torchdata.datapipes.iter.AISFileLister.html)
and
[AISFileLoader](https://pytorch.org/data/main/generated/torchdata.datapipes.iter.AISFileLoader.html#torchdata.datapipes.iter.AISFileLoader).
[AISFileLoader](https://pytorch.org/data/0.8/generated/torchdata.datapipes.iter.AISFileLoader.html).

[AIStore](https://github.com/NVIDIA/aistore) (AIS for short) fully supports
Amazon S3, Google Cloud, and Microsoft Azure backends, providing a
Expand Down
4 changes: 2 additions & 2 deletions docs/_posts/2022-07-20-python-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,14 +202,14 @@ Eviction of a cloud storage bucket destroys any instance of the cloud storage bu

## PyTorch

PyTorch provides built-in [tools](https://github.com/pytorch/data/tree/main/torchdata/datapipes/iter/load#aistore-io-datapipe) for AIStore integration, allowing machine learning developers to easily use AIStore as a viable storage system option with PyTorch. In fact, the dataloading classes [`AISFileLister`](https://pytorch.org/data/main/generated/torchdata.datapipes.iter.AISFileLister.html#aisfilelister) and [`AISFileLoader`](https://pytorch.org/data/main/generated/torchdata.datapipes.iter.AISFileLoader.html#torchdata.datapipes.iter.AISFileLoader) found in [`aisio.py`](https://github.com/pytorch/data/blob/main/torchdata/datapipes/iter/load/aisio.py) provided by PyTorch make use of several of the client-side APIs referenced in this article.
PyTorch provides built-in [tools](https://github.com/NVIDIA/aistore/blob/main/python/aistore/pytorch/aisio.py) for AIStore integration, allowing machine learning developers to easily use AIStore as a viable storage system option with PyTorch. In fact, the dataloading classes [`AISFileLister`](https://pytorch.org/data/0.8/generated/torchdata.datapipes.iter.AISFileLister.html) and [`AISFileLoader`](https://pytorch.org/data/0.8/generated/torchdata.datapipes.iter.AISFileLoader.html) found in [`aisio.py`](https://github.com/NVIDIA/aistore/blob/main/python/aistore/pytorch/aisio.py) provided by PyTorch make use of several of the client-side APIs referenced in this article.

For more information on dataloading from AIStore with PyTorch, please refer to this [article](https://aiatscale.org/blog/2022/07/12/aisio-pytorch).


## More Examples & Resources

For more examples, please refer to additional documentation [AIStore Python SDK](https://github.com/NVIDIA/aistore/tree/main/python/aistore) and try out the [SDK tutorial (Jupyter Notebook)](https://github.com/NVIDIA/aistore/blob/main/python/aistore/sdk-tutorial.ipynb).
For more examples, please refer to additional documentation [AIStore Python SDK](https://github.com/NVIDIA/aistore/tree/main/python/aistore) and try out the [SDK tutorial (Jupyter Notebook)](https://github.com/NVIDIA/aistore/blob/main/python/examples/sdk/sdk-basics.ipynb).

For information on specific API usage, please refer to the [API reference](https://aistore.nvidia.com/docs/python-sdk).

Expand Down
2 changes: 1 addition & 1 deletion docs/_posts/2024-11-22-put-performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ First, we ran a series of benchmarks to determine AIS performance given a simple
Our AIStore cluster is a production-level 16-node cluster running in OCI (Oracle Cloud Infrastructure) managed Kubernetes.
We've selected the [E5 DenseIO nodes](https://docs.oracle.com/iaas/Content/Compute/References/computeshapes.htm#bm-dense), each with 12 5.8 TiB NVME drives and 100 Gb networking.

For the benchmark client we used a single instance of the same node shape (E5 DenseIO) running our [AISLoader client]((https://github.com/NVIDIA/aistore/blob/main/docs/aisloader.md)) -- a simple multi-threaded HTTP-based workload generator.
For the benchmark client we used a single instance of the same node shape (E5 DenseIO) running our [AISLoader client](https://github.com/NVIDIA/aistore/blob/main/docs/aisloader.md) -- a simple multi-threaded HTTP-based workload generator.
For this test we ran the client with 10 worker threads in order to keep the test small and avoid any external factors.
With larger numbers of client threads, we could run into rate limits imposed by AWS S3 (3,500 PUTs per prefix at time of writing) or throughput limits of our outbound network.

Expand Down
2 changes: 1 addition & 1 deletion docs/environment-vars.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,7 +262,7 @@ As far as, specifically, StatsD alternative, additional environment includes:

| name | comment |
| ---- | ------- |
| `AIS_STATSD_PORT` | use it to override the default `8125` (see https://github.com/etsy/stats) |
| `AIS_STATSD_PORT` | use it to override the default `8125` (see https://github.com/etsy/statsd) |
| `AIS_STATSD_PROBE` | a startup option that, when true, tells an ais node to _probe_ whether StatsD server exists (and responds); if the probe fails, the node will disable its StatsD functionality completely - i.e., will not be sending any metrics to the StatsD port (above) |

## Package: memsys
Expand Down
2 changes: 1 addition & 1 deletion python/examples/aisio-pytorch/aisio_pytorch_example.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"source": [
"# PyTorch: Loading Data from AIStore \n",
"\n",
"Listing and loading data from AIS buckets (buckets that are not 3rd party backend-based) and remote cloud buckets (3rd party backend-based cloud buckets) using [AISFileLister](https://pytorch.org/data/beta/generated/torchdata.datapipes.iter.AISFileLister.html) and [AISFileLoader](https://pytorch.org/data/beta/generated/torchdata.datapipes.iter.AISFileLoader.html).\n",
"Listing and loading data from AIS buckets (buckets that are not 3rd party backend-based) and remote cloud buckets (3rd party backend-based cloud buckets) using [AISFileLister](https://pytorch.org/data/0.8/generated/torchdata.datapipes.iter.AISFileLister.html) and [AISFileLoader](https://pytorch.org/data/0.8/generated/torchdata.datapipes.iter.AISFileLoader.html).\n",
"\n",
"In the following example, we use the [Caltech-256 Object Category Dataset](https://authors.library.caltech.edu/7694/) containing 256 object categories and a total of 30607 images stored on an AIS bucket and the [Microsoft COCO Dataset](https://cocodataset.org/#home) which has 330K images with over 200K labels of more than 1.5 million object instances across 80 object categories stored on Google Cloud. "
]
Expand Down
4 changes: 2 additions & 2 deletions stats/common_statsd.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
// Package stats provides methods and functionality to register, track, log,
// and StatsD-notify statistics that, for the most part, include "counter" and "latency" kinds.
/*
* Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.
* Copyright (c) 2018-2025, NVIDIA CORPORATION. All rights reserved.
*/
package stats

Expand Down Expand Up @@ -65,7 +65,7 @@ func (s *coreStats) statsdDisabled() bool { return s.statsdC == nil }

func (s *coreStats) initStarted(snode *meta.Snode) {
var (
port = 8125 // StatsD default port, see https://github.com/etsy/stats
port = 8125 // StatsD default port, see https://github.com/etsy/statsd
probe = false // test-probe StatsD server at init time
)
if portStr := os.Getenv("AIS_STATSD_PORT"); portStr != "" {
Expand Down

0 comments on commit 98cf030

Please sign in to comment.