Add TokenizeVocabulary to api docs (#17208)

Adds the `TokenizeVocabulary` class to the cuDF API guide. Also removes the `SubwordTokenizer` which is to be deprecated in the future. Authors: - David Wendt (https://github.com/davidwendt) Approvers: - Vyas Ramasubramani (https://github.com/vyasr) URL: #17208
rapidsai · Oct 31, 2024 · 3db6a0e · 3db6a0e
1 parent 9657c9a
commit 3db6a0e
Show file tree

Hide file tree

Showing 3 changed files with 13 additions and 13 deletions.
diff --git a/docs/cudf/source/user_guide/api_docs/index.rst b/docs/cudf/source/user_guide/api_docs/index.rst
@@ -19,7 +19,7 @@ This page provides a list of all publicly accessible modules, methods and classe
     general_utilities
     window
     io
-    subword_tokenize
+    tokenize_vocabulary
     string_handling
     list_handling
     struct_handling

diff --git a/docs/cudf/source/user_guide/api_docs/subword_tokenize.rst b/docs/cudf/source/user_guide/api_docs/subword_tokenize.rst
diff --git a/docs/cudf/source/user_guide/api_docs/tokenize_vocabulary.rst b/docs/cudf/source/user_guide/api_docs/tokenize_vocabulary.rst
@@ -0,0 +1,12 @@
+==================
+TokenizeVocabulary
+==================
+.. currentmodule:: cudf.core.tokenize_vocabulary
+
+Constructor
+~~~~~~~~~~~
+.. autosummary::
+   :toctree: api/
+
+   TokenizeVocabulary
+   TokenizeVocabulary.tokenize