-
Notifications
You must be signed in to change notification settings - Fork 571
Commit
- Loading branch information
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# Sphinx build info version 1 | ||
# This file records the configuration used when building these files. When it is not found, a full rebuild will be done. | ||
config: 31899dca82da6cb45546146b22ddc4c6 | ||
tags: 645f666f9bcd5a90fca523b33c5a78b7 |
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
Abstract Class | ||
============== | ||
|
||
.. toctree:: | ||
abc/inference | ||
abc/finetune |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
Finetune | ||
======== | ||
|
||
.. toctree:: | ||
finetune/embedder | ||
finetune/reranker |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
Embedder | ||
======== | ||
|
||
.. toctree:: | ||
embedder/AbsArguments | ||
embedder/AbsDataset | ||
embedder/AbsModeling | ||
embedder/AbsTrainer | ||
embedder/AbsRunner |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
AbsArguments | ||
============ | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.reranker.AbsRerankerModelArguments | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.reranker.AbsRerankerDataArguments |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,53 @@ | ||
========== | ||
AbsDataset | ||
========== | ||
|
||
AbsEmbedderTrainDataset | ||
======================= | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderTrainDataset | ||
|
||
Methods | ||
------- | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderTrainDataset._load_dataset | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderTrainDataset._shuffle_text | ||
|
||
AbsEmbedderCollator | ||
=================== | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderCollator | ||
|
||
AbsEmbedderSameDatasetTrainDataset | ||
================================== | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderSameDatasetTrainDataset | ||
|
||
Methods | ||
------- | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderSameDatasetTrainDataset.refresh_epoch | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderSameDatasetTrainDataset._load_dataset | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderSameDatasetTrainDataset._get_file_batch_size | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderSameDatasetTrainDataset._get_train_group_size | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderSameDatasetTrainDataset._create_batch_data | ||
|
||
AbsEmbedderSameDatasetCollator | ||
============================== | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderSameDatasetCollator | ||
|
||
EmbedderTrainerCallbackForDataRefresh | ||
===================================== | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.embedder.EmbedderTrainerCallbackForDataRefresh | ||
|
||
Methods | ||
------- | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.EmbedderTrainerCallbackForDataRefresh.on_epoch_end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
=========== | ||
AbsModeling | ||
=========== | ||
|
||
AbsEmbedderModel | ||
================ | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderModel | ||
|
||
Methods | ||
------- | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderModel.encode | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderModel.compute_loss | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderModel.compute_score | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderModel.save | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderModel.get_local_score | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderModel.compute_local_score | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderModel.forward | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderModel.distill_loss | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderModel._compute_no_in_batch_neg_loss | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderModel._compute_in_batch_neg_loss | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderModel._compute_cross_device_neg_loss | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderModel._dist_gather_tensor | ||
|
||
|
||
EmbedderOutput | ||
============== | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.embedder.EmbedderOutput |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
========= | ||
AbsRunner | ||
========= | ||
|
||
AbsEmbedderTrainer | ||
================== | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderRunner | ||
|
||
Methods | ||
------- | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderRunner.load_tokenizer_and_model | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderRunner.load_trainer | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderRunner.load_train_dataset | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderRunner.load_data_collator | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderRunner.run |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
========== | ||
AbsTrainer | ||
========== | ||
|
||
AbsEmbedderTrainer | ||
================== | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderTrainer | ||
|
||
Methods | ||
------- | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderTrainer.compute_loss |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
Reranker | ||
======== | ||
|
||
.. toctree:: | ||
reranker/AbsArguments | ||
reranker/AbsDataset | ||
reranker/AbsModeling | ||
reranker/AbsTrainer | ||
reranker/AbsRunner |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
AbsArguments | ||
============ | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderModelArguments | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.embedder.AbsEmbedderDataArguments |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
========== | ||
AbsDataset | ||
========== | ||
|
||
AbsRerankerTrainDataset | ||
======================= | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.reranker.AbsRerankerTrainDataset | ||
|
||
Methods | ||
------- | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerTrainDataset.create_one_example | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerTrainDataset._load_dataset | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerTrainDataset._shuffle_text | ||
|
||
AbsRerankerCollator | ||
=================== | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.reranker.AbsRerankerCollator | ||
|
||
AbsLLMRerankerTrainDataset | ||
========================== | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.reranker.AbsLLMRerankerTrainDataset | ||
|
||
AbsLLMRerankerCollator | ||
====================== | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.reranker.AbsLLMRerankerCollator |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
=========== | ||
AbsModeling | ||
=========== | ||
|
||
AbsRerankerModel | ||
================ | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.reranker.AbsRerankerModel | ||
|
||
Methods | ||
------- | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerModel.encode | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerModel.gradient_checkpointing_enable | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerModel.enable_input_require_grads | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerModel.forward | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerModel.compute_loss | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerModel.save | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerModel.save_pretrained | ||
|
||
|
||
RerankerOutput | ||
============== | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.reranker.RerankerOutput |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
========= | ||
AbsRunner | ||
========= | ||
|
||
AbsRerankerTrainer | ||
================== | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.reranker.AbsRerankerRunner | ||
|
||
Methods | ||
------- | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerRunner.load_tokenizer_and_model | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerRunner.load_trainer | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerRunner.load_train_dataset | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerRunner.load_data_collator | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerRunner.run |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
========== | ||
AbsTrainer | ||
========== | ||
|
||
AbsRerankerTrainer | ||
================== | ||
|
||
.. autoclass:: FlagEmbedding.abc.finetune.reranker.AbsRerankerTrainer | ||
|
||
Methods | ||
------- | ||
|
||
.. automethod:: FlagEmbedding.abc.finetune.reranker.AbsRerankerTrainer.compute_loss |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
Inference | ||
========= | ||
|
||
.. toctree:: | ||
inference/AbsEmbedder | ||
inference/AbsReranker |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
AbsEmbedder | ||
=========== | ||
|
||
.. autoclass:: FlagEmbedding.abc.inference.AbsEmbedder | ||
|
||
Methods | ||
------- | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsEmbedder.get_target_devices | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsEmbedder.get_detailed_instruct | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsEmbedder.encode_queries | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsEmbedder.encode_corpus | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsEmbedder.encode | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsEmbedder.encode_single_device | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsEmbedder.start_multi_process_pool | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsEmbedder._encode_multi_process_worker | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsEmbedder.stop_multi_process_pool | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsEmbedder.encode_multi_process | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsEmbedder._concatenate_results_from_multi_process |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
AbsReranker | ||
=========== | ||
|
||
.. autoclass:: FlagEmbedding.abc.inference.AbsReranker | ||
|
||
Methods | ||
------- | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsReranker.get_target_devices | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsReranker.get_detailed_instruct | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsReranker.get_detailed_inputs | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsReranker.compute_score | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsReranker.compute_score_single_gpu | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsReranker.start_multi_process_pool | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsReranker.encode_multi_process | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsReranker._encode_multi_process_worker | ||
|
||
.. automethod:: FlagEmbedding.abc.inference.AbsReranker.stop_multi_process_pool |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
Evaluation | ||
========== |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
Finetune | ||
======== | ||
|
||
.. toctree:: | ||
finetune/embedder | ||
finetune/reranker |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
Embedder | ||
======== | ||
|
||
.. toctree:: | ||
embedder/encoder_only | ||
embedder/decoder_only |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
Decoder Only | ||
============ | ||
|
||
.. toctree:: | ||
decoder_only/base | ||
decoder_only/icl |