-
Notifications
You must be signed in to change notification settings - Fork 53
/
Copy pathtemplate.cpt
132 lines (132 loc) · 6.45 KB
/
template.cpt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
#Concept Concept-fa Keyword
# ---------- Optimizers ----------
optim-sgd null SGD|gradient decent
optim-adam optim-sgd Adam
optim-adagrad optim-sgd Adagrad
optim-adadelta optim-sgd Adadelta
optim-noam optim-adam specialized Transformer learning rate
optim-momentum optim-sgd SGD with Momentum
optim-amsgrad optim-sgd AMSGrad
optim-projection optim-sgd projection|projected gradient descent
# ---------- Initialization ----------
init-glorot null Xavier|Glorot
init-he null He initialization
# ---------- Regularization ----------
reg-dropout null Dropout|dropout
reg-worddropout null word dropout
reg-stopping null early stopping
reg-patience null patience
reg-norm null Norm (L1/L2) Regularization|L2 regularization
reg-decay null weight decay
reg-labelsmooth null label smooth
# ---------- Normalization ----------
norm-layer null Layer Normalization|layer normalization
norm-batch null Batch Normalization|batch normalization
norm-gradient null gradient clipping|gradient normalization|clipnorm
# ---------- Training Paradigms: Multi-task/Multi-lingual/Transfer ----------
train-mtl null multi-task learning
train-mll train-multitask cross-lingual|multi-lingual|cross language
train-transfer null transfer learning|domain adaptation
train-active null active learning
train-augment null data augmentation|Data Augmentation
train-curriculum null data curriculum
train-parallel null parallelism
# ---------- Activation Functions ----------
activ-tanh null Hyperbolic Tangent|hyperbolic tangent
activ-relu null Rectified Linear Units|rectified linear
# ---------- Pooling Operations ----------
pool-max null Max Pooling|max-pooling|max pooling
pool-mean null Mean Pooling|mean pooling|Average Pooling|average pooling
pool-kmax null k-Max Pooling|k-max pooling
# ---------- Recurrent Architectures ----------
arch-rnn null Recurrent Neural Network|RNN|recurrent neural networks
arch-birnn arch-rnn Bi-directional Recurrent Neural Network|Bi-RNN|BiRNN
arch-lstm arch-rnn Long Short-term Memory|LSTMs|LSTM
arch-bilstm arch-rnn Bi-directional Long Short-term Memory|BiLSTM|BiLSTMs|Bi-LSTM|BLSTM
arch-gru arch-rnn Gated Recurrent Units|GRU|GRUs
arch-bigru arch-rnn Bi-directional GRU|Bi-GRU|BiGRU
# ---------- Other Sequential Architectures ----------
arch-bow null bag-of-words|bag-of-embeddings|deep averaging network
arch-cnn null Convolutional Neural Networks|CNNs|convolutional neural network
arch-att null attention
arch-selfatt arch-att Self Attention|self attention|self-attention
arch-recnn null Recursive Neural Network
arch-treelstm null Tree-LSTM|TreeLSTM|Tree-structured Long Short-term Memory
arch-gnn null Graph Neural Network|GNN
arch-gcnn null Graph Convolutional Neural Network|GCNN
# ---------- Architectural Techniques ----------
arch-residual null residual connections
arch-gating null gating connections|Highway
arch-memo null memory network|external memory
arch-copy null copy mechanism|copying mechanism
arch-bilinear null bilinear|bi-linear|biaffine|bi-affine
arch-coverage null coverage
arch-subword null subword|BPE|sentencepiece
arch-energy null energy-based|globally normalized|global normalization
# ---------- Standard Composite Architectures ----------
arch-transformer arch-selfatt Transformer
# ---------- Model Combination ----------
comb-ensemble null ensemble|ensembling
# ---------- Search Algorithms ----------
search-greedy null Greedy Search|greedy search
search-beam null Beam Search|beam search
search-astar null A\* Search
search-viterbi null Viterbi Algorithm|Viterbi|viterbi
search-sampling null Ancestral Sampling|ancestral sampling
search-gumbel search-sampling Gumbel Max|gumbel max
# ---------- Pre-trained Embedding Techniques ----------
pre-word2vec null word2vec|Word2vec
pre-fasttext null fasttext|FastText|fastText
pre-glove null glove|GloVe
pre-paravec null paragraph vector|ParaVector
pre-skipthought task-seq2seq Skip-thought|skip-thought|skipthought
pre-elmo task-lm ELMo
pre-bert arch-transformer BERT
pre-use null Universal Sentence Encoder|universal sentence encoder
# ---------- Pre-trained Embedding Techniques ----------
struct-hmm null Hidden Markov Models|hidden markov
struct-crf null Conditional Random Fields|conditional random fields|CRF
struct-cfg null Context-free Grammar|context-free grammar
struct-ccg null Combinatorial Categorical Grammar|combinatorial categorical grammar
# ---------- Relaxation/Training Methods for Non-differentiable Functions ----------
nondif-enum null Complete Enumeration|complete enumeration
nondif-straightthrough null Straight-through Estimator|straight-through estimator
nondif-gumbelsoftmax null Gumbel Softmax|gumbel softmax
nondif-minrisk null Minimum Risk Training|minimum risk
nondif-reinforce null REINFORCE
# ---------- Adversarial Methods ----------
adv-gan null Generative Adversarial Networks|GAN|generative adversarial
adv-feat null Adversarial Feature Learning|adversarial feature
adv-examp null Adversarial Examples|adversarial examples
adv-train adv-examp Adversarial Training|adversarial trainin
# ---------- Latent Variable Models ----------
latent-vae null Variational Auto-encoder|variational auto-encoder|latent variable
latent-topic null topic model
# ---------- Loss Functions ----------
loss-cca null Canonical Correlation Analysis|canonical correlation analysis
loss-svd null Singular Value Decomposition|SVD|singular value decomposition
loss-margin null Margin-based Loss Functions|margin-based|ranking-based loss
loss-cons null Contrastive Loss
loss-nce loss-cons Noise Contrastive Estimation|NCE
loss-triplet loss-cons Triplet loss|triplet loss
# ---------- Prediction Tasks ----------
task-textclass null Text Classification|text classification
task-textpair null natural language inference|semantic matching|question answering matching
task-seqlab null named entity recognition|Part-of-Speech|word segmentation|text chunking
task-extractive task-seqlab extractive summarization
task-spanlab null span labeling|machine reading comprehension|SQuAD
task-lm null language model
task-condlm null image caption
task-seq2seq null machine translat|abstractive summarization
task-cloze null cloze-style prediction|masked language model|text cloze
task-context null context prediction
task-relation null dependency pars
task-tree null syntactic pars|semantic pars
task-graph null AMR|UDD
task-lexicon null lexicon induction|bi-lingual embedding|embedding alignment|MUSE
task-alignment null word alignment|GIZA
# ---------- Meta Learning ----------
meta-init null MAML
meta-optim null meta optimizer|meta learner
meta-loss null meta learning loss
meta-arch null architecture search