Skip to content

v1.2.2

Latest
Compare
Choose a tag to compare
@belladoreai belladoreai released this 27 Jun 17:59

Minor fixes to decode, which should not have any effect except for situations where the user inputs invalid parameter combinations:

  • When user decodes with add_bos_token set to True, before this change we would just cut the first token out, assuming it must be bos token, but now we check if it's the bos token, and don't cut the first token if it's something else
  • When user decodes with add_preceding_space set to True, before this change we would just assume that the first character after decoding must be space, and we would cut it out, but now we check if it's space, and don't cut it out if it's something else