NLP Support, New Layers, Faster Memory Access, New Examples

CAI Neural API v2.0.0 Release Notes

New Source Code Examples

Several new examples to help you get started with various neural network applications:

1. Malaria Cell Infection Image Classification
2. Colorectal Cancer Image Classification
3. Plant Leaf Disease Image Classification for the PlantVillage Dataset
4. Sentiment Analysis
5. NLP Support: Tokenizer, Samplers, Transformer Decoder
6. Pre-trained Models
7. GPT-3 Small

New Layers

Enhance your neural networks with the following new layers:

• TNNetPadXY
• TNNetCrop
• TNNetMaxPoolWithPosition
• TNNetTransposeXD
• TNNetTransposeYD
• TNNetDotProducts
• TNNetEmbedding
• TNNetAddPositionalEmbedding
• TNNetTokenAndPositionalEmbedding
• TNNetChannelNorm
• TNNetSignedSquareRoot
• TNNetSignedSquareRoot1
• TNNetSignedSquareRootN
• TNNetReLUP
• TNNetPointwiseNorm
• TNNetPointwiseSoftMax
• TNNet.AddSelfAttentionCAI
• TNNet.AddTransformerBlockCAI

Other New Features

Enhancements and new functionalities introduced in this release:

• Image Support:
  • Added TIFF image support.
• Classification Enhancements:
  • Added TNeuralFitWithImageBase.ClassifyImageFromFile.
• Data Handling:
  • Added TStringStringList.LoadFromCsv and TStringStringList.SaveToCsv.
  • Added TVolume.OneHotEncoding(aTokens: array of integer), TVolume.OneHotEncoding(aTokens: string), and TVolume.OneHotEncodingReversed(aTokens: string).
• Neural Network Enhancements:
  • Added TNNetNeuron.Bias property.
• Volume Operations:
  • Added volume entropy and volume perplexity calculations.
  • Added TNNetVolume.DotProductsPointwise.
• Debugging and Logging:
  • Added OpenCL debug status.
• Training Enhancements:
  • Added Adam Optimizer: TNeuralOptimizerAdam.
  • Added the option to save the neural network when the best loss is found (commit details).
  • Added MinBackpropagationErrorProportion fitting property.
  • Added TNeuralFitBase.LogEveryBatches to control log frequency.
• Dependencies:
  • Removed MTPCPU dependency.
• NLP:
  • Added Tokenizer: TNeuralTokenizer.
  • Added Samplers: TNNetSamplerGreedy, TNNetSamplerTopK, and TNNetSamplerTopP.
  • Added Dataset.

Enhancements and Fixes

• Stability Improvements:
  • Fitting (training) is now a lot more stable.
• Memory Optimizations:
  • Implemented numerous memory optimizations.
  • Enhanced memory-efficient grouped pointwise convolutions.
• Bug Fixes:
  • Various bugs have been fixed.
• Documentation:
  • Improved documentation and added more comprehensive source code comments.
  • Added plenty of YouTube Videos for better learning and implementation guidance.

---

Thank you for using the Pascal based Neural API! For any questions or feedback, please visit the GitHub repository.