fastBERTtokens: Tokenizing for BERT in parallel

This function simply divides your text into batches, and tokenizes in parallel. Provides significant speed-up.

Version 1.0.0 (1.43 KB)
31 Downloads
(0)
24 Feb 2023
Download

  • Install in MATLAB

    Requires R2026b and later. Learn more
Download

  • Install in MATLAB

    Requires R2026b and later. Learn more

You are now following this Submission

Cite As

Ralf Elsas (2026). fastBERTtokens: Tokenizing for BERT in parallel (https://se.mathworks.com/matlabcentral/fileexchange/125295-fastberttokens-tokenizing-for-bert-in-parallel), MATLAB Central File Exchange. Retrieved .

Acknowledgements

Inspired by: Transformer Models

General Information

MATLAB Release Compatibility

  • Compatible with R2021a and later releases

Platform Compatibility

  • Windows
  • macOS
  • Linux
Open in new tab
Version Published Release Notes
1.0.0

Download

  • Install in MATLAB

    Requires R2026b and later. Learn more