.. ICU-Tokenizer documentation master file, created by sphinx-quickstart on Thu Jun 18 23:55:49 2020. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. Welcome to ICU-Tokenizer's documentation! ========================================= **ICU-tokenizer** is a python package used to perform universal language normalization and tokenization using the International Components for Unicode. .. toctree:: :maxdepth: 2 :caption: Library Reference tools api