tokenize
verb [ T ] (UK usually tokenise) uk/ˈtəʊ.kən.aɪz/ us/ˈtoʊ.kən.aɪz/
(COMPUTING)
computing, language specialized将文本分割成离散单元
to divide a series of characters (= letters, numbers, or other marks or signs used in writing or printing) into separate tokens (= units) that have a meaning
The stream of characters in a natural language text must be tokenized (broken up into distinct meaningful units) before any language processing can happen. 在进行任何语言处理之前，自然语言文本中的字符流必须被分割成离散单元（被分割成不同的有意义的单元）。
computing specialized（将含隐私信息的数据用另一信息，即令牌，替代，以此保护隐私）令牌化
to replace a private piece of data with a token (= a different piece of data that represents the first one), in order to prevent private information being seen by someone who is not allowed to do so
When we receive the transaction securely, we tokenize it and send back a token to the merchant so they never have to see a card number for settlement. 当我们安全地收到交易后，我们会将其令牌化，并将令牌发回给商家，这样他们就不必看到结算时的卡号。
- More examples
- First, your script must be tokenised and converted into meaningful units.
- Use the huge archive of previous customer searches and tokenize these into words.
- Tokenising a card transaction is considered safer as the actual card details are not shared.
(PERSON)
做门面功夫
to do something that seems to support or help a group of people who are treated unfairly in society, such as giving a member of that group an important or public position, but that does not make changes that would help that group of people in a lasting way
He did not feel as if he had been tokenized while at college but had been held to the same standards as everyone else. 他没有觉得在大学的时候他被刻意优待用来做门面功夫，反而觉得对他的要求和标准同其他人是一样的。
- More examples
- Many employees feel that they are being tokenised in publicity materials, while their concerns about bias and inequality are not actually being addressed.
- How can you tell if you're being tokenized at your workplace?
- They listed 8 ways people of colour are tokenized in nonprofits.