tokenization
noun [ U ] (UK usually tokenisation) uk/ˌtəʊ.kən.aɪˈzeɪ.ʃən/ us/ˌtoʊ.kən.əˈzeɪ.ʃən/
(COMPUTING)
computing, language specialized将文本分割成离散单元的过程
the process of dividing a series of characters (= letters, numbers, or other marks or signs used in writing or printing) into separate tokens (= units) that have a meaning
Tokenization shows the text as a series of tokens or useful meaningful units that are larger than individual characters, but smaller than phrases and sentences.
The lowest-level form of annotation is tokenisation. 最低层次的注释是离散单元化。
computing specialized（将含隐私信息的数据用另一信息，即令牌，替代，以此保护隐私）令牌化
the process of replacing a private piece of data with a token (= a different piece of data that represents the first one), in order to prevent private information being seen by someone who is not allowed to do so
Tokenization protects sensitive cardholder data by removing it from merchant systems entirely. 令牌化通过将敏感的持卡人数据从商户系统中完全移除来保护这些数据。
- More examples
- Orthography-oriented tokenization and segmentation are tackled in the main by using orthographic hints such as blank spaces that mark the limits of words.
- For languages such as Chinese and Japanese, the use of a dictionary for sentence tokenisation and segmentation is essential.
- The integrated tokenization technology means that no cardholder data is stored, transmitted or processed at the property.
(PERSON)
门面功夫
the act of doing something that seems to support or help a group of people who are treated unfairly in society, such as giving a member of that group an important or public position, but that does not make changes that would help that group of people in a lasting way
Research shows that groups who experience minority status often encounter tokenisation. 研究表明那些有过成为少数族裔经历的人常常被人（优待）用来做门面功夫。
Compare: tokenism
- More examples
- There has been a tokenisation of minority rights without a real empowerment of minorities themselves.
- Tokenization often results in token employees reporting as feeling lonely and disconnected from the majority employees.
- By tokenization, an impression is created that a business is inclusive and diverse.