A calculation cost reduction method for a log-likelihood maximization in word2vec

Sakuya Nakamura, Masaomi Kimura

研究成果: Conference contribution

抜粋

Word2vec models learn text data and provide distributed representations to words. The distributed representations use vectors which show the meaning of the words. Thus the word2vec models are useful for Natural Language Processing (NLP). However, it is difficult to update the models for new data addition because it takes a long time to generate the word2vec model. This calculation time has become an impediment to analize text data which contains a lot of unknown words. This is caused by computational time in the calculation of the likelihood function. The purpose of this study was to speed up the training of Continuous Bag-of-Word Model(CBOW), which is one of the word2vec models, by reducing the calculation cost of the likelihood function. The likelihood function in CBOW has been expressed by the use of a softmax function and has a huge amount of computational time. In this paper, a sigmoid function replaces the softmax function as the approximated likelihood function, because the sigmoid function can reproduce the charactaristic change of the likelihood function in CBOW.

元の言語English
ホスト出版物のタイトルICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing
編集者Hui Yu
出版者Institute of Electrical and Electronics Engineers Inc.
ISBN(電子版)9781861376664
DOI
出版物ステータスPublished - 2019 9
イベント25th IEEE International Conference on Automation and Computing, ICAC 2019 - Lancaster, United Kingdom
継続期間: 2019 9 52019 9 7

出版物シリーズ

氏名ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing

Conference

Conference25th IEEE International Conference on Automation and Computing, ICAC 2019
United Kingdom
Lancaster
期間19/9/519/9/7

    フィンガープリント

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications
  • Control and Optimization

これを引用

Nakamura, S., & Kimura, M. (2019). A calculation cost reduction method for a log-likelihood maximization in word2vec. : H. Yu (版), ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing [8895214] (ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.23919/IConAC.2019.8895214