Yuna-yun Chao

0 followers · 308 views

on the atlas · 17

highlights · 20

Allows the information to go back from the cost backward through the network in order to compute the gradient.
Coding Neural Network — Forward Propagation and Backpropagtion | by Imad Dabbura | Towards Data Science
The encoded audio sequence is passed to a decoder, where a softmax function is applied to all the words in a set vocabulary
What is Beam Search? Explaining The Beam Search Algorithm | Width.ai
Lets set our beam width to 3 and grab the top three predicted words at each position in a given sequence.
What is Beam Search? Explaining The Beam Search Algorithm | Width.ai
The co-occurrence matrix tells us the information about the occurrence of the words in different pairs.
Word2Vec vs GloVe - A Comparative Guide to Word Embedding Techniques
It starts working by building a large matrix which consists of the words co-occurrence information
Word2Vec vs GloVe - A Comparative Guide to Word Embedding Techniques
This unsupervised learning algorithm maps the words into space where the semantic similarity between the words is observed by the distance between the words.
Word2Vec vs GloVe - A Comparative Guide to Word Embedding Techniques
more frequent words are more likely to be selected as negative samples
Word2Vec Tutorial Part 2 - Negative Sampling · Chris McCormick
the probability for picking the word “couch” would be equal to the number of times “couch” appears in the corpus, divided the total number of word occus in the corpus.
Word2Vec Tutorial Part 2 - Negative Sampling · Chris McCormick
如果我們詞彙表的大小是 10,000 的話，則在輸出層，我們期望對應 “quick” 這個字詞的神經元節點輸出是 1，其他 9.999 個神經元輸出都是 0，這 9,999 個期望輸出為 0 的神經元節點所對應的字詞我們就稱為 “negative word”。
讓電腦聽懂人話: 直觀理解 Word2Vec 模型. Word2Vec 是 Google 於 2013 年由 Tomas… | by TengYuan Chang | Medium
只更新一部分的權重，而非整個神經網路的權重都被更新。
讓電腦聽懂人話: 直觀理解 Word2Vec 模型. Word2Vec 是 Google 於 2013 年由 Tomas… | by TengYuan Chang | Medium
Negative sampling 的好處是，透過隨機取樣的方式，降低錯誤信號對整體模型造成的影響。
讓電腦聽懂人話: 直觀理解 Word2Vec 模型. Word2Vec 是 Google 於 2013 年由 Tomas… | by TengYuan Chang | Medium
instead going to randomly select just a small number of “negative” words (let’s say 5) to update the weights for.
Word2Vec Tutorial Part 2 - Negative Sampling · Chris McCormick
Negative sampling addresses this by having each training sample only modify a small percentage of the weights, rather than all of them.
Word2Vec Tutorial Part 2 - Negative Sampling · Chris McCormick
If two different words have very similar “contexts” (that is, what words are likely to appear around them), then our model needs to output very similar results for these two words.
Word2Vec Tutorial - The Skip-Gram Model · Chris McCormick
Regression問題時，Normal Distribution使用Linear當Activation Function Binary Classification問題時，Bernoulli Distribution使用Sigmoid當Activation Function Multi-class Classification問題時，Categorical Distribution使用Softmax當Activation Function
剖析深度學習 (4)：Sigmoid, Softmax怎麼來？為什麼要用MSE和Cross Entropy？談廣義線性模型 - YC Note
Softmax 函數通常會放在類神經網路的最後一層，將最後一層所有節點的輸出都通過指數函數 (exponential function)，並將結果相加作為分母，個別的輸出作為分子。
使用 TensorFlow 學習 Softmax 回歸 (Softmax Regression) | by Airwaves | 手寫筆記 | Medium
Softmax 回歸是使用 Softmax 運算使得最後一層輸出的機率分佈總和為 1
使用 TensorFlow 學習 Softmax 回歸 (Softmax Regression) | by Airwaves | 手寫筆記 | Medium
target word is input while context words are output.
Skip-Gram: NLP context words prediction algorithm | by Sanket Doshi | Towards Data Science
Skip-gram is one of the unsupervised learning techniques used to find the most related words for a given word.
Skip-Gram: NLP context words prediction algorithm | by Sanket Doshi | Towards Data Science
Skip-gram is used to predict the context word for a given target word.
Skip-Gram: NLP context words prediction algorithm | by Sanket Doshi | Towards Data Science