Recurrent Neural Networks

Hui Lin @Google

Types of Neural Network

Why sequency?

Speech Recognition \(\longrightarrow\) Get your facts first, then you can distort them as you please.
Music generation \(\emptyset\) \(\longrightarrow\)
Sentiment classification Great movie ? Are you kidding me ! Not worth the money. \(\longrightarrow\)
DNA sequence analysis ACGGGGCCTACTGTCAACTG \(\longrightarrow\) AC GGGGCCTACTG TCAACTG
Machine translation 网红脸 \(\longrightarrow\) Internet celebrity face
Video activity recognition \(\longrightarrow\) Running
Name entity recognition Use Netlify and Hugo. \(\longrightarrow\) Use Netlify and Hugo.

RNN types

Notation

Representing words

\(\left[\begin{array}{c} a[1]\\ aaron[2]\\ \vdots\\ and[360]\\ \vdots\\ Hugo[4075]\\ \vdots\\ Netlify[5210]\\ \vdots\\ use[8320]\\ \vdots\\ Zulu[10000] \end{array}\right]\Longrightarrow use=\left[\begin{array}{c} 0\\ 0\\ \vdots\\ 0\\ \vdots\\ 0\\ \vdots\\ 0\\ \vdots\\ 1\\ \vdots\\ 0 \end{array}\right], Netlify=\left[\begin{array}{c} 0\\ 0\\ \vdots\\ 0\\ \vdots\\ 0\\ \vdots\\ 1\\ \vdots\\ 0\\ \vdots\\ 0 \end{array}\right], and=\left[\begin{array}{c} 0\\ 0\\ \vdots\\ 1\\ \vdots\\ 0\\ \vdots\\ 0\\ \vdots\\ 0\\ \vdots\\ 0 \end{array}\right], Hugo=\left[\begin{array}{c} 0\\ 0\\ \vdots\\ 0\\ \vdots\\ 1\\ \vdots\\ 0\\ \vdots\\ 0\\ \vdots\\ 0 \end{array}\right]\)

What is RNN?