Skip to content

Commit 66522ea

Browse files
committed
Fix HTML tags
1 parent 2887ae1 commit 66522ea

File tree

1 file changed

+5
-9
lines changed

1 file changed

+5
-9
lines changed

Recurrent Neural Networks/LSTM-Language-Modelling.ipynb

+5-9
Original file line numberDiff line numberDiff line change
@@ -18,12 +18,12 @@
1818
"## What exactly is Language Modelling?\n",
1919
"Language Modelling, to put it simply, **is the task of assigning probabilities to sequences of words**. This means that, given a context of one or a few words in the language the model was trained on, the model should have a knowledge of what are the most probable words or sequence of words for the sentence. Language Modelling is one of the tasks under Natural Language Processing, and one of the most important.\n",
2020
"\n",
21-
"<img src=\"https://ibm.box.com/shared/static/1d1i5gub6wljby2vani2vzxp0xsph702.png\" width=\"768\"/>\n",
21+
"<img src=\"https://ibm.box.com/shared/static/1d1i5gub6wljby2vani2vzxp0xsph702.png\"/>\n",
2222
"<center>*Example of a sentence being predicted*</center>\n",
2323
"\n",
2424
"In this example, one can see the predictions for the next word of a sentence, given the context \"This is an\". As you can see, this boils down to a sequential data analysis task - you are given a word or a sequence of words (the input data), and, given the context (the state), you need to find out what is the next word (the prediction). This kind of analysis is very important for language-related tasks such as **Speech Recognition, Machine Translation, Image Captioning, Text Correction** and many other very relevant problems. \n",
2525
"\n",
26-
"<img src=\"https://ibm.box.com/shared/static/az39idf9ipfdpc5ugifpgxnydelhyf3i.png\" width=\"1080\"/>\n",
26+
"<img src=\"https://ibm.box.com/shared/static/az39idf9ipfdpc5ugifpgxnydelhyf3i.png\"/>\n",
2727
"<center>*The above example schematized as an RNN in execution*</center>\n",
2828
"\n",
2929
"As the above image shows, Recurrent Network models fit this problem like a glove. Alongside LSTM and its capacity to maintain the model's state for over one thousand time steps, we have all the tools we need to undertake this problem. The goal is to create a model that can reach **low levels of perplexity** on our desired dataset.\n",
@@ -38,11 +38,9 @@
3838
"The dataset is divided in different kinds of annotations, such as Piece-of-Speech, Syntactic and Semantic skeletons. For this example, we will simply use a sample of clean, non-annotated words (with the exception of one tag - `<unk>`, which is used for rare words such as uncommon proper nouns) for our model. This means that we just want to predict what the next words would be, not what they mean in context or their classes on a given sentence. \n",
3939
"<br/>\n",
4040
"<div class=\"alert alert-block alert-info\">\n",
41-
"<center>the percentage of lung cancer deaths among the workers at the west `<unk>` mass. paper factory appears to be the highest for any asbestos workers studied in western industrialized countries he said \n",
41+
"the percentage of lung cancer deaths among the workers at the west `<unk>` mass. paper factory appears to be the highest for any asbestos workers studied in western industrialized countries he said \n",
4242
" the plant which is owned by `<unk>` & `<unk>` co. was under contract with `<unk>` to make the cigarette filters \n",
4343
" the finding probably will support those who argue that the u.s. should regulate the class of asbestos including `<unk>` more `<unk>` than the common kind of asbestos `<unk>` found in most schools and other buildings dr. `<unk>` said\n",
44-
" \n",
45-
"</center>\n",
4644
"</div>\n",
4745
"<center>*Example of text from the dataset we are going to use, `ptb.train`*</center>"
4846
]
@@ -55,11 +53,9 @@
5553
"\n",
5654
"For better processing, in this example, we will make use of [**word embeddings**]( [https://www.tensorflow.org/tutorials/word2vec/), which are **a way of representing sentence structures or words as n-dimensional vectors (where n is a reasonably high number, such as 200 or 500) of real numbers**. Basically, we will assign each word a randomly-initialized vector, and input those into the network to be processed. After a number of iterations, these vectors are expected to assume values that help the network to correctly predict what it needs to - in our case, the probable next word in the sentence. This is shown to be very effective in Natural Language Processing tasks, and is a commonplace practice.\n",
5755
"<br/><br/>\n",
58-
"<font size = 4>\n",
59-
" <strong>\n",
56+
"<strong>\n",
6057
"$$Vec(\"Example\") = [0.02, 0.00, 0.00, 0.92, 0.30,...]$$\n",
61-
" </strong>\n",
62-
"</font>\n",
58+
"</strong>\n",
6359
"<br/>\n",
6460
"Word Embedding tends to group up similarly used words *reasonably* together in the vectorial space. For example, if we use T-SNE (a dimensional reduction visualization algorithm) to flatten the dimensions of our vectors into a 2-dimensional space and use the words these vectors represent as their labels, we might see something like this:\n",
6561
"\n",

0 commit comments

Comments
 (0)