1、dense_4(Dense)(None,24)408
2. Designing a sarcasm detector using "embedding" techniques
3、tf.keras.layers.Embedding(vocab_size,embedding_dim),
4、=================================================================
5、tf.keras.layers.Dense(1,activation='sigmoid')
6、tf.keras.layers.Dense(24,activation='relu'),
7、embedding_2(Embedding)(None,None,16)160000
8. Using the "embedding" layer in the TensorFlow AI development framework
9, the output of this layer is then passed on to the final single neuron layer, where there will be (1 x 24) + 1 = 25 parameters to learn.
10. For example, consider the following model architecture:
11、Trainableparams:160,433
12、model=tf.keras.Sequential([
13, Over time, these dimensions will be obtained through backpropagation learning as the network learns how to match the training data to the labels.
14、tf.keras.layers.GlobalAveragePooling1D(),
15、training_padded=np.array(training_padded)
16, they are fairly abstract, so I think the best way to understand embedding is to do it hands-on. Let's start by building a sarcasm detector using the sarcasm dataset from Chapter Five.
17、Non-trainableparams:0
18、=================================================================
19、testing_padded=np.array(testing_padded)
20. These data were generated by a lexer that specified the maximum vocabulary size and out-of-vocabulary (OOV) tokens:
21、importnumpyasnp
22、testing_labels=np.array(testing_labels)
23、dense_5(Dense)(None,1)25
24, To initialize the embedding layer, you need the vocabulary size and the specified embedding dimension:
25、])
26、training_labels=np.array(training_labels)
27、Totalparams:160,433
28、Layer(type)OutputShapeParam#
29、Model:"sequential_2"
30, overfitting like this is common in NLP models because of a certain unpredictable nature of language. In the next sections, we will explore how some techniques can be used to mitigate this effect.
31 and if we train the model, after 30 training cycles (epochs), we can obtain a training accuracy of more than 99%, but a validation accuracy of only about 81% (Figure 6-2).
32、tf.keras.layers.Embedding(10000,16),
33、tokenizer=Tokenizer(num_words=vocab_size,oov_token=oov_tok)
34、global_average_pooling1d_2((None,16)0
35、
36、model.compile(loss='binary_crossentropy',
37、optimizer='adam',metrics=['accuracy'])