The Author Online Book Forums are Moving

The Author Online Book Forums will soon redirect to Manning's liveBook and liveVideo. All book forum content will migrate to liveBook's discussion forum and all video forum content will migrate to liveVideo. Log in to liveBook or liveVideo with your Manning credentials to join the discussion!

Thank you for your engagement in the AoF over the years! We look forward to offering you a more enhanced forum experience.

580532 (3) [Avatar] Offline
#1
Hi

In section 6.1.3, a word index is created on the train data of the imdb dataset.
The word_index is created with the function text_tokenizer(num_words = max_words) and max_words = 10,000.

The resulting word index contains 88,524 words... Why not just 10,000 max. ?