While waiting for the publication of the rest of the book, I personally enjoy "Deep Reinforcement Learning Hands-On: Apply modern RL methods, with deep Q-networks, value iteration, policy gradients, TRPO, AlphaGo Zero and more" - June 21 , 2018 by Maxim Lapan, despite fact it was published by Packt smilie It’s cover almost of all the topics promised by authors of this book.

I also recommend RL Course by David Silver (on youtube: https://www.youtube.com/playlist?reload=9&list=PLzuuYNsE1EZAXYR4FJ75jcJseBmo4KQ9-) and more advanced course CS294-112 at UC Berkeley (http://rail.eecs.berkeley.edu/deeprlcourse/resources/#prevoffs) (you also can find all lectures on youtube: https://www.youtube.com/playlist?list=PLkFD6_40KJIznC9CDbVTjAF2oyt8_VAe3)
Brandon B wrote:...I may not be understanding your question entirely...

Alex also wants to mention reinforce.io < https://github.com/reinforceio/tensorforce >, an "off-the-shelf" Deep RL library in case you were also looking for an easier way to employ RL than writing algorithms with raw PyTorch.


Thank you so much for your reply! That's exactly what I wanted to know.

And thanx to Alex for link to reinforce.io - never heard about it before!
In field of image classification or NLP, deep learning architecture more less established for now, I mean conv nets or recurrent networks works fine. But concerning deep reinforcement learning - what kind of architectures is applicable and what is the best practice?
I was play around with code from chapter 3 and now have a question. You reward for every move -1 (also it indicate of game still in progress), -10 for loss, +10 for win a game.
But, what if we are will try to simulate game where every move produce some reward (positive or negative) and in the same time we have overall total score for whole game. It may be some custom metric, which can be different then just sum of every move rewards. How I can apply algorithm from chapter 3 for this scenario?
Great book, guys! I am very appreciate to authors.

I was little bit familiar with deep learning before and was curious about deep reinforcement learning for while, but yet not found end-to-end manual not overload with math or too simplistic and practical the other hand. Your book is just what I was looking for.

Can’t wait for future chapters!
Jena weather dataset was introduced in 6.3.1 “A temperature-forecasting problem”, but it contains only numerical features. But many real-life times series forecasting problems have both numerical and categorical features. I suppose it’s no big deal for experienced R user prepare features in right way to NN, but for beginner R users such recipe in this book may be very useful.
In part 6.3 “Advanced use of recurrent neural networks” through all demo code “steps_per_epoch = 500”, but why it fixed? Should it be something like: train_steps <- (200000 - lookback) / batch_size ?
I suppose you keep to Chekhov's gun principle in coding smilie

"If in the first act you have hung a pistol on the wall, then in the following one it should be fired. Otherwise don't put it there."
Listing 6.34 Preparing the training, validation, and test generators (the same source code at Listing 6.48 Preparing higher-resolution data generators for the Jena dataset).

Three generator function introduced: train_gen(), val_gen(), test_gen(), and all three based on generator() function.
But test_gen() is never used and as I suppose must be used in predict_generator(), but in this case, error will be raised, because our test_gen() will be return both features data and labels, and predict_generator() needs only features data.
I understand, what one book can’t fit all deep learning wisdom.

I saw NOTE about it on page 213, but I will be appreciate for some tiny sequence masking demo. I google a little, but still not found clean explanation and best practice for this technique.

Also, what about Stateful LSTM? No word about it in the book.
In the text on page 236 (RESIDUAL CONNECTIONS) reference to figure 7.9 exist, but it is missing. Also Footnote 20 is empty.
I have some computer science background and I tried don’t miss any available practical piece of information about Deep Learning.

I’m defiantly slow learner, but all books / MOOC’s I saw before, from my point of view suffer of imbalance – to much math or too much code, without proper explanation (‘recipe’ books), too simple or too complicated, etc.

But this book is real masterpiece!

Bravo!
Oh boy, it’s my fault, I was miss remark at section 3-5. I jumped to RNN section, straight after finished introduction.
In Listing 6.35 “Computing the common-sense baseline MAE” multiple assignment operator %<-% used, but library(keras) introduced only in Listing 6.37 “Training and evaluating a densely connected model”, so if you start from clean R environment, as I did, you will be got error, beside lack in documentation about this nice keras library feature.
I wonder, is any chance to know from R environment, witch Keras/Tensorflow version installed on my system, and if newer version is available how to update to it properly?
In section 6.3 “Advanced use of recurrent neural networks” page 198 - 6.3.2 “Preparing the data” lookback period defined as follow:
“ lookback = 720—Observations will go back 5 days.”
but on page 201 «Listing 6.34 Preparing the training, validation, and test generators» the first line of code is:
lookback <- 1440