528039 (5) [Avatar] Offline
#1
necessary data structure for history <- model %>% fit(...)
book citation
[ 43 KB ]
Hi,-

I am trying to transfer my positive reading experience with the examples from the book to my own application. Unfortunately, the data generators available from Keras are not suitable to my original data. Thus, I have either to design my own data generator satisfying the 'fit_generator-function' or to know the data structure required by the instance history <- model %>% fit(...) i.e. the 'fit-function'. In the last case and according to p128 (see screen-shot enclosed) it should be possible to present the data "as a list of arrays...".

However, nowhere in the book nor in the rather scarce help files of the kerasR-documentation do I find any indication on how this list of arrays should be structured. Moreover, nowhere in the book are examples to be found on how conv-layers as input layers can be directly addressed except with data generators. This makes me suspicious and leads me to my first question:

(1) Can conv-layers as input layers be directly addressed from suitable arrays at all?

Since designing a data generator would require the exactly the same information, plus the additional effort to program the generator, I decided to stick to the 'fit-function' anyway. My data consists of 200 batches of 20 arrays with dimension (50,1,1) each. The input layer of my model looks as follows:

model <- keras_model_sequential() %>%
layer_conv_2d(filters = 32, kernel_size = c(7, 1),
activation = "relu",
input_shape = c(50, 1, 1),
batch_size = c(20)) %>%
layer_max_pooling_2d(pool_size = c(2, 1)) %>%
...

The summary of the model seems to be ok, i.e. no error message appears until and including compilation.

Now, running the history instance, however, produces the following error message. (The numbers given in [[ ]] are the first components of the first few (50,1,1)-arrays within the first batch):

> history <- model %>% fit(
+ x_train,
+ y_train,
+ epochs = 10,
+ steps_per_epoch = 180, (NOTE: same results with 'batch_size = 20' in this line)
+ validation_data = list(x_val, y_val),
+ validation_steps = 20
+ )

Error in py_call_impl(callable, dots$args, dots$keywords) :

ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 1 array(s), but instead got the following list of 180 arrays: [array([[[[ 0.60603744]],
[[ 0.72773063]],
[[ 0.6198473 ]],
[[ 0.62175697]],
[[ 0.67492896]],
[[ 0.63466322]],
[[ 0.71152276]],
[[ 0.7016...

12.
stop(structure(list(message = "ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 1 array(s), but instead got the following list of 180 arrays: [array([[[[ 0.60603744]],\n\n [[ 0.72773063]],\n\n [[ 0.6198473 ]],\n\n [[ 0.62175697]],\n\n [[ 0.67492896]],\n\n [[ 0.63466322]],\n\n [[ 0.71152276]],\n\n [[ 0.7016...",
call = py_call_impl(callable, dots$args, dots$keywords),
cppstack = structure(list(file = "", line = -1L, stack = "C++ stack not available on this system"), .Names = c("file",
"line", "stack"), class = "Rcpp_stack_trace")), .Names = c("message", ...

11.
(structure(function (...)
{
dots <- py_resolve_dots(list(...))
result <- py_call_impl(callable, dots$args, dots$keywords) ...

10.
do.call(object$fit, args)

9.
fit(., x_train, y_train, epochs = 10, steps_per_epoch = 180,
view_metrics = TRUE, validation_data = list(x_val, y_val),
validation_steps = 20)

8.
function_list[[k]](value)

7.
withVisible(function_list[[k]](value))

6.
freduce(value, `_function_list`)

5.
`_fseq`(`_lhs`)

4.
eval(quote(`_fseq`(`_lhs`)), env, env)

3.
eval(quote(`_fseq`(`_lhs`)), env, env)

2.
withVisible(eval(quote(`_fseq`(`_lhs`)), env, env))

1.
model %>% fit(x_train, y_train, epochs = 10, steps_per_epoch = 180,
view_metrics = TRUE, validation_data = list(x_val, y_val),
validation_steps = 20)

This brings me to my question:

(2) Where do I find rules on how to structure my data suitably "as a list of arrays" in order to feed it correctly to the input layer?

Any help pointing towards an answer to these 2 questions would be highly appreciated.

Cheers,
Leo





256385 (46) [Avatar] Offline
#2
necessary data structure for history &lt;- model %&gt;% fit(...)
The `fit()` function generally takes nd arrays (often matrices) rather than lists of arrays (unless it's a multi-input model).

If you could move this question to the GitHub issues section of the keras repo (https://github.com/rstudio/keras/issues) along with an end to end reproducible example we will get to the bottom of it ASAP!
528039 (5) [Avatar] Offline
#3
necessary data structure for history &lt;- model %&gt;% fit(...)
Thanks for your reply and the hint contained in it! See the corresponding (already closed) thread in https://github.com/rstudio/keras/issues/219 .