The Author Online Book Forums are Moving

The Author Online Book Forums will soon redirect to Manning's liveBook and liveVideo. All book forum content will migrate to liveBook's discussion forum and all video forum content will migrate to liveVideo. Log in to liveBook or liveVideo with your Manning credentials to join the discussion!

Thank you for your engagement in the AoF over the years! We look forward to offering you a more enhanced forum experience.

tempusfugit (144) [Avatar] Offline
Over the decades that I've been using books to continue my education I've had to adjust my expectations of what I can hope to get out of reading/working through a technical book. This is especially true in light of how in more recent years the breakneck pace of some technologies is rendering the associated books obsolete even as they are being written.

In the past I would have usually looked for a book that would give a fairly comprehensive coverage of the topic I chose to investigate. These days I typically rate my experience as positive if one key criterion is met:

  • After working through the book can I make better sense of the associated online documentation?

  • In my opinion section 4.3.1 and 4.3.2 fall short of that objective because I would imagine that some readers would still not be able to make head nor tails out some parts of the Json.Decode page even after reading section 4.3.1 and 4.3.2.

    End of July (2016) I was banging my head against the "Json.Decode" wall (I don't recall intro to JSON decoders being around then, at least not in that particular form). After a significant amount of frustration my first real insight was that I was pursuing the wrong question when I was looking at "Json.Decode". I initially was asking "how do I get this data out of this JSON object" but that question invariably landed me in places where too many other unknowns were being referenced to even form a seed of understanding. That changed when I finally asked:

  • What is a Decoder?

  • To anyone with a heavy procedural, object-oriented, and imperative background, sum types can look stange enough (as referenced on page 97(101))
    type Maybe a
        = Just a
        | Nothing

    type Result error value
        = Ok value
        | Err error

    But they are beaming beacons of transparency when compared to:

    type Decoder a

    ... say what?

    When working with functions there is the expectation of creating a function that returns a value based on some internal logic that processes some specific input parameters. But "Decoder" is a type, not a function! So apparently here we aren't using functions to decode stuff.

    Okaaay ...

    In object-oriented languages you typically have to adhere to some kind of interface or protocol when implementing decoders. But the only thing we have here is the name of the type - "Decoder" - and a single type variable.

    Ok, I would guess that the type variable represents the type being decoded to - but what do I know? Why is this type being so darn introverted anyway?

    Ok, lets look at the description:
    A value that knows how to decode JSON values.

    How can a value do anything? It simply is a value, it may represent something, but since when does a value do anything? And how does it know what I want decoded and how to decode it - I haven't even had a chance to "talk" to it yet. Does it claim to be telepathic or ominiscient? This thing better stop being so damn mysterious - there's work that needs to be done!

    Does anyone know how to talk to a Decode a???
    decodeString : Decoder a -> String -> Result String a
    decodeValue : Decoder a -> Value -> Result String a
    Oh - hi guys!

    I already understand Result String a and String so lets have a closer look at decodeString.
    Parse the given string into a JSON value and then run the Decoder on it. This will fail if the string is not well-formed JSON or if the Decoder fails for some reason.
    Ok - so whatever I want decoded, I hand to you (together with the Decoder) and I will either get the decoded value or an error back. I guess that is progress on the "what I want decoded" front. But for some reason you two need to get together in a backroom, do whatever you do, before coming back to me with a Result. You guys are taking this whole "secret society" stuff pretty far and I'm still pretty clueless about the "how to decode it" part. I guess it's time to take another look around.
    string : Decoder String
    bool : Decoder Bool
    int : Decoder Int
    float : Decoder Float
    Hello there! Thanks for offering to help and I'll give you a shout when the time comes - but right now I have to decode something way more complicated.
    succeed : a -> Decoder a
    fail : String -> Decoder a

    Wait. Who might you be? And why do you even need to exist?
    Ignore the JSON and produce a certain Elm value.
    decodeString (succeed 42) "true"    == Ok 42
    This is handy when used with oneOf or andThen.
    andThen : (a -> Decoder b) -> Decoder a -> Decoder b
  • Takes a function with a type a parameter which returns a Decoder to type b
  • Takes a Decoder to type a
  • Produces a Decoder to type b

  • Wait a minute - does this transform one decoder into another decoder? So basically if I provide a decoder of type a and a function the produces a decoder of type b in response to a value of a then this will give me a Decoder of type b.

    But my fundamental problem is that I don't know how to code/configure a decoder - much less a function that returns one!

    Or do I?
    succeed : a -> Decoder a

    That's a special case of (a -> Decoder b) where b = a.

    fail : String -> Decoder b

    Another special case of (a -> Decoder b) where a = String.

    Hmm ...
    string : Decoder String
    bool : Decoder Bool
    int : Decoder Int
    float : Decoder Float
    Oh, it's you guys again - are you trying to tell me something?
    $ elm-repl
    ---- elm-repl 0.18.0 -----------------------------------------------------------
     :help for help, :exit to exit, more at <>
    > import Json.Decode as D
    > D.decodeString "-1"
    Ok -1 : Result.Result String Int
    > D.decodeString D.string "\"-1\""
    Ok "-1" : Result.Result String String
    > D.decodeString D.string "\"0\""
    Ok "0" : Result.Result String String
    > intToBoolDecoder = \n -> \
    |     case compare n 0 of \
    |         LT -> D.succeed False \
    |         _ -> D.succeed True
        : number -> Json.Decode.Decoder Bool
    > myDecoder = D.andThen intToBoolDecoder
    <decoder> : Json.Decode.Decoder Bool
    > D.decodeString myDecoder "1"
    Ok True : Result.Result String Bool
    > D.decodeString myDecoder "-1"
    Ok False : Result.Result String Bool
    > D.decodeString myDecoder "0"
    Ok True : Result.Result String Bool
    > D.decodeString myDecoder "True"
    Err "Given an invalid JSON: Unexpected token T in JSON at position 0"
        : Result.Result String Bool
    -- repositioning "andThen" in the code with "forward function application"
    > myDecoder = |> D.andThen intToBoolDecoder
    <decoder> : Json.Decode.Decoder Bool
    > D.decodeString myDecoder "1"
    Ok True : Result.Result String Bool
    > D.decodeString myDecoder "-1"
    Ok False : Result.Result String Bool
    > D.decodeString myDecoder "0"
    Ok True : Result.Result String Bool
    So I basically just succeeded in creating a decoder without actually "coding" a decoder from the ground up. Instead I "composed" the decoder and the ones returned by Json.Decode.succeed with the help of Json.Decode.andThen. The resulting decoder generates "False" for integers "n" where "n < 0", "True" for all other integers while failing any other type of value.

    Are there other ways of composing these stock decoders?
    field : String -> Decoder a -> Decoder a
  • Takes a String
  • Takes a Decoder to type a
  • Produces a Decoder to type a

  • Same kind of pattern - give it a decoder and you get a decoder.
    Decode a JSON object, requiring a particular field.
    So that String is the name of the field that holds the value that I'm trying to get at.
    Ok, knowing what I know now, these examples are pretty straightforward.
    > D.decodeString (D.field "x" "{ \"x\": 3, \"y\": 4 }"
    Ok 3 : Result.Result String Int
    But I want to get at multiple values at the same time ...
    Check out map2 to see how to decode multiple fields!
    Thanks for the tip!
        :  (a -> b -> value)
        -> Decoder a
        -> Decoder b
        -> Decoder value
  • Takes a function. The function takes two values of different types and produces a value of a third type
  • Takes a Decoder to type a - the first type
  • Takes a Decoder to type b - the second type
  • Produces a Decoder to type value - the third type

  • Hmm ... going by the example (a -> b -> value) is typically a constructor for value.
    > myDecoder = D.map2 (,) (D.field "name" D.string) (D.field "age"
    <decoder> : Json.Decode.Decoder ( String, Int )
    > D.decodeString myDecoder "{ \"name\": \"Felix\", \"age\": 25 }"
    Ok ("Felix",25) : Result.Result String ( String, Int )
    Note: (,) is a tuple constructor.

    Ok - to recap:
    type Decoder a
    is Elm's way of saying "none of your business" and "keep your grubby, filthy paws and mitts out of the insides of my decoders" - but I am also getting all the facilities necessary to compose Elm's decoders into the new decoders necessary to access data in arbitrarily complex data structures.

    Some of this seems a bit clandestine - but whatever works...

    The above is basically a reflection of my initial "Json.Decode" experience that ultimately lead to the exploratory code from my earlier post. That experience forms the context of the comments below.

    Page 107(111):
    4.3.1 Decoding JSON Strings into Results >> The DecodeString function
    I guess it's Json.Decode.decodeValue's turn when the book gets to ports.

    Page 108(112):
    4.3.1 Decoding JSON Strings into Results >> Decoding primitives
    I realize that bool, int, float, string are the fundamental decoders but I think sidelining the succeed and fail functions may be a mistake. They may just be "strange" enough to provoke a beneficial shift in the reader's mindset from "writing decoding logic" to "composing decoders" - the difference may seem subtle but in my mind is an important one.

    Page 109(113):
    4.3.2 Decoding JSON Collections >> Decoding Json Arrays into Lists
    list : Decoder a -> Decode (List a)

    Basically the flow here seems similar to JSON · An Introduction to Elm - Combining Decoders. However in either case I'm not convinced that this approach has the necessary impact. It's all too easy for the reader to simply start memorizing visual coding patterns rather than conceptually grasping that they are composing decoders.

    4.3.2 Decoding JSON Collections >> Decoding Objects
    When this decoder runs, it performs three checks:
    1. Are we decoding an Object?
    2. If so, does that Object have a field called email?
    3. If so, is the Object’s email field a String?
    How do I put this? Yes, I did notice that it starts out with "When this decoder runs". But this (accurate) description focuses on doing things - which feels imperative. While our objective is to "decode stuff" ultimately the "doing" is up to Json.Decode.decodeString, Json.Decode.decodeValue and the decoders - all we as programmer's have control over is setup, configuration, and composition which in this context is mostly declarative.

    This is what actually goes through my mind when I write code like this:
    field : String -> Decoder a -> Decoder a
  • String - I give it a the name of the field to access
  • Decoder a - I give it the decoder for the value of the field
  • Decode a - I get a decoder that knows how to extract the value stored in the object under the specified name

  • I focus on composing the decoder that I want - not running the decoding process.

    Page 110(114):
    Table 4.4. Decoding objects with field decoders
    I have to wonder whether an elm-repl session would be more effective than this table (similarly with "Table 4.5. Decoding objects with field decoders"). It isn't that there is anything wrong with the table, especially as the heading clearly says "Decoder".

    My concern is that the visual layout may be read as:
  • field "email" string decodes {"email": ""} to Ok ""
  • rather than
  • field "email" string creates a Decoder that will decode {"email": ""} to Ok ""

  • Table 4.5. Decoding objects with field decoders
    Consider using the tuple constructor (with a footnote) instead of the anonymous function in order to drive home the fact that typically a constructor is specified as the first parameter of a Json.Decode.mapX function:
      (field "x" int)
      (field "y" int)

    My general advice for Chapter 4 is to just use the map3 code (bottom of page 110(114) - and with the Photo constructor rather than the anonymous function) in the project with maybe a paragraph of explanation and the promise to return to the topic of JSON decoding in a dedicated chapter that shows in detail how to compose decoders in a separate workbench style project (that would end up looking something like the code I posted here).

    Page 110(114):
    4.3.2 Decoding JSON Collections >> Pipeline Decoding
    Consider skipping this section. I'm not trying the denegrate the accomplishments of the contributors but I have to admit that for me personally this yanks my "taking on needless dependencies" chain.


    Now granted because of this I haven't looked too deeply into the capabilties of the package - but on the surface it comes across as something that is most optimal in an environment where you also control the backend and you tend to keep your structures relatively flat. My experience with third party APIs is that one often has to wrestle data from atrociously nested and convoluted structures - and I feel that under these circumstances it is essential that one clearly understands how to compose decoders.

    I'm not blind to the maintenance benefit that NoRedInk/elm-decode-pipeline can provide considering some of the notational compactness and that one doesn't have to constantly change the function name whenever the number of decoders change. But in the context of learning Elm I think it is a mistake to immediately run for the convenient way of doing things. That could essentially deprive readers of the opportunity to really learn and understand "how to compose decoders" which is worthwhile, especially if the reader is making a genuine effort to adopt "Thinking in Elm" - it simply is a necessary part of making the unfamiliar more familiar.