The Author Online Book Forums are Moving

The Author Online Book Forums will soon redirect to Manning's liveBook and liveVideo. All book forum content will migrate to liveBook's discussion forum and all video forum content will migrate to liveVideo. Log in to liveBook or liveVideo with your Manning credentials to join the discussion!

Thank you for your engagement in the AoF over the years! We look forward to offering you a more enhanced forum experience.

264194 (2) [Avatar] Offline
#1
Please take a look at http://stackoverflow.com/questions/36050597/what-is-wrong-with-this-ref-set-up for the details of how I run it in a multithreaded setting.



264194 (2) [Avatar] Offline
#2
The example is not wrong in and of itself, but somewhat misleading (it was at least for me).The :id should actually be the key of the ref, which is what is being changed by the threads. I think the text tries to allude to the significance of the :id a bit in the final note, in that you will see way more clashes of id's if you run this example with commute instead of alter, but it is a bit obscure.
Francis Avila (16) [Avatar] Offline
#3
This example is a toy and I didn't lock down its semantics as thoroughly as I should have. Basically, the problem is that id issuance is directly tied to the size of the map, so non-duplicate ids will only be issued if adding a user always grows the map size. This means deleting users will cause id overlaps for subsequent adds. Adding a new user with a duplicate login (which is conceptually the same as deleting the existing user and adding a new one) is also a problem.

If this never-delete-entries invariant does not hold, the id issuance policy is bad no matter how you key the map and will be bad even without concurrency. refs and dosync will allow you to reason about your code as if it were not concurrent and have it still work when run concurrently, but if the algorithm doesn't work even without concurrency dosync won't help!

In this case, if we disallow user deletes, the only thing we need to do to ensure safety is throw an exception (or keep the old user entry) if the login already exists. If we allow user deletes, we need to visit every entry in the map to find a valid max-id or (better) keep a separate next-user-id ref as a counter.

I'll add some examples here exploring these options when I get a chance.
Francis Avila (16) [Avatar] Offline
#4
I've created a gist with a somewhat realistic use of dosync applied to this create-user example, and an equivalent implementation using an atom.

Honestly, I've been writing Clojure for years and have only once, maybe, found a use for refs and dosync. (But I still used an atom because it was simpler.) Its rare that you have state which is complicated enough to be too much for atoms, yet which also shouldn't be in some kind of database or external state store. Also, there are also other idiomatic alternatives in Clojure which are much less complicated and more natural, such as using a single-writer design (with an agent or a core.async channel) or using parallel map+reduce (with clojure.core.reducers/fold or tesser or claypoole). In the gist I tried to come up with cases where refs would be a better choice than atoms.

Its interesting to me that even with an 8-core machine, the atom implementation in that gist is still about 20% faster than the fine-grained ref implementation (as benchmarked by the ref-test and atom-test functions). Maybe a workload very heavily skewed in favor of updates would beat an atom implementation?
Francis Avila (16) [Avatar] Offline
#5
The original demo for refs (now 5 years old!) was Rich Hickey's Ants simulation, which I have not studied closely. (That link is a very thorough literate-programming explanation of his original code.) I am curious whether this problem is still best solved with refs or whether different approaches would be superior nowadays, either in speed or in code clarity.

An interesting historical note: The last mutable type to be added to Clojure was atoms. When Rich Hickey wrote this demo, atoms did not yet exist, only vars, refs and agents.