This is due to an ugly Java implementation detail around boxed numbers. compareandset! uses identity (Java ==) for the comparison, so a comparison of two boxed numbers which are different objects will fail even if they have the same value. This problem does not show up with smaller numbers because Oracle and OpenJDK will intern (i.e., ensure there is only one shared instance of) small numbers from 127 to 128. This range is both configurable from the command line, and may vary by JVM implementation.
So in summary:
compareandset! takes Objects, so its numeric arguments are autoboxed by the JVM.
If the numbers are small enough, the autoboxing produces the same (identical) object.
compareandset! checks the old value using identity (not equality) comparison. This works as expected for small numbers due to interning.
But once the number gets big enough, the autoboxed number is not interned and you may have two different objects.
Observe:
(def twohundred 200)
=> #'user/twohundred
(def a (atom twohundred))
=> #'user/a
(compareandset! a 200 300)
=> false
(compareandset! a twohundred 300)
=> true
I have added more detail to a note on the ClojureDoc page for compareandset!
