You give general explanations of akka guarantees: least once, at most once and exactly once in chapter 6.5.2. Then you go into examples of the first two, but leave out exactly once. Please show how to implement that. Also you say that unless the business need absolutely requires it, try to use at least once. Please explain why at least once is good enough for most cases and then give examples of real world use cases for each type of guarantee. Like an inventory system where I want to make sure I have enough quantity on hand and not over sell, if at least once is ok, why is it ok and what fail-safes should I use.

In chapter 8.5.6 you start to discuss retry patterns and lightly go over the akka guarantees again. How about just referring to ch 6.5.2 instead here. Maybe you could elaborate a little more on utilizing the durable message systems like kafka and Rabbit MQ. Why would they be a good choice over what akka includes?

Hey there,

I'm no expert on this but exactly-once processing is very specific to your domain so it's a bit difficult to just talk about this generally like the other two. My team is in the process of trying to build a view/query in a CQRS application that is used for billing so we require this strong exactly-once/effectively-once guarantee. If you are using Akka Persistence and making the Akka Persistence Query's eventsByTag feed off the journal then you can achieve exactly-once delivery by using an offset-tracking table in addition to your read side table. You would need to atomically persist the read-side update and the offset (that each event has) from the event journal in order to achieve exactly-once delivery.

Like I said, this approach is very specific to the implementation and the use case. If you want a bit more information, I would be happy to dive into more detail about how we are doing it. I would also advise you to check out Lagom's source code because they do a lot of this for you.

The tricky part comes when you want to introduce something like Kafka if you have a lot of query sides and you are trying to build an event-driven backbone because it's very hard to achieve exactly-once with Kafka in the mix (easy to get at-least once). The recommendation is to take each event (or at least a subset) and migrate them out into the query side databases so if you do have duplicates, they will result in upserts and no harm will be done (effectively once behavior). You will have to perform further aggregation of data on the query side.

Thanks
Cal

Like an inventory system where I want to make sure I have enough quantity on hand and not over sell, if at least once is ok, why is it ok and what fail-safes should I use.

Can you elaborate a bit more about what your command and query sides are for this use case. It seems to me like the command-side is the inventory system itself and the query-sides would be used to do some analytics.

385429 wrote:You give general explanations of akka guarantees: least once, at most once and exactly once in chapter 6.5.2. Then you go into examples of the first two, but leave out exactly once. Please show how to implement that. Also you say that unless the business need absolutely requires it, try to use at least once. Please explain why at least once is good enough for most cases and then give examples of real world use cases for each type of guarantee. Like an inventory system where I want to make sure I have enough quantity on hand and not over sell, if at least once is ok, why is it ok and what fail-safes should I use.

There is no description of how to implement exactly once because there is no general way to do it. Delivering a message exactly once would be delivering it at least once with the additional guarantee that it's not more than once. Chapter 6.5.1 explains why at least once isn't a true guarantee either. In practice, it's really best efforts based on a maximum number of retries and a time bounds.

What to do about this reality depends upon your use case. For an inventory system, consider assigning a unique identifier to the request. You then can use a pair of messages that contain the identifier: a message to reserve an item from inventory, and a confirmation back to the client that the item is reserved for that request. The messages can be sent with at least once semantics. The at-most-once aspect (identifying duplicates) is handled quite differently on each end: On the inventory end, you may be relying on the persistence provided by a database, whereas on the client end you are relying on a UI to match the request and reservation.