The Author Online Book Forums are Moving

The Author Online Book Forums will soon redirect to Manning's liveBook and liveVideo. All book forum content will migrate to liveBook's discussion forum and all video forum content will migrate to liveVideo. Log in to liveBook or liveVideo with your Manning credentials to join the discussion!

Thank you for your engagement in the AoF over the years! We look forward to offering you a more enhanced forum experience.

manderegg (2) [Avatar] Offline
#1
To start, thank you all so much for writing this book. I have found the first couple of chapters to be very enlightening and useful in my attempt to learn and start leveraging Storm.

Regarding the Credit Card Authorization sample presented in chapter 4, it seems to me that there is a possibility for a given tuple to not be successfully processed by all of the bolts in the topology. One scenario as an example:

1. Tuple emitted by the spout.
2. Tuple processed by the VerifyOrderStatus bolt, new tuple emitted, spout tuple marked as processed.
3. Tuple processed by the AuthorizeCreditCard bolt, new tuple emitted, order status tuple marked as processed.
4. Tuple fails to complete processing in the ProcessedOrderNotification bolt (service call fails, worker dies, etc).
5. Message timeout expires, tuple replayed by spout.
6. Tuple examined by VerifyOrderStatus bolt and ignored (order status was successfully updated in step 3), tuple marked as processed.
7. Spout receives ack for tuple.

Is my understanding correct that this progression would mean that the tuple may never be successfully processed by the ProcessedOrderNotification bolt?

I could see as a solution allowing the tuple to continue through the topology and adding individual checks at each bolt to see if that logic needed to be executed or not. However, this seems to be approaching some of what I understand Trident topologies aim to address.

Please let me know if my understanding is off base and I am looking forward to the subsequent chapters of the book (especially the ones on tuning and Trident).

Thanks again,
Matt
sean.allen (19) [Avatar] Offline
#2
Re: Queston on message processing semantics of Ch4 example
Nicely found.

You're correct.

We intentionally introduced that as an issue and we will be referring back to chapter 4's solution when we discuss Trident and a less problematic way to deal with such a scenario.
manderegg (2) [Avatar] Offline
#3
Re: Queston on message processing semantics of Ch4 example
Hi Sean,

Thanks for the quick response, I'm still trying to wrap my head around if service calls to an external entity (the credit card authority) can be implemented with exactly once message processing semantics without some type of distributed transaction.

The best I can come up with is storing a flag in the state that triggers or bypasses execution of the service call. However, as updating the state and making the call would be separate actions one could fail while the other still goes through. Is this a valid assessment?

I understand the semantics work when the processing changes internal state and can be executed atomically with the update to the current txid, but is there any way to make this guarantee when this is not true?

Thanks again,
Matt