Morgatz (10) [Avatar] Offline
In chapter 7 Kinesis streams with lambdas are being used to process order events. I was wondering if using SNS trigger lambda could work as well and what are the tradeoffs to this approach?

So for every order event created, it would be published to SNS and then lambdas subscribed to the SNS would process the event they care about.

For example:

order_placed event created -> SNS <- order_placed listener notifies restaurant and creates restaurant_notified event
restaurant_notified event created -> SNS <- restaurant_notified listener accepts order and creates order_accepted event

and so on...
Yan Cui (71) [Avatar] Offline
In this specific instance, you could use SNS to chain the functions together, but it wouldn't be my choice. You lose the secondary benefit with Kinesis that it acts as a transaction log of state changes on the order flows. You'll likely want to stream to some data warehouse solution such as Athena or BigQuery for BI.

For a business critical workflow like this, I'd have preferred to use Step Functions as I mentioned in the course. The business criticality justifies the significant higher cost for Step Functions. It gives you a lot more control around error handling and you get good debugging tools out-of-the-box, and you can even visualize the workflow itself which is a big bonus! Even in this setup, I'll still publish events from the individual functions, as a way of reporting state changes to a unified log so other subsystems can subscribe to any events that happens along the order flow and do something. And of course, the BI process would simply stream all these events into S3/Google BigQuery, etc.
Morgatz (10) [Avatar] Offline
Thanks, I'm starting that section of the course today. I've never used step functions before, they look pretty neat
Yan Cui (71) [Avatar] Offline
oh you're going to like it smilie