jfcha (1) [Avatar] Offline
#1
The book is great so far. Thanks for doing this.

I'm building a streaming stock prediction application and I'm looking to use Kafka Streams. Is there a good way to build a best effort stream with Kafka Streams? My inputs come in relatively fast and my prediction algorithm is relatively slow, so what I want is for the stream to take the latest message, run the calculation, and then, whenever it finishes, take the new latest message from the partition and begin again. For my particular use case, completeness isn't important, but timeliness (minus the lag for the actual calculation) is. In essence, I'd like to do something like a consumer's seekToEnd call at the end of each process call. Any suggestions on how to approach this with streams?

Thanks.