The Author Online Book Forums are Moving

The Author Online Book Forums will soon redirect to Manning's liveBook and liveVideo. All book forum content will migrate to liveBook's discussion forum and all video forum content will migrate to liveVideo. Log in to liveBook or liveVideo with your Manning credentials to join the discussion!

Thank you for your engagement in the AoF over the years! We look forward to offering you a more enhanced forum experience.

MR M (1) [Avatar] Offline
#1
I'm fairly new to netty and I'm trying to write an upload server that can do the following using netty 4.1.11:

* accepts multipart POST upload requests for files up to 10gb in size
* keep track of all the MixedAttribute from the POST request
* on LastHttpContent, hash the uploaded file, do some business logic, then calls another web service about the upload
* return response to client based on the response from the web service, or error response if anything fails along the way

I've looked at the official netty upload example, and I've tried to implement this by adding a new private variable "myUpload" to keep track of MixedAttribute's, and when I see LastHttpContent, I call static function "processUpload" (which will make a system call to hash my file then calls another web service, and can be CPU intensive or take some time), and finally return the result of the function back to the client.

I understand that this may not be ideal, since the static function is probably blocking. I think I need a pipeline that has the following:

* HttpRequestDecoder
* HttpResponseEncoder
* Some kind of MixedAttribute tracker
* HttpUploadServerHandler
* Some kind of pooled execution handler that takes the final FileUpload and the MixedAttributes, and calls my function


Am I on the right track?
Let's say that I'm running this on a 32 core machine with 128gb of RAM, what should the worker group and execution pool sizes be to maximize throughput?

Thanks so much in advance!

if it is easier to answer in SO, please check out the same question here