MR M (1) [Avatar] Offline
#1
I'm fairly new to netty and I'm trying to write an upload server that can do the following using netty 4.1.11:

* accepts multipart POST upload requests for files up to 10gb in size
* keep track of all the MixedAttribute from the POST request
* on LastHttpContent, hash the uploaded file, do some business logic, then calls another web service about the upload
* return response to client based on the response from the web service, or error response if anything fails along the way

I've looked at the official netty upload example, and I've tried to implement this by adding a new private variable "myUpload" to keep track of MixedAttribute's, and when I see LastHttpContent, I call static function "processUpload" (which will make a system call to hash my file then calls another web service, and can be CPU intensive or take some time), and finally return the result of the function back to the client.

I understand that this may not be ideal, since the static function is probably blocking. I think I need a pipeline that has the following:

* HttpRequestDecoder
* HttpResponseEncoder
* Some kind of MixedAttribute tracker
* HttpUploadServerHandler
* Some kind of pooled execution handler that takes the final FileUpload and the MixedAttributes, and calls my function


Am I on the right track?
Let's say that I'm running this on a 32 core machine with 128gb of RAM, what should the worker group and execution pool sizes be to maximize throughput?

Thanks so much in advance!

if it is easier to answer in SO, please check out the same question here