[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    Re: Why not use Serializable?
From:       Doug Cutting <cutting () apache ! org>
Date:       2006-09-27 16:24:42
Message-ID: 451AA5CA.6070208 () apache ! org
[Download RAW message or body]

Feng Jiang wrote:
> In my implementation, I still permit the out-of-order RPC call by the same
> way. the only difference between my impl and your previous impl is:
> 1. I made use of threadpool(JDK1.5) to replace the "Handler" threads. I
> believe the JDK's impl should not be worse than ourselves, and threadpool
> could be grows or shrinks dynamically, even more, I can control the pool
> size, core size, etc.
> 2. Since I just put the connection into the threadpool directly, I removed
> the queue, and that is why i believe my impl should save more memory than
> your previous impl. Because you have only 100 threads, if the 101th client
> sends a request to you, you have to put the connection into the queue, so
> the queue may become bigger and bigger. In my impl, if the 101th client
> comes, the listener will be blocked until the threadpool is available (if i
> limit the threadpool's max size. If not, the listener will never be 
> blocked,
> the each request always has chance to be executed).

To me it seems that ThreadPoolExecutor must still have a thread switch 
per request, so I don't understand why it should be much faster.  And 
the cost of the queue should not be great.  It's intent was to absorb 
small bursts of traffic without increasing the number of worker threads.

If you believe that Java 1.5's ThreadPoolExecutor would improve things 
significantly, please submit a patch to Server.java, changing it to use 
this.  Please attach the patch to a bug in Jira.  Even if it does not 
prove significantly faster, making the Hadoop code smaller could alone 
justify the change.

Thanks,

Doug

[prev in list] [next in list] [prev in thread] [next in thread]