Hi,

I've written a single-threaded Java NIO server, that I'm now trying to make multi-threaded. However, the only approaches I've seen for doing this, are based on two threads: one for accepting connections and one for reading/writing to channels. I tried using a thread pool that I dispatch selected keys to, but I found this approach failed.

My approach now:
- select keys that are ready for operation
- for each key, dispatch operation on it to the thread pool

This approach fails because the next time I select keys, some keys are returned again, because the thread pool has not finished working on them yet.

Example:
- selector returns a key that's ready for accepting a connection
- the key is handed to the thread pool where a thread is waiting
to operate on it
- the server loop finishes and the selector selects keys again
- the same key is returned, because the thread pool has not accepted
the connection yet

Is there a way to write a Java NIO server with more than two threads? I need this for a very high-performance server.


Thanks in advance,
Ronald Wildenberg.

 

 

Sounds like it should be possible, but first a couple questions.

You didn't define key. Key to what?

Why a thread pool? I'd start with a ThreadGroup and then just "new" Threads in that group as needed. Thread.start() really means "create this Thread, call its run() method, let it run, discard the Thread when run() is finished. Usually this let's Java do all the work you'd think you have to do for yourself.

Anyhow, I came here looking for advice on java.nio, which you might have: any experience with direct mapped buffers? Will these get me from disk to RAM w/o passing through the CPU (Wintel boxes)? (I'm working on high-speed, enhanced b-trees - heavy into keys but, I suspect, not the same keys you're talking about.)

 

 

The purpose of Selectors and SelectionKeys in NIO is to allow an application to be single threaded where before NIO it would have to have been multi threaded. If you are using Selectors, then you should probably be doing all of your Socket IO in the same thread, then if you need to you can pass the data to be processed on a different thread. If you think you need to do all the reading on different threads, then using Selectors probably isn't the best method.

 

 

By key I mean java.nio.channels.SelectionKey. It would be beyond the scope of this discussion to elaborate on this. For some more resources, see the following links:
http://www.onjava.com/pub/a/onjava/2002/09/04/nio.html
http://www.owlmountain.com/tutorials/NonBlockingIo.htm




The main advantage of a thread pool is that it saves
you the creation time of threads in the middle of a
running program. Thread creation is quite expensive and
a thread pool creates threads on startup of my server.
These threads are then waiting for something to do.
Creating a thread each time some work comes in is generally
not a good idea if you can avoid it.

See http://developer.java.sun.com/developer/onlineTraining/Programming/JDCBook/perf.html#pool

for an excellent example of a thread pool implementation.




Do you mean the class MappedByteBuffer here, which maps a
file directly to a ByteBuffer or do you mean a ByteBuffer
as returned by ByteBuffer.allocateDirect(int)? The first
provides a direct mapping from a physical file to a
ByteBuffer, the second provides you with a ByteBuffer that
is directly mapped to native memory (outside the JVM heap).

 

 

Of course NIO allows you to write a single-threaded application, where with IO it would have been multi-threaded. However, if I/O operations are heavy, you do not wish them all to run in a single-threaded application.

However, I did some thinking and I've come up with a way of multi-threading a NIO server. First, there's the way you describe: socket I/O in one thread, any data you read passed to other threads that do the processing.

A second way would be to divide incoming client connections (in the form of SocketChannel's) over different selectors. Each selector then serves a subset of the client population. Each selector is wrapped by a thread that calls its selector.select() method inside some loop.

 

 

I gave up on a 100% multithreaded approach to this
for these reasons.
You have to to a whole bunch of synching.
Selectors should be thread safe, but their keys are not.
Otherwise you will wipe out the other Thread's Iterator
and be rewarded with a ConcurrentModificationException.
Thus:
Synchronize on
The Selector
Then the registered Keys set
Then the selected key set.
I think it should follow that order.
Now that means 3 synchs for every access which
my opinion is way too much.
My solution to this was thus.
Make 2 linked lists. One for current IO, and one
and one for impending IO.
Have ONE thread doing the IO, and the other
adding to to its "ADD" list.
When the IO Thread has time it can synch on the linked
list (One synch) and then add the inpending IO
list to the tail of the current IO list.
This works very well with the one potential caveate
that the Iterator loop may not actually stop if
it is getting a pile of IO. In other words,
the Iterator.hasNext() may continue looping
as long as ther eis IO in process so you may
want to check the impending list from time to time.
The advantage ebing you can still use multiple
threads but 1/3rd the synching.
Good Luck!
Sincerely:
Tony Swain.
<Shameless Plug>
This (And several weeks of debugging)
helped me quite a bit.
http://www.javanio.info/filearea/bookexamples/
</Shameless Plug>

 

 

I co-developed an NIO based server that is currently free and most likely it will
stay this way. It is performing the select on one thread and then queueing the
requests to be processed on worker threads (pooled). This is more
than just using NIO but see if it helps:

http://www.accendia.com

 

 

This smells a bit like SPAM.
- new user
- points to commercial site
- answers very old message