GameDevMike

Normally, before writing to a java.nio.Buffer, you should call its clear() method. In a project I am currently working on, I may not be able to process all of the data in a buffer before I need to read into it again. In those cases, compact() should be called instead of clear() so that the existing data is not lost — the new data will be tacked on at the end of the existing. To implement this simple system, I coded this up:

    public void beginWrite() {
        // only clear the buffer if it has been completely processed, otherwise
        // compact it so that partial packet data is not lost
        if(buffer.hasRemaining()) {
            buffer.compact();
        } else {
            buffer.clear();
        }
    }

So far so good. All I need do is call beginWrite() before writing to the buffer and everything should be kosher. Or so I thought. This is part of a networked app and the buffer is written to by a call to ReadableByteChannel.read(). When I began testing things for the first time, I was surprised to see that 0 bytes were being read from the socket. Selector.select() was returning immediately, indicating that bytes were waiting to be read. They just weren’t.

After a bit of fiddling around and close consultation with the JavaDocs, I realized what the problem was. Take a look at what the documentation for java.nio.Buffer.clear() says:

Clears this buffer. The position is set to zero, the limit is set to the capacity, and the mark is discarded.

Invoke this method before using a sequence of channel-read or put operations to fill this buffer. For example:

    buf.clear();     // Prepare buffer for reading
    in.read(buf);    // Read data

This method does not actually erase the data in the buffer, but it is named as if it did because it will most often be used in situations in which that might as well be the case.

That last sentence (emphasis mine) explains the problem. When a ByteBuffer is first constructed, its internal state is as if the clear() method has been called on it (which the constructor likely does anyway). What is not documented, but must be inferred from the documentation quoted above, is that calling hasRemaining() on a new or cleared buffer will always return true. Now look at the documentation for Buffer.hasRemaining():

Tells whether there are any elements between the current position and the limit.

When you create a new ByteBuffer, it is assumed to have no elements since you haven’t put anything into it yet. Well, the documentation is a bit misleading. The buffers do not internally track the number of elements they contain. Has remaining returns true if and only if the current position is not equal to the limit. Create a new byte buffer with a capacity of 1024 and immediately call its remaining() method. You’d think it would return 0. It actually returns 1024. Check the documentation for that one:

Returns the number of elements between the current position and the limit.

Well, again, a bit misleading. A more appropriate term to use in place of elements would be something like slots. Because what element really means in this case is the number of bytes, ints, or whatever that the buffer can hold between the current position and the limit, not the number that have been read into it.

So what was happening in my code to cause the 0 byte reads was that the first time I called beginWrite(), the call to hasRemaining() was returning true, causing the buffer’s compact() method to be called. Calling compact() on a new/cleared buffer has the same effect as if you read a number of bytes into the buffer before compacting it — the buffer’s current position is set to the limit. So allocating a new buffer with a capacity of 1024 and immediately compacting it sets the position to the limit, which in a new/cleared buffer happens to equal the capacity. See what happened? Because the position was being set to the capacity of the buffer, there was no room to write any new data into it, thereby causing the 0 byte reads.

So to solve the problem, I now compact the buffer immediately after allocating it. When hasRemaining() is called on it the first time, it returns false (as it should) and the buffer is cleared rather than compacted.

Technorati Tags: Java, NIO, buffers, ByteBuffer, networking