RFC: [Patch] rewrite of classpath/gnu/java/net/protocol/http/*

David Daney ddaney@avtrex.com
Mon Sep 12 21:05:00 GMT 2005


As I threatened last week, I made some major changes to the 
gnu.java.net.protocol.http package.  The patch is probably not in its 
final form, but I thought I would post it to see what the reaction to it 
is.  At a minimum I would update the copyright dates.

The current version reads the entire body of a HTTP response in to a 
memory buffer (ByteArrayResponseBodyReader) and if user code needs the 
content it gets a ByteArrayInputStream wrapped around the buffer.

There are several problems with this approach  The main one is that if 
the response is larger than one half of the heap limit (a copy of the 
data is made so you need space for two copies) You get an 
OutOfMemoryError.  And related is that if the response is larger than 
Integer.MAX_VALUE you run into the java language limit on array sizes.

A secondary problem that I noticed (but did not fully analyze) is that 
the connection pool and keep-alive logic is not thread safe and had some 
other logic bugs in it.

Here is an outline of what I did:

Gratuitously remove RequestListener support.  Perhaps it is used by a 
third party package, I did not bother to check or investigate.  If 
needed, I suppose it could be restored.

Removed ResponseBodyReader and ByteArrayResponseBodyReader.  Since we 
are not going to buffer anything anymore these are no longer needed.

The core to the rewrite is the new LimitedLengthInputStream class.  It 
serves a couple of purposes.  First it controls/protects access to the 
underlying socket's InputStream so that it does not get closed too soon 
or have too much data read from it.  The second thing it does is to help 
manage the connection pool as described below.

When a connection is made, a LimitedLengthInputStream is attached to the 
socket's InputStream ahead of any other encoding filters that might be 
needed.  This prevents actions by the users code from harming the socket 
in any way that would make it unusable for re-use in a keep-alive situation.

When all the data has been been read from the body of the response or if 
the InputStream is closed, then the connection is returned to the 
connection pool.

With this patch I can successfully transfer (via HTTP) content that is 
much larger than my heap.

Some testing done with GCJ (HEAD) on i686-pc-linux-gnu as well as 
mipsel-linux.  More testing is probably needed.

2005-09-12  David Daney  <ddaney@avtrex.com>

	* classpath/gnu/java/net/protocol/http/ByteArrayResponseBodyReader.java:
	Removed.
	* classpath/gnu/java/net/protocol/http/ResponseBodyReader.java:
	Removed.
	* classpath/gnu/java/net/protocol/http/HTTPConnection.java
	(requestListeners): Removed.
	(pool): New field.
	(Constructor): Don't initialize requestListeners.
	(getPoolKey): New method.
	(setPool): New method.
	(release): New method.
	(newRequest): Don't call fireRequestEvent.
	(addRequestListener): Removed.
	(fireRequestEvent): Removed.
	* classpath/gnu/java/net/protocol/http/HTTPURLConnection.java
	(java.util.ByteArrayInputStream): Don't import.
	(connectionPool): Changed type to LinkedHashMap.
	(maxConnections): Made static.
	(responseSink): Changed type to InputStream.
	(errorSink): Likewise.
	(connect): Eliminate reader and get responseSink from response.
	(getConnection): Rewrote.
	* classpath/gnu/java/net/protocol/http/Headers.java (getLongValue): New
	method.
	* classpath/gnu/java/net/protocol/http/LimitedLengthInputStream.java:
	New class.
	* classpath/gnu/java/net/protocol/http/Request.java
	(responseBodyReader): Removed.
	(setResponseBodyReader): Removed.
	(dispatch): Don't call fireRequestEvent.  Handle unsolicited 100
	Continue  response.
	(readResponse): Rewrote.
	(readResponseBody): Rewrote.
	* classpath/gnu/java/net/protocol/http/Response.java
	(java.io.InputStream): Import it.
	(codeClass): Removed.
	(Constructor): Don't initialize codeClass.  Initialize body.
	(body): New field.
	(getCodeClass): Rewrote.
	(getLongHeader): New method.
	(getBody): New nethod.


Comments?

David Daney
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: http.d
URL: <http://gcc.gnu.org/pipermail/java-patches/attachments/20050912/32b6c762/attachment.ksh>


More information about the Java-patches mailing list