RFC: [Patch] rewrite of classpath/gnu/java/net/protocol/http/*
David Daney
ddaney@avtrex.com
Mon Sep 12 21:05:00 GMT 2005
As I threatened last week, I made some major changes to the
gnu.java.net.protocol.http package. The patch is probably not in its
final form, but I thought I would post it to see what the reaction to it
is. At a minimum I would update the copyright dates.
The current version reads the entire body of a HTTP response in to a
memory buffer (ByteArrayResponseBodyReader) and if user code needs the
content it gets a ByteArrayInputStream wrapped around the buffer.
There are several problems with this approach The main one is that if
the response is larger than one half of the heap limit (a copy of the
data is made so you need space for two copies) You get an
OutOfMemoryError. And related is that if the response is larger than
Integer.MAX_VALUE you run into the java language limit on array sizes.
A secondary problem that I noticed (but did not fully analyze) is that
the connection pool and keep-alive logic is not thread safe and had some
other logic bugs in it.
Here is an outline of what I did:
Gratuitously remove RequestListener support. Perhaps it is used by a
third party package, I did not bother to check or investigate. If
needed, I suppose it could be restored.
Removed ResponseBodyReader and ByteArrayResponseBodyReader. Since we
are not going to buffer anything anymore these are no longer needed.
The core to the rewrite is the new LimitedLengthInputStream class. It
serves a couple of purposes. First it controls/protects access to the
underlying socket's InputStream so that it does not get closed too soon
or have too much data read from it. The second thing it does is to help
manage the connection pool as described below.
When a connection is made, a LimitedLengthInputStream is attached to the
socket's InputStream ahead of any other encoding filters that might be
needed. This prevents actions by the users code from harming the socket
in any way that would make it unusable for re-use in a keep-alive situation.
When all the data has been been read from the body of the response or if
the InputStream is closed, then the connection is returned to the
connection pool.
With this patch I can successfully transfer (via HTTP) content that is
much larger than my heap.
Some testing done with GCJ (HEAD) on i686-pc-linux-gnu as well as
mipsel-linux. More testing is probably needed.
2005-09-12 David Daney <ddaney@avtrex.com>
* classpath/gnu/java/net/protocol/http/ByteArrayResponseBodyReader.java:
Removed.
* classpath/gnu/java/net/protocol/http/ResponseBodyReader.java:
Removed.
* classpath/gnu/java/net/protocol/http/HTTPConnection.java
(requestListeners): Removed.
(pool): New field.
(Constructor): Don't initialize requestListeners.
(getPoolKey): New method.
(setPool): New method.
(release): New method.
(newRequest): Don't call fireRequestEvent.
(addRequestListener): Removed.
(fireRequestEvent): Removed.
* classpath/gnu/java/net/protocol/http/HTTPURLConnection.java
(java.util.ByteArrayInputStream): Don't import.
(connectionPool): Changed type to LinkedHashMap.
(maxConnections): Made static.
(responseSink): Changed type to InputStream.
(errorSink): Likewise.
(connect): Eliminate reader and get responseSink from response.
(getConnection): Rewrote.
* classpath/gnu/java/net/protocol/http/Headers.java (getLongValue): New
method.
* classpath/gnu/java/net/protocol/http/LimitedLengthInputStream.java:
New class.
* classpath/gnu/java/net/protocol/http/Request.java
(responseBodyReader): Removed.
(setResponseBodyReader): Removed.
(dispatch): Don't call fireRequestEvent. Handle unsolicited 100
Continue response.
(readResponse): Rewrote.
(readResponseBody): Rewrote.
* classpath/gnu/java/net/protocol/http/Response.java
(java.io.InputStream): Import it.
(codeClass): Removed.
(Constructor): Don't initialize codeClass. Initialize body.
(body): New field.
(getCodeClass): Rewrote.
(getLongHeader): New method.
(getBody): New nethod.
Comments?
David Daney
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: http.d
URL: <http://gcc.gnu.org/pipermail/java-patches/attachments/20050912/32b6c762/attachment.ksh>
More information about the Java-patches
mailing list