[prev in list] [next in list] [prev in thread] [next in thread] 

List:       jakarta-commons-user
Subject:    [io] BroadcastInputStream if this would be a useful addition
From:       John Crowley <jdcrowley () gmail ! com>
Date:       2020-03-08 14:03:55
Message-ID: D53C231F-D3F5-43A2-B102-9F16FAA4F411 () computer ! org
[Download RAW message or body]


Hi,

Wrote this component to satisfy a local issue, and would contribute to =
commons.io <http://commons.io/> if the community feels it would be =
generally useful.

Initial JavaDoc description is pasted below.

Let me know if you feel that this would be a useful component, not =
useful to a broad enough group of developers, or some other opinion.

Best,

John Crowley
Charlotte, NC
203-856-2396


/** Reads a single InputStream and broadcasts it to 1..N =
BroadcastConsumers. Each BroadcastConsumer will
 *  receive a copy of the entire original InputStream. Note that mark() =
and reset() are not supported.
 *
 *  An efficient alternative to setting up multiple =
PipeOutputStream/PipedInputStream connections, even if
 *  only broadcasting to 1 consumer. Internal buffer space is shared =
among all consumers, and full synchronization
 *  is required only when buffers are switched - instead of =
synchronizing the read and write of each byte.
 *
 *  Normally each of the BroadcastConsumers will execute in a separate =
thread in order to allow parallel
 *  processing of the data. Fast BroadcastConsumers (for example, in =
memory processing) may be throttled
 *  until slower consumers (for example, writing to an external device) =
catch up.
 *
 *  This also implies that if any BroadcastConsumer terminates =
abnormally, without closing itself,
 *  then all other consumers may enter an indefinite wait since the =
shared buffers are never released.
 *
 *  Once instantiated, the run() method of the BroadcastInputStream =
starts reading the original input and
 *  broadcasting to all of the consumers. The BroadcastInputStream may =
also be executed in a separate thread.
 *
 *  Example: Reading a large XML document generated by a sub-process, =
where the XML must be parsed, written to an
 *           output file, and also used to compute a word-frequency =
table.
 *
 *           A BroadcastInputstream with 3 BroadcastConsumers allows =
parallel processes - one to parse the XML,
 *           another to write it to a file, and a third to compute the =
word-frequencies - for minimal overall
 *           elapsed time and buffer space.
 *
 * Alternate constructors may specify the size and number of internal =
buffers.
 *
 * The overall structure is thread-safe, but the read() methods of the =
BroadcastConsumer instances are NOT
 * thread-safe. If processed in parallel, each BroadcastConsumer should =
be processed within a single thread.
 *
 * Note: run() cannot be listed as throwing an exception, so check =
hadError() to determine if any exception
 *       was thrown during processing. If true, use getError to =
retrieve. These methods also exist on each
 *       of the BroadcastConsumer instances to pick up any such =
Exception (except IOException on read()).
 *
 * @Author John Crowley Jan 2020
 */




[prev in list] [next in list] [prev in thread] [next in thread]