# Stream Stability: 2 - Stable A stream is an abstract interface for working with streaming data in Node.js. The `stream` module provides a base API that makes it easy to build objects that implement the stream interface. There are many stream objects provided by Node.js. For instance, a [request to an HTTP server][http-incoming-message] and [`process.stdout`][] are both stream instances. Streams can be readable, writable, or both. All streams are instances of [`EventEmitter`][]. The `stream` module can be accessed using: ```js const stream = require('stream'); ``` While it is important for all Node.js users to understand how streams works, the `stream` module itself is most useful for developer's that are creating new types of stream instances. Developer's who are primarily *consuming* stream objects will rarely (if ever) have need to use the `stream` module directly. ## Organization of this document This document is divided into two primary sections and third section for additional notes. The first section explains the elements of the stream API that are required to *use* streams within an application. The second section explains the elements of the API that are required to *implement* new types of streams. ## Types of Streams There are four fundamental stream types within Node.js: * [Readable][] - streams from which data can be read (for example [`fs.createReadStream()`][]). * [Writable][] - streams to which data can be written (for example [`fs.createWriteStream()`][]). * [Duplex][] - streams that are both Readable and Writable (for example [`net.Socket`][]). * [Transform][] - Duplex streams that can modify or transform the data as it is written and read (for example [`zlib.createDeflate()`][]). ### Object Mode All streams created by Node.js APIs operate exclusively on strings and `Buffer` objects. It is possible, however, for stream implementations to work with other types of JavaScript values (with the exception of `null` which serves a special purpose within streams). Such streams are considered to operate in "object mode". Stream instances are switched into object mode using the `objectMode` option when the stream is created. Attempting to switch an existing stream into object mode is not safe. ### Buffering Both [Writable][] and [Readable][] streams will store data in an internal buffer that can be retrieved using `writable._writableState.getBuffer()` or `readable._readableState.buffer`, respectively. The amount of data potentially buffered depends on the `highWaterMark` option passed into the streams constructor. For normal streams, the `highWaterMark` option specifies a total number of bytes. For streams operating in object mode, the `highWaterMark` specifies a total number of objects. Data is buffered in Readable streams when the implementation calls [`stream.push(chunk)`][stream-push]. If the consumer of the Stream does not call [`stream.read()`][stream-read], the data will sit in the internal queue until it is consumed. Once the total size of the internal read buffer reaches the threshold specified by `highWaterMark`, the stream will temporarily stop reading data from the underlying resource until the data currently buffered can be consumed (that is, the stream will stop calling the internal `readable._read()` method that is used to fill the read buffer). Data is buffered in Writable streams when the [`writable.write(chunk)`][stream-write] method is called repeatedly. While the total size of the internal write buffer is below the threshold set by `highWaterMark`, calls to `writable.write()` will return `true`. Once the the size of the internal buffer reaches or exceeds the `highWaterMark`, `false` will be returned. A key goal of the `stream` API, and in particular the [`stream.pipe()`] method, is to limit the buffering of data to acceptable levels such that sources and destinations of differing speeds will not overwhelm the available memory. Because [Duplex][] and [Transform][] streams are both Readable and Writable, each maintain *two* separate internal buffers used for reading and writing, allowing each side to operate independently of the other while maintaining an appropriate and efficient flow of data. For example, [`net.Socket`][] instances are [Duplex][] streams whose Readable side allows consumption of data received *from* the socket and whose Writable side allows writing data *to* the socket. Because data may be written to the socket at a faster or slower rate than data is received, it is important each side operate (and buffer) independently of the other. ## API for Stream Consumers Almost all Node.js applications, no matter how simple, use streams in some manner. The following is an example of using streams in a Node.js application that implements an HTTP server: ```js const http = require('http'); const server = http.createServer( (req, res) => { // req is an http.IncomingMessage, which is a Readable Stream // res is an http.ServerResponse, which is a Writable Stream let body = ''; // Get the data as utf8 strings. // If an encoding is not set, Buffer objects will be received. req.setEncoding('utf8'); // Readable streams emit 'data' events once a listener is added req.on('data', (chunk) => { body += chunk; }); // the end event indicates that the entire body has been received req.on('end', () => { try { const data = JSON.parse(body); } catch (er) { // uh oh! bad json! res.statusCode = 400; return res.end(`error: ${er.message}`); } // write back something interesting to the user: res.write(typeof data); res.end(); }); }); server.listen(1337); // $ curl localhost:1337 -d '{}' // object // $ curl localhost:1337 -d '"foo"' // string // $ curl localhost:1337 -d 'not json' // error: Unexpected token o ``` [Writable][] streams (such as `res` in the example) expose methods such as `write()` and `end()` that are used to write data onto the stream. [Readable][] streams use the [`EventEmitter`][] API for notifying application code when data is available to be read off the stream. That available data can be read from the stream in multiple ways. Both [Writable][] and [Readable][] streams use the [`EventEmitter`][] API in various ways to communicate the current state of the stream. [Duplex][] and [Transform][] streams are both [Writable][] and [Readable][]. Applications that are either writing data to or consuming data from a stream are not required to implement the stream interfaces directly and will generally have no reason to call `require('stream')`. Developers wishing to implement new types of streams should refer to the section [API for Stream Implementers][]. ### Writable Streams Writable streams are an abstraction for a *destination* to which data is written. Examples of [Writable][] streams include: * [HTTP requests, on the client][] * [HTTP responses, on the server][] * [fs write streams][] * [zlib streams][zlib] * [crypto streams][crypto] * [TCP sockets][] * [child process stdin][] * [`process.stdout`][], [`process.stderr`][] *Note*: Some of these examples are actually [Duplex][] streams that implement the [Writable][] interface. All [Writable][] streams implement the interface defined by the `stream.Writable` class. While specific instances of [Writable][] streams may differ in various ways, all Writable streams follow the same fundamental usage pattern as illustrated in the example below: ```js const myStream = getWritableStreamSomehow(); myStream.write('some data'); myStream.write('some more data'); myStream.end('done writing data'); ``` #### Class: stream.Writable ##### Event: 'close' The `'close'` event is emitted when the stream and any of its underlying resources (a file descriptor, for example) have been closed. The event indicates that no more events will be emitted, and no further computation will occur. Not all Writable streams will emit the `'close'` event. ##### Event: 'drain' If a call to [`stream.write(chunk)`][stream-write] returns `false`, the `'drain'` event will be emitted when it is appropriate to resume writing data to the stream. ```js // Write the data to the supplied writable stream one million times. // Be attentive to back-pressure. function writeOneMillionTimes(writer, data, encoding, callback) { let i = 1000000; write(); function write() { var ok = true; do { i--; if (i === 0) { // last time! writer.write(data, encoding, callback); } else { // see if we should continue, or wait // don't pass the callback, because we're not done yet. ok = writer.write(data, encoding); } } while (i > 0 && ok); if (i > 0) { // had to stop early! // write some more once it drains writer.once('drain', write); } } } ``` ##### Event: 'error' * {Error} The `'error'` event is emitted if an error occurred while writing or piping data. The listener callback is passed a single `Error` argument when called. *Note*: The stream is not closed when the `'error'` event is emitted. ##### Event: 'finish' The `'finish'` event is emitted after the [`stream.end()`][stream-end] method has been called, and all data has been flushed to the underlying system. ```js const writer = getWritableStreamSomehow(); for (var i = 0; i < 100; i ++) { writer.write('hello, #${i}!\n'); } writer.end('This is the end\n'); writer.on('finish', () => { console.error('All writes are now complete.'); }); ``` ##### Event: 'pipe' * `src` {stream.Readable} source stream that is piping to this writable The `'pipe'` event is emitted when the [`stream.pipe()`][] method is called on a readable stream, adding this writable to its set of destinations. ```js const writer = getWritableStreamSomehow(); const reader = getReadableStreamSomehow(); writer.on('pipe', (src) => { console.error('something is piping into the writer'); assert.equal(src, reader); }); reader.pipe(writer); ``` ##### Event: 'unpipe' * `src` {[Readable][] Stream} The source stream that [unpiped][`stream.unpipe()`] this writable The `'unpipe'` event is emitted when the [`stream.unpipe()`][] method is called on a [Readable][] stream, removing this [Writable][] from its set of destinations. ```js const writer = getWritableStreamSomehow(); const reader = getReadableStreamSomehow(); writer.on('unpipe', (src) => { console.error('Something has stopped piping into the writer.'); assert.equal(src, reader); }); reader.pipe(writer); reader.unpipe(writer); ``` ##### writable.cork() The `writable.cork()` method forces all written data to be buffered in memory. The buffered data will be flushed when either the [`stream.uncork()`][] or [`stream.end()`][stream-end] methods are called. The primary intent of `writable.cork()` is to avoid a situation where writing many small chunks of data to a stream do not cause an backup in the internal buffer that would have an adverse impact on performance. In such situations, implementations that implement the `writable._writev()` method can perform buffered writes in a more optimized manner. ##### writable.end([chunk][, encoding][, callback]) * `chunk` {String|Buffer|any} Optional data to write. For streams not operating in object mode, `chunk` must be a string or a `Buffer`. For object mode streams, `chunk` may be any JavaScript value other than `null`. * `encoding` {String} The encoding, if `chunk` is a String * `callback` {Function} Optional callback for when the stream is finished Calling the `writable.end()` method signals that no more data will be written to the [Writable][]. The optional `chunk` and `encoding` arguments allow one final additional chunk of data to be written immediately before closing the stream. If provided, the optional `callback` function is attached as a listener for the [`'finish'`][] event. Calling the [`stream.write()`][stream-write] method after calling [`stream.end()`][stream-end] will raise an error. ```js // write 'hello, ' and then end with 'world!' const file = fs.createWriteStream('example.txt'); file.write('hello, '); file.end('world!'); // writing more now is not allowed! ``` ##### writable.setDefaultEncoding(encoding) * `encoding` {String} The new default encoding * Return: `this` The `writable.setDefaultEncoding()` method sets the default `encoding` for a [Writable][] stream. ##### writable.uncork() The `writable.uncork()` method flushes all data buffered since [`stream.cork()`][] was called. When using `writable.cork()` and `writable.uncork()` to manage the buffering of writes to a stream, it is recommended that calls to `writable.uncork()` be deferred using `process.nextTick()`. Doing so allows batching of all `writable.write()` calls that occur within a given Node.js event loop phase. ```js stream.cork(); stream.write('some '); stream.write('data '); process.nextTick(() => stream.uncork()); ``` If the `writable.cork()` method is called multiple times on a stream, the same number of calls to `writable.uncork()` must be called to flush the buffered data. ``` stream.cork(); stream.write('some '); stream.cork(); stream.write('data '); process.nextTick(() => { stream.uncork(); // The data will not be flushed until uncork() is called a second time. stream.uncork(); }); ``` ##### writable.write(chunk[, encoding][, callback]) * `chunk` {String|Buffer} The data to write * `encoding` {String} The encoding, if `chunk` is a String * `callback` {Function} Callback for when this chunk of data is flushed * Returns: {Boolean} `false` if the stream wishes for the calling code to wait for the `'drain'` event to be emitted before continuing to write additional data; otherwise `true`. The `writable.write()` method writes some data to the stream, and calls the supplied `callback` once the data has been fully handled. If an error occurs, the `callback` *may or may not* be called with the error as its first argument. To reliably detect write errors, add a listener for the `'error'` event. The return value indicates whether the written `chunk` was buffered internally and the buffer has exceeded the `highWaterMark` configured when the stream was created. If `false` is returned, further attempts to write data to the stream should be paused until the `'drain'` event is emitted. A Writable stream in object mode will always ignore the `encoding` argument. ### Readable Streams Readable streams are an abstraction for a *source* from which data is consumed. Examples of Readable streams include: * [HTTP responses, on the client][http-incoming-message] * [HTTP requests, on the server][http-incoming-message] * [fs read streams][] * [zlib streams][zlib] * [crypto streams][crypto] * [TCP sockets][] * [child process stdout and stderr][] * [`process.stdin`][] All [Readable][] streams implement the interface defined by the `stream.Readable` class. #### Two Modes Readable streams effectively operate in one of two modes: flowing and paused. When in flowing mode, data is read from the underlying system automatically and provided to an application as quickly as possible using events via the [`EventEmitter`][] interface. In paused mode, the [`stream.read()`][stream-read] method must be called explicitly to read chunks of data from the stream. All [Readable][] streams begin in paused mode but can be switched to flowing mode in one of the following ways: * Adding a [`'data'`][] event handler. * Calling the [`stream.resume()`][stream-resume] method. * Calling the [`stream.pipe()`][] method to send the data to a [Writable][]. The Readable can switch back to paused mode using one of the following: * If there are no pipe destinations, by calling the [`stream.pause()`][stream-pause] method. * If there are pipe destinations, by removing any [`'data'`][] event handlers, and removing all pipe destinations by calling the [`stream.unpipe()`][] method. The important concept to remember is that a Readable will not generate data until a mechanism for either consuming or ignoring that data is provided. If the consuming mechanism is disabled or taken away, the Readable will *attempt* to stop generating the data. *Note*: For backwards compatibility reasons, removing [`'data'`][] event handlers will **not** automatically pause the stream. Also, if there are piped destinations, then calling [`stream.pause()`][stream-pause] will not guarantee that the stream will *remain* paused once those destinations drain and ask for more data. *Note*: If a [Readable][] is switched into flowing mode and there are no consumers available handle the data, that data will be lost. This can occur, for instance, when the `readable.resume()` method is called without a listener attached to the `'data'` event, or when a `'data'` event handler is removed from the stream. #### Three States The "two modes" of operation for a Readable stream are a simplified abstraction for the more complicated internal state management that is happening within the Readable stream implementation. Specifically, at any given point in time, every Readable is in one of three possible states: * `readable._readableState.flowing = null` * `readable._readableState.flowing = false` * `readable._readableState.flowing = true` When `readable._readableState.flowing` is `null`, no mechanism for consuming the streams data is provided so the stream will not generate its data. Attaching a listener for the `'data'` event, calling the `readable.pipe()` method, or calling the `readable.resume()` method will switch `readable._readableState.flowing` to `true`, causing the Readable to begin actively emitting events as data is generated. Calling `readable.pause()`, `readable.unpipe()`, or receiving "back pressure" will cause the `readable._readableState.flowing` to be set as `false`, temporarily halting the flowing of events but *not* halting the generation of data. While `readable._readableState.flowing` is `false`, data may be accumulating within the streams internal buffer. #### Choose One The Readable stream API evolved across multiple Node.js versions and provides multiple methods of consuming stream data. In general, developers should choose *one* of the methods of consuming data and *should never* use multiple methods to consume data from a single stream. Use of the `readable.pipe()` method is recommended for most users as it has been implemented to provide the easiest way of consuming stream data. Developers that require more fine-grained control over the transfer and generation of data can use the [`EventEmitter`][] and `readable.pause()`/`readable.resume()` APIs. #### Class: stream.Readable ##### Event: 'close' The `'close'` event is emitted when the stream and any of its underlying resources (a file descriptor, for example) have been closed. The event indicates that no more events will be emitted, and no further computation will occur. Not all [Readable][] streams will emit the `'close'` event. ##### Event: 'data' * `chunk` {Buffer|String|any} The chunk of data. For streams that are not operating in object mode, the chunk will be either a string or `Buffer`. For streams that are in object mode, the chunk can be any JavaScript value other than `null`. The `'data'` event is emitted whenever the stream is relinquishing ownership of a chunk of data to a consumer. This may occur whenever the stream is switched in flowing mode by calling `readable.pipe()`, `readable.resume()`, or by attaching a listener callback to the `'data'` event. The `'data'` event will also be emitted whenever the `readable.read()` method is called and a chunk of data is available to be returned. Attaching a `'data'` event listener to a stream that has not been explicitly paused will switch the stream into flowing mode. Data will then be passed as soon as it is available. The listener callback will be passed the chunk of data as a string if a default encoding has been specified for the stream using the `readable.setEncoding()` method; otherwise the data will be passed as a `Buffer`. ```js const readable = getReadableStreamSomehow(); readable.on('data', (chunk) => { console.log(`Received ${chunk.length} bytes of data.`); }); ``` ##### Event: 'end' The `'end'` event is emitted when there is no more data to be consumed from the stream. *Note*: The `'end'` event **will not be emitted** unless the data is completely consumed. This can be accomplished by switching the stream into flowing mode, or by calling [`stream.read()`][stream-read] repeatedly until all data has been consumed. ```js const readable = getReadableStreamSomehow(); readable.on('data', (chunk) => { console.log(`Received ${chunk.length} bytes of data.`); }); readable.on('end', () => { console.log('There will be no more data.'); }); ``` ##### Event: 'error' * {Error} The `'error'` event may be emitted by a Readable implementation at any time. Typically, this may occur if the underlying stream in unable to generate data due to an underlying internal failure, or when a stream implementation attempts to push an invalid chunk of data. The listener callback will be passed a single `Error` object. ##### Event: 'readable' The `'readable'` event is emitted when there is data available to be read from the stream. In some cases, attaching a listener for the `'readable'` event will cause some amount of data to be read into an internal buffer. ```javascript const readable = getReadableStreamSomehow(); readable.on('readable', () => { // there is some data to read now }); ``` The `'readable'` event will also be emitted once the end of the stream data has been reached but before the `'end'` event is emitted. Effectively, the `'readable'` event indicates that the stream has new information: either new data is available or the end of the stream has been reached. In the former case, [`stream.read()`][stream-read] will return the available data. In the latter case, [`stream.read()`][stream-read] will return `null`. For instance, in the following example, `foo.txt` is an empty file: ```js const fs = require('fs'); const rr = fs.createReadStream('foo.txt'); rr.on('readable', () => { console.log('readable:', rr.read()); }); rr.on('end', () => { console.log('end'); }); ``` The output of running this script is: ``` $ node test.js readable: null end ``` *Note*: In general, the `readable.pipe()` and `'data'` event mechanisms are preferred over the use of the `'readable'` event. ##### readable.isPaused() * Return: {Boolean} The `readable.isPaused()` method returns the current operating state of the Readable. This is used primarily by the mechanism that underlies the `readable.pipe()` method. In most typical cases, there will be no reason to use this method directly. ```js const readable = new stream.Readable readable.isPaused() // === false readable.pause() readable.isPaused() // === true readable.resume() readable.isPaused() // === false ``` ##### readable.pause() * Return: `this` The `readable.pause()` method will cause a stream in flowing mode to stop emitting [`'data'`][] events, switching out of flowing mode. Any data that becomes available will remain in the internal buffer. ```js const readable = getReadableStreamSomehow(); readable.on('data', (chunk) => { console.log(`Received ${chunk.length} bytes of data.`); readable.pause(); console.log('There will be no additional data for 1 second.'); setTimeout(() => { console.log('Now data will start flowing again.'); readable.resume(); }, 1000); }); ``` ##### readable.pipe(destination[, options]) * `destination` {stream.Writable} The destination for writing data * `options` {Object} Pipe options * `end` {Boolean} End the writer when the reader ends. Defaults to `true`. The `readable.pipe()` method attaches a [Writable][] stream to the `readable`, causing it to switch automatically into flowing mode and push all of its data to the attached [Writable][]. The flow of data will be automatically managed so that the destination Writable stream is not overwhelmed by a faster Readable stream. The following example pipes all of the data from the `readable` into a file named `file.txt`: ```js const readable = getReadableStreamSomehow(); const writable = fs.createWriteStream('file.txt'); // All the data from readable goes into 'file.txt' readable.pipe(writable); ``` It is possible to attach multiple Writable streams to a single Readable stream. The `readable.pipe()` method returns a reference to the *destination* stream making it possible to set up chains of piped streams: ```js const r = fs.createReadStream('file.txt'); const z = zlib.createGzip(); const w = fs.createWriteStream('file.txt.gz'); r.pipe(z).pipe(w); ``` By default, [`stream.end()`][stream-end] is called on the destination Writable stream when the source Readable stream emits [`'end'`][], so that the destination is no longer writable. To disable this default behavior, the `end` option can be passed as `false`, causing the destination stream to remain open, as illustrated in the following example: ```js reader.pipe(writer, { end: false }); reader.on('end', () => { writer.end('Goodbye\n'); }); ``` One important caveat is that if the Readable stream emits an error during processing, the Writable destination *is not closed* automatically. If an error occurs, it will be necessary to *manually* close each stream in order to prevent memory leaks. *Note*: The [`process.stderr`][] and [`process.stdout`][] Writable streams are never closed until the Node.js process exits, regardless of the specified options. ##### readable.read([size]) * `size` {Number} Optional argument to specify how much data to read. * Return {String|Buffer|Null} The `readable.read()` method pulls some data out of the internal buffer and returns it. If no data available to be read, `null` is returned. By default, the data will be returned as a `Buffer` object unless an encoding has been specified using the `readable.setEncoding()` method or the stream is operating in object mode. The optional `size` argument specifies a specific number of bytes to read. If `size` bytes are not available to be read, `null` will be returned *unless* the stream has ended, in which case all of the data remaining in the internal buffer will be returned (*even if it exceeds `size` bytes*). If the `size` argument is not specified, all of the data contained in the internal buffer will be returned. The `readable.read()` method should only be called on Readable streams operating in paused mode. In flowing mode, `readable.read()` is called automatically until the internal buffer is fully drained. ```js const readable = getReadableStreamSomehow(); readable.on('readable', () => { var chunk; while (null !== (chunk = readable.read())) { console.log(`Received ${chunk.length} bytes of data.`); } }); ``` In general, it is recommended that developers avoid the use of the `'readable'` event and the `readable.read()` method in favor of using either `readable.pipe()` or the `'data'` event. A Readable stream in object mode will always return a single item from a call to [`readable.read(size)`][stream-read], regardless of the value of the `size` argument. *Note:* If the `readable.read()` method returns a chunk of data, a `'data'` event will also be emitted. *Note*: Calling [`stream.read([size])`][stream-read] after the [`'end'`][] event has been emitted will return `null`. No runtime error will be raised. ##### readable.resume() * Return: `this` The `readable.resume()` method causes an explicitly paused Readable stream to resume emitting [`'data'`][] events, switching the stream into flowing mode. The `readable.resume()` method can be used to fully consume the data from a stream without actually processing any of that data as illustrated in the following example: ```js getReadableStreamSomehow() .resume() .on('end', () => { console.log('Reached the end, but did not read anything.'); }); ``` ##### readable.setEncoding(encoding) * `encoding` {String} The encoding to use. * Return: `this` The `readable.setEncoding()` method sets the default character encoding for data read from the Readable stream. Setting an encoding causes the stream data to be returned as string of the specified encoding rather than as `Buffer` objects. For instance, calling `readable.setEncoding('utf8')` will cause the output data will be interpreted as UTF-8 data, and passed as strings. Calling `readable.setEncoding('hex')` will cause the data to be encoded in hexadecimal string format. The Readable stream will properly handle multi-byte characters delivered through the stream that would otherwise become improperly decoded if simply pulled from the stream as `Buffer` objects. Encoding can be disabled by calling `readable.setEncoding(null)`. This approach is useful when working with binary data or with large multi-byte strings spread out over multiple chunks. ```js const readable = getReadableStreamSomehow(); readable.setEncoding('utf8'); readable.on('data', (chunk) => { assert.equal(typeof chunk, 'string'); console.log('got %d characters of string data', chunk.length); }); ``` ##### readable.unpipe([destination]) * `destination` {stream.Writable} Optional specific stream to unpipe The `readable.unpipe()` method detaches a Writable stream previously attached using the [`stream.pipe()`][] method. If the `destination` is not specified, then *all* pipes are detached. If the `destination` is specified, but no pipe is set up for it, then the method does nothing. ```js const readable = getReadableStreamSomehow(); const writable = fs.createWriteStream('file.txt'); // All the data from readable goes into 'file.txt', // but only for the first second readable.pipe(writable); setTimeout(() => { console.log('Stop writing to file.txt'); readable.unpipe(writable); console.log('Manually close the file stream'); writable.end(); }, 1000); ``` ##### readable.unshift(chunk) * `chunk` {Buffer|String} Chunk of data to unshift onto the read queue The `readable.unshift()` method pushes a chunk of data back into the internal buffer. This is useful in certain situations where a stream is being consumed by code that needs to "un-consume" some amount of data that it has optimistically pulled out of the source, so that the data can be passed on to some other party. *Note*: The `stream.unshift(chunk)` method cannot be called after the [`'end'`][] event has been emitted or a runtime error will be thrown. Developers using `stream.unshift()` often should consider switching to use of a [Transform][] stream instead. See the [API for Stream Implementers][] section for more information. ```js // Pull off a header delimited by \n\n // use unshift() if we get too much // Call the callback with (error, header, stream) const StringDecoder = require('string_decoder').StringDecoder; function parseHeader(stream, callback) { stream.on('error', callback); stream.on('readable', onReadable); const decoder = new StringDecoder('utf8'); var header = ''; function onReadable() { var chunk; while (null !== (chunk = stream.read())) { var str = decoder.write(chunk); if (str.match(/\n\n/)) { // found the header boundary var split = str.split(/\n\n/); header += split.shift(); const remaining = split.join('\n\n'); const buf = Buffer.from(remaining, 'utf8'); if (buf.length) stream.unshift(buf); stream.removeListener('error', callback); stream.removeListener('readable', onReadable); // now the body of the message can be read from the stream. callback(null, header, stream); } else { // still reading the header. header += str; } } } } ``` *Note*: Unlike [`stream.push(chunk)`][stream-push], `stream.unshift(chunk)` will not end the reading process by resetting the internal reading state of the stream. This can cause unexpected results if `readable.unshift()` is called during a read (i.e. from within a [`stream._read()`][stream-_read] implementation on a custom stream). Following the call to `readable.unshift()` with an immediate [`stream.push('')`][stream-push] will reset the reading state appropriately, however it is best to simply avoid calling `readable.unshift()` while in the process of performing a read. ##### readable.wrap(stream) * `stream` {Stream} An "old style" readable stream Versions of Node.js prior to v0.10 had streams that did not implement the entire `stream` module API as it is currently defined. (See [Compatibility][] for more information.) When using an older Node.js library that emits [`'data'`][] events and has a [`stream.pause()`][stream-pause] method that is advisory only, the `readable.wrap()` method can be used to create a [Readable][] stream that uses the old stream as its data source. It will rarely be necessary to use `readable.wrap()` but the method has been provided as a convenience for interacting with older Node.js applications and libraries. For example: ```js const OldReader = require('./old-api-module.js').OldReader; const Readable = require('stream').Readable; const oreader = new OldReader; const myReader = new Readable().wrap(oreader); myReader.on('readable', () => { myReader.read(); // etc. }); ``` ### Duplex and Transform Streams #### Class: stream.Duplex Duplex streams are streams that implement both the [Readable][] and [Writable][] interfaces. Examples of Duplex streams include: * [TCP sockets][] * [zlib streams][zlib] * [crypto streams][crypto] #### Class: stream.Transform Transform streams are [Duplex][] streams where the output is in some way related to the input. Like all [Duplex][] streams, Transform streams implement both the [Readable][] and [Writable][] interfaces. Examples of Transform streams include: * [zlib streams][zlib] * [crypto streams][crypto] ## API for Stream Implementers The `stream` module API has been designed to make it possible to easily implement streams using JavaScript's prototypical inheritance model. First, a stream developer would declare a new JavaScript class that extends one of the four basic stream classes (`stream.Writable`, `stream.Readable`, `stream.Duplex`, or `stream.Transform`), making sure the call the appropriate parent class constructor: ```js const Writable = require('stream').Writable; class MyWritable extends Writable { constructor(options) { super(options); } } ``` The new stream class must then implement one or more specific methods, depending on the type of stream being created, as detailed in the chart below:
Use-case |
Class |
Method(s) to implement |
---|---|---|
Reading only |
[Readable](#stream_class_stream_readable) |
|
Writing only |
[Writable](#stream_class_stream_writable) |
|
Reading and writing |
[Duplex](#stream_class_stream_duplex) |
|
Operate on written data, then read the result |
[Transform](#stream_class_stream_transform) |
|