diff options
Diffstat (limited to 'node_modules/unique-stream/README.md')
-rw-r--r-- | node_modules/unique-stream/README.md | 133 |
1 files changed, 133 insertions, 0 deletions
diff --git a/node_modules/unique-stream/README.md b/node_modules/unique-stream/README.md new file mode 100644 index 000000000..fce2ec019 --- /dev/null +++ b/node_modules/unique-stream/README.md @@ -0,0 +1,133 @@ +# unique-stream + +node.js through stream that emits a unique stream of objects based on criteria + +[](https://travis-ci.org/eugeneware/unique-stream) +[](https://coveralls.io/github/eugeneware/unique-stream?branch=master) + +## Installation + +Install via [npm](https://www.npmjs.com/): + +``` +$ npm install unique-stream +``` + +## Examples + +### Dedupe a ReadStream based on JSON.stringify: + +``` js +var unique = require('unique-stream') + , Stream = require('stream'); + +// return a stream of 3 identical objects +function makeStreamOfObjects() { + var s = new Stream; + s.readable = true; + var count = 3; + for (var i = 0; i < 3; i++) { + setImmediate(function () { + s.emit('data', { name: 'Bob', number: 123 }); + --count || end(); + }); + } + + function end() { + s.emit('end'); + } + + return s; +} + +// Will only print out one object as the rest are dupes. (Uses JSON.stringify) +makeStreamOfObjects() + .pipe(unique()) + .on('data', console.log); + +``` + +### Dedupe a ReadStream based on an object property: + +``` js +// Use name as the key field to dedupe on. Will only print one object +makeStreamOfObjects() + .pipe(unique('name')) + .on('data', console.log); +``` + +### Dedupe a ReadStream based on a custom function: + +``` js +// Use a custom function to dedupe on. Use the 'number' field. Will only print one object. +makeStreamOfObjects() + .pipe(function (data) { + return data.number; + }) + .on('data', console.log); +``` + +## Dedupe multiple streams + +The reason I wrote this was to dedupe multiple object streams: + +``` js +var aggregator = unique(); + +// Stream 1 +makeStreamOfObjects() + .pipe(aggregator); + +// Stream 2 +makeStreamOfObjects() + .pipe(aggregator); + +// Stream 3 +makeStreamOfObjects() + .pipe(aggregator); + +aggregator.on('data', console.log); +``` + +## Use a custom store to record keys that have been encountered + +By default a set is used to store keys encountered so far, in order to check new ones for +uniqueness. You can supply your own store instead, providing it supports the add(key) and +has(key) methods. This could allow you to use a persistant store so that already encountered +objects are not re-streamed when node is reloaded. + +``` js +var keyStore = { + store: {}, + + add: function(key) { + this.store[key] = true; + }, + + has: function(key) { + return this.store[key] !== undefined; + } +}; + +makeStreamOfObjects() + .pipe(unique('name', keyStore)) + .on('data', console.log); +``` + +## Contributing + +unique-stream is an **OPEN Open Source Project**. This means that: + +> Individuals making significant and valuable contributions are given commit-access to the project to contribute as they see fit. This project is more like an open wiki than a standard guarded open source project. + +See the [CONTRIBUTING.md](https://github.com/eugeneware/unique-stream/blob/master/CONTRIBUTING.md) file for more details. + +### Contributors + +unique-stream is only possible due to the excellent work of the following contributors: + +<table><tbody> +<tr><th align="left">Eugene Ware</th><td><a href="https://github.com/eugeneware">GitHub/eugeneware</a></td></tr> +<tr><th align="left">Craig Ambrose</th><td><a href="https://github.com/craigambrose">GitHub/craigambrose</a></td></tr> +<tr><th align="left">Shinnosuke Watanabe</th><td><a href="https://github.com/shinnn">GitHub/shinnn</a></td></tr> +</tbody></table> |