ephemeral-set

This is a simple node.js class for tracking "ephemeral" things. It was built to expire temporary, public objects in AWS S3 (on flexible timescales faster than lifecycle policies allow) but has other uses.

ephemeral-set has no non-core node.js dependencies, is a single file, accepts float and ISO 8601 durations for object lifetimes, tracks objects using only a key, includes on style events to track execution, and you specify the function(s) that get called when objects "expire". A simple linked list construct is used to store objects in expiration-time order for efficient processing.

Quickstart

As usual, include in your node.js project with

$ npm install --save ephemeral-set

from the command line and

const EphemeralSet = require( 'ephemeral-set' );

in your code.

Using

Creating a new EphemeralSet should seem straightforward; simply import and instantiate:

const EphemeralSet = require( 'ephemeral-set' );
const es = new EphemeralSet( k => { console.log(`expiring ${k}`); } , 5 , 2 );

Here we create a new set whose objects will have a default lifetime of 5 seconds, with an expiration "check loop" firing every 2 seconds, that will simply print an object's key when that object is expired. This "check loop" must be started with

es.start();

Note the EphemeralSet does not need any objects to start the loop.

We can then add objects with unique keys with

es.add( "object 1" ); 

To see the effect of this addition, you can call

es.print();

This object will automatically "expire" after 5 seconds. With a check loop firing every 2 seconds, at some point between 5 and 6 seconds the code should print

expiring object 1

to your terminal (if the node.js process hasn't already exited).

To stop the check loop you can call

es.stop();

Events

You can provide callbacks for the following events:

• es.on( 'start' , f ) calls f when the set's check loop starts. f should expect no arguments.

• es.on( 'check' , f ) calls f when the check loop fires. This event is emitted once per expiration check, not with every object. f should expect no arguments.

• es.on( 'expire' , f ) calls f when an object should actually be expired. f should expect a single argument, the key.

• es.on( 'empty' , f ) calls f when the set is empty; that is, all objects have been "expired". f should expect no arguments.

• es.on( 'stop' , f ) calls f when the set's check loop is stopped. f should expect no arguments.

In each of these cases you can add multiple routines, all will be called.

Examples

See test/test.js for some examples. You can run these with either npm test or node test/test.js. Here is some sample output:

EphemeralSet: printing contents...
  expire time: 1574512711
  keys: 
    a50d6508c1cf30702e0bc1de
    cb6e7cb7b002cb0a1e21899d
    3320c1b804b4b1c89ba11ed3
    3f154a997e359065521229bd
    4382b45d0a68629f6a2684f8
    587e17e5484177d426750551
  expire time: 1574512712
  keys: 
    bcd1dc1c1c6f5b9226682ffc
  expire time: 1574512713
  keys: 
    388a951d5fb3b42899709aac
  expire time: 1574512714
  keys: 
    c7bffe4543fd0cb622a94476
    35a0fce5a1d5b358d6c7f0ca
    01c42ebba4355cb5c1217f62
    f8680981a35ef0a0273bf245
  expire time: 1574512715
  keys: 
    573a1584bc06d098466211fd
    2b97167cf688a0d3051144b2
    8062044e9adda614a4bc5dca
    0b47e0a2f1225508a610864d
    781de043d8d90f3163392409
    c99cdb9736e65248d78f4547
    a3fd5bba509e13273a941731
    0ce5a1980730cec4ade90ef0
EphemeralSet starting expire check loop at 1574512710.662
Checking expirations at 1574512712.665
expiring a50d6508c1cf30702e0bc1de at 1574512712.665
expiring cb6e7cb7b002cb0a1e21899d at 1574512712.666
expiring 3320c1b804b4b1c89ba11ed3 at 1574512712.666
expiring 3f154a997e359065521229bd at 1574512712.666
expiring 4382b45d0a68629f6a2684f8 at 1574512712.666
expiring 587e17e5484177d426750551 at 1574512712.666
expiring bcd1dc1c1c6f5b9226682ffc at 1574512712.666
Checking expirations at 1574512714.668
expiring 388a951d5fb3b42899709aac at 1574512714.668
expiring c7bffe4543fd0cb622a94476 at 1574512714.668
expiring 35a0fce5a1d5b358d6c7f0ca at 1574512714.668
expiring 01c42ebba4355cb5c1217f62 at 1574512714.668
expiring f8680981a35ef0a0273bf245 at 1574512714.668
Checking expirations at 1574512716.671
expiring 573a1584bc06d098466211fd at 1574512716.671
expiring 2b97167cf688a0d3051144b2 at 1574512716.671
expiring 8062044e9adda614a4bc5dca at 1574512716.671
expiring 0b47e0a2f1225508a610864d at 1574512716.671
expiring 781de043d8d90f3163392409 at 1574512716.671
expiring c99cdb9736e65248d78f4547 at 1574512716.671
expiring a3fd5bba509e13273a941731 at 1574512716.671
expiring 0ce5a1980730cec4ade90ef0 at 1574512716.671
EphemeralSet is empty at 1574512716.672

Contact

Written by W. Ross Morrow, for Data, Analytics, and Research Computing (DARC) at the Stanford GSB Research Hub.