Downstream --- Cerebras Software Language (CSL) Implementation

downstream provides efficient, constant-space implementations of stream curation algorithms.

• Free software: MIT license
• Documentation: https://mmore500.github.io/downstream

Installation

CSL downstream is packaged as a header-only library. It can be added to a system-wide include path, or incorporated as a git submodule in another project.

API Reference

See the Python quickstart for outline and intuition.

Each algorithm variant is accessible through its own module:

• Steady: dstream.steady_algo
• Stretched: dstream.stretched_algo
• Tilted: dstream.tilted_algo

See selecting a dstream algorithm for more information.

has_ingest_capacity

fn has_ingest_capacity(S: u32, T: u32) bool

Determines if there is capacity to ingest a data item at logical time T.

• S: Buffer size (must be a power of two)
• T: Stream position of data item (zero-indexed)
• T: Logical time of data item

assign_storage_site

fn assign_storage_site(S: u32, T: u32) u32

Site selection algorithm for steady curation. Returns selected site or S if data should be discarded.

• S: Buffer size (must be a power of two)
• T: Stream position of data item (zero-indexed)

Citing

If downstream contributes to a scientific publication, please cite it as

Yang C., Wagner J., Dolson E., Zaman L., & Moreno M. A. (2025). Downstream: efficient cross-platform algorithms for fixed-capacity stream downsampling. arXiv preprint arXiv:2506.12975. https://doi.org/10.48550/arXiv.2506.12975

@misc{yang2025downstream,
      doi={10.48550/arXiv.2506.12975},
      url={https://arxiv.org/abs/2506.12975},
      title={Downstream: efficient cross-platform algorithms for fixed-capacity stream downsampling},
      author={Connor Yang and Joey Wagner and Emily Dolson and Luis Zaman and Matthew Andres Moreno},
      year={2025},
      eprint={2506.12975},
      archivePrefix={arXiv},
      primaryClass={cs.DS},
}

And don't forget to leave a star on GitHub!