Skip to content

Downstream --- Cerebras Software Language (CSL) Implementation

downstream wordmark

CI GitHub stars DOI

downstream provides efficient, constant-space implementations of stream curation algorithms.

Installation

CSL downstream is packaged as a header-only library. It can be added to a system-wide include path, or incorporated as a git submodule in another project.

Requires Cerebras SDK, available through invitation. Library CSL code targets compatibility with Cerebras SDK v1.X releases. As of October 2025, library code is tested against Cerebras SDK v1.4.0.

API Reference

See the Python quickstart for outline and intuition.

Each algorithm variant is accessible through its own module:

  • Steady: dstream.steady_algo
  • Stretched: dstream.stretched_algo
  • Tilted: dstream.tilted_algo

See selecting a dstream algorithm for more information.

has_ingest_capacity

fn has_ingest_capacity(S: u32, T: u32) bool
Determines if there is capacity to ingest a data item at logical time T.

  • S: Buffer size (must be a power of two)
  • T: Stream position of data item (zero-indexed)
  • T: Logical time of data item

assign_storage_site

fn assign_storage_site(S: u32, T: u32) u32
Site selection algorithm for steady curation. Returns selected site or S if data should be discarded.

  • S: Buffer size (must be a power of two)
  • T: Stream position of data item (zero-indexed)

Citing

If downstream contributes to a scientific publication, please cite it as

Yang C., Wagner J., Dolson E., Zaman L., & Moreno M. A. (2025). Downstream: efficient cross-platform algorithms for fixed-capacity stream downsampling. arXiv preprint arXiv:2506.12975. https://doi.org/10.48550/arXiv.2506.12975

@misc{yang2025downstream,
      doi={10.48550/arXiv.2506.12975},
      url={https://arxiv.org/abs/2506.12975},
      title={Downstream: efficient cross-platform algorithms for fixed-capacity stream downsampling},
      author={Connor Yang and Joey Wagner and Emily Dolson and Luis Zaman and Matthew Andres Moreno},
      year={2025},
      eprint={2506.12975},
      archivePrefix={arXiv},
      primaryClass={cs.DS},
}

And don't forget to leave a star on GitHub!