Warning: Work in progress! Leave feedback on Zulip or Github if you'd like this doc to be updated.

Solutions to the Framing Problem

YSH is a shell that lets you write correct programs. Programs often parse multiple records or strings sent over a pipe, i.e. the "framing problem". (This terminology is borrowed from network engineering.)

YSH provides a few solutions for that, and each solution is useful in certain contexts.

Table of Contents
Delimiter Based Solutions
Newline as Delimiter
NUL as delimiter: read -0
Solutions That Can Express Arbitrary Bytes
Quoting / Escaping Special characters: QSN
Length-Prefixed: Netstrings

Delimiter Based Solutions

Newline as Delimiter

The traditional Unix solution. It doesn't work when the payload contains newlines.

Note that Unix filenames, argv entries, and environ entries can all contain newlines, but not NUL (\0).

NUL as delimiter: read -0

So naturally we also support the format that find -print0 emits, and xargs -0 consumes.

Solutions That Can Express Arbitrary Bytes

Quoting / Escaping Special characters: QSN

QSN uses backslash escapes like \n, \x00, and \u{3bc} to express arbitrary bytes (and characters).

Length-Prefixed: Netstrings

TODO: Implement this.

Like QSN, this format is "8-bit clean", but:

However, the encoded output may contain binary data, which is hard to view in a terminal.

Generated on Mon, 01 Jul 2024 21:48:37 +0000