| 1 | ---
|
| 2 | in_progress: yes
|
| 3 | ---
|
| 4 |
|
| 5 | Solutions to the Framing Problem
|
| 6 | ================================
|
| 7 |
|
| 8 | YSH is a shell that lets you write correct programs. Programs often parse
|
| 9 | multiple records or strings sent over a pipe, i.e. the "framing problem".
|
| 10 | (This terminology is borrowed from network engineering.)
|
| 11 |
|
| 12 | YSH provides a few solutions for that, and each solution is useful in certain
|
| 13 | contexts.
|
| 14 |
|
| 15 | <div id="toc">
|
| 16 | </div>
|
| 17 |
|
| 18 | ## Delimiter Based Solutions
|
| 19 |
|
| 20 | ### Newline as Delimiter
|
| 21 |
|
| 22 | The traditional Unix solution. It doesn't work when the payload contains
|
| 23 | newlines.
|
| 24 |
|
| 25 | Note that Unix filenames, `argv` entries, and `environ` entries can all
|
| 26 | contain newlines, but not `NUL` (`\0`).
|
| 27 |
|
| 28 | ### NUL as delimiter: `read -0`
|
| 29 |
|
| 30 | So naturally we also support the format that `find -print0` emits, and
|
| 31 | `xargs -0` consumes.
|
| 32 |
|
| 33 | ## Solutions That Can Express Arbitrary Bytes
|
| 34 |
|
| 35 | ### Quoting / Escaping Special characters: [QSN][]
|
| 36 |
|
| 37 | QSN uses backslash escapes like `\n`, `\x00`, and `\u{3bc}` to express
|
| 38 | arbitrary bytes (and characters).
|
| 39 |
|
| 40 | ### Length-Prefixed: Netstrings
|
| 41 |
|
| 42 | TODO: Implement this.
|
| 43 |
|
| 44 | Like [QSN][], this format is "8-bit clean", but:
|
| 45 |
|
| 46 | - A netstring encoder is easier to write than a QSN encoder. This may be
|
| 47 | useful if you don't have a library handy.
|
| 48 | - It's more efficient to decode, in theory.
|
| 49 |
|
| 50 | However, the encoded output may contain binary data, which is hard to view in a
|
| 51 | terminal.
|
| 52 |
|
| 53 | [QSN]: qsn.html
|
| 54 |
|