1 | ---
|
2 | default_highlighter: oils-sh
|
3 | ---
|
4 |
|
5 | A Tour of YSH
|
6 | =============
|
7 |
|
8 | <!-- author's note about example names
|
9 |
|
10 | - people: alice, bob
|
11 | - nouns: ale, bean
|
12 | - peanut, coconut
|
13 | - 42 for integers
|
14 | -->
|
15 |
|
16 | This doc describes the [YSH]($xref) language from **clean slate**
|
17 | perspective. We don't assume you know Unix shell, or the compatible
|
18 | [OSH]($xref). But shell users will see the similarity, with simplifications
|
19 | and upgrades.
|
20 |
|
21 | Remember, YSH is for Python and JavaScript users who avoid shell! See the
|
22 | [project FAQ][FAQ] for more color on that.
|
23 |
|
24 | [FAQ]: https://www.oilshell.org/blog/2021/01/why-a-new-shell.html
|
25 | [path dependence]: https://en.wikipedia.org/wiki/Path_dependence
|
26 |
|
27 | This document is **long** because it demonstrates nearly every feature of the
|
28 | language. You may want to read it in multiple sittings, or read [The Simplest
|
29 | Explanation of
|
30 | Oil](https://www.oilshell.org/blog/2020/01/simplest-explanation.html) first.
|
31 | (Until 2023, YSH was called the "Oil language".)
|
32 |
|
33 |
|
34 | Here's a summary of what follows:
|
35 |
|
36 | 1. YSH has interleaved *word*, *command*, and *expression* languages.
|
37 | - The command language has Ruby-like *blocks*, and the expression language
|
38 | has Python-like *data types*.
|
39 | 2. YSH has both builtin *commands* like `cd /tmp`, and builtin *functions* like
|
40 | `join()`.
|
41 | 3. Languages for *data*, like [JSON][], are complementary to YSH code.
|
42 | 4. OSH and YSH share both an *interpreter data model* and a *process model*
|
43 | (provided by the Unix kernel). Understanding these common models will make
|
44 | you both a better shell user and YSH user.
|
45 |
|
46 | Keep these points in mind as you read the details below.
|
47 |
|
48 | [JSON]: https://json.org
|
49 |
|
50 | <div id="toc">
|
51 | </div>
|
52 |
|
53 | ## Preliminaries
|
54 |
|
55 | Start YSH just like you start bash or Python:
|
56 |
|
57 | <!-- oils-sh below skips code block extraction, since it doesn't run -->
|
58 |
|
59 | ```sh-prompt
|
60 | bash$ ysh # assuming it's installed
|
61 |
|
62 | ysh$ echo 'hello world' # command typed into YSH
|
63 | hello world
|
64 | ```
|
65 |
|
66 | In the sections below, we'll save space by showing output **in comments**, with
|
67 | `=>`:
|
68 |
|
69 | echo 'hello world' # => hello world
|
70 |
|
71 | Multi-line output is shown like this:
|
72 |
|
73 | echo one
|
74 | echo two
|
75 | # =>
|
76 | # one
|
77 | # two
|
78 |
|
79 | ## Examples
|
80 |
|
81 | ### Hello World Script
|
82 |
|
83 | You can also type commands into a file like `hello.ysh`. This is a complete
|
84 | YSH program, which is identical to a shell program:
|
85 |
|
86 | echo 'hello world' # => hello world
|
87 |
|
88 | ### A Taste of YSH
|
89 |
|
90 | Unlike shell, YSH has `var` and `const` keywords:
|
91 |
|
92 | const name = 'world' # const is rarer, used the top-level
|
93 | echo "hello $name" # => hello world
|
94 |
|
95 | They take rich Python-like expressions on the right:
|
96 |
|
97 | var x = 42 # an integer, not a string
|
98 | setvar x = x * 2 + 1 # mutate with the 'setvar' keyword
|
99 |
|
100 | setvar x += 5 # Increment by 5
|
101 | echo $x # => 6
|
102 |
|
103 | var mylist = [x, 7] # two integers [6, 7]
|
104 |
|
105 | Expressions are often surrounded by `()`:
|
106 |
|
107 | if (x > 0) {
|
108 | echo 'positive'
|
109 | } # => positive
|
110 |
|
111 | for i, item in (mylist) { # 'mylist' is a variable, not a string
|
112 | echo "[$i] item $item"
|
113 | }
|
114 | # =>
|
115 | # [0] item 6
|
116 | # [1] item 7
|
117 |
|
118 | YSH has Ruby-like blocks:
|
119 |
|
120 | cd /tmp {
|
121 | echo hi > greeting.txt # file created inside /tmp
|
122 | echo $PWD # => /tmp
|
123 | }
|
124 | echo $PWD # prints the original directory
|
125 |
|
126 | And utilities to read and write JSON:
|
127 |
|
128 | var person = {name: 'bob', age: 42}
|
129 | json write (person)
|
130 | # =>
|
131 | # {
|
132 | # "name": "bob",
|
133 | # "age": 42,
|
134 | # }
|
135 |
|
136 | echo '["str", 42]' | json read # sets '_reply' variable by default
|
137 |
|
138 | The `=` keyword evaluates and prints an expression:
|
139 |
|
140 | = _reply
|
141 | # => (List) ["str", 42]
|
142 |
|
143 | (Think of it like `var x = _reply`, without the `var`.)
|
144 |
|
145 | ## Word Language: Expressions for Strings (and Arrays)
|
146 |
|
147 | Let's describe the word language first, and then talk about commands and
|
148 | expressions. Words are a rich language because **strings** are a central
|
149 | concept in shell.
|
150 |
|
151 | ### Three Kinds of String Literals
|
152 |
|
153 | You can choose the quoting style that's most convenient to write a given
|
154 | string.
|
155 |
|
156 | #### Double-Quoted, Single-Quoted, and J8 strings (like JSON)
|
157 |
|
158 | Double-quoted strings allow **interpolation with `$`**:
|
159 |
|
160 | var person = 'alice'
|
161 | echo "hi $person, $(echo bye)" # => hi alice, bye
|
162 |
|
163 | Write operators by escaping them with `\`:
|
164 |
|
165 | echo "\$ \" \\ " # => $ " \
|
166 |
|
167 | In single-quoted strings, all characters are **literal** (except `'`, which
|
168 | can't be expressed):
|
169 |
|
170 | echo 'c:\Program Files\' # => c:\Program Files\
|
171 |
|
172 | If you want C-style backslash **character escapes**, use a J8 string, which is
|
173 | like JSON, but with single quotes::
|
174 |
|
175 | echo u' A is \u{41} \n line two, with backslash \\'
|
176 | # =>
|
177 | # A is A
|
178 | # line two, with backslash \
|
179 |
|
180 | The `u''` strings are guaranteed to be valid Unicode (unlike JSON), but you can
|
181 | also use `b''` strings:
|
182 |
|
183 | echo b'byte \yff' # byte that's not valid unicode, like \xff in other languages
|
184 | # do not confuse with \u{ff}
|
185 |
|
186 | #### Multi-line Strings
|
187 |
|
188 | Multi-line strings are surrounded with triple quotes. They come in the same
|
189 | three varieties, and leading whitespace is stripped in a convenient way.
|
190 |
|
191 | sort <<< """
|
192 | var sub: $x
|
193 | command sub: $(echo hi)
|
194 | expression sub: $[x + 3]
|
195 | """
|
196 | # =>
|
197 | # command sub: hi
|
198 | # expression sub: 9
|
199 | # var sub: 6
|
200 |
|
201 | sort <<< '''
|
202 | $2.00 # literal $, no interpolation
|
203 | $1.99
|
204 | '''
|
205 | # =>
|
206 | # $1.99
|
207 | # $2.00
|
208 |
|
209 | sort <<< u'''
|
210 | C\tD
|
211 | A\tB
|
212 | ''' # b''' strings also supported
|
213 | # =>
|
214 | # A B
|
215 | # C D
|
216 |
|
217 | (Use multiline strings instead of shell's [here docs]($xref:here-doc).)
|
218 |
|
219 | ### Three Kinds of Substitution
|
220 |
|
221 | YSH has syntax for 3 types of substitution, all of which start with `$`. These
|
222 | things can all be converted to a **string**:
|
223 |
|
224 | 1. Variables
|
225 | 2. The output of commands
|
226 | 3. The value of expressions
|
227 |
|
228 | #### Variable Sub
|
229 |
|
230 | The syntax `$a` or `${a}` converts a variable to a string:
|
231 |
|
232 | var a = 'ale'
|
233 | echo $a # => ale
|
234 | echo _${a}_ # => _ale_
|
235 | echo "_ $a _" # => _ ale _
|
236 |
|
237 | The shell operator `:-` is occasionally useful in YSH:
|
238 |
|
239 | echo ${not_defined:-'default'} # => default
|
240 |
|
241 | #### Command Sub
|
242 |
|
243 | The `$(echo hi)` syntax runs a command and captures its `stdout`:
|
244 |
|
245 | echo $(hostname) # => example.com
|
246 | echo "_ $(hostname) _" # => _ example.com _
|
247 |
|
248 | #### Expression Sub
|
249 |
|
250 | The `$[myexpr]` syntax evaluates an expression and converts it to a string:
|
251 |
|
252 | echo $[a] # => ale
|
253 | echo $[1 + 2 * 3] # => 7
|
254 | echo "_ $[1 + 2 * 3] _" # => _ 7 _
|
255 |
|
256 | <!-- TODO: safe substitution with "$[a]"html -->
|
257 |
|
258 | ### Arrays of Strings: Globs, Brace Expansion, Splicing, and Splitting
|
259 |
|
260 | There are four constructs that evaluate to an **list of strings**, rather than
|
261 | a single string.
|
262 |
|
263 | #### Globs
|
264 |
|
265 | Globs like `*.py` evaluate to a list of files.
|
266 |
|
267 | touch foo.py bar.py # create the files
|
268 | write *.py
|
269 | # =>
|
270 | # foo.py
|
271 | # bar.py
|
272 |
|
273 | If no files match, it evaluates to an empty list (`[]`).
|
274 |
|
275 | #### Brace Expansion
|
276 |
|
277 | The brace expansion mini-language lets you write strings without duplication:
|
278 |
|
279 | write {alice,bob}@example.com
|
280 | # =>
|
281 | # alice@example.com
|
282 | # bob@example.com
|
283 |
|
284 | #### Splicing
|
285 |
|
286 | The `@` operator splices an array into a command:
|
287 |
|
288 | var myarray = :| ale bean |
|
289 | write S @myarray E
|
290 | # =>
|
291 | # S
|
292 | # ale
|
293 | # bean
|
294 | # E
|
295 |
|
296 | You also have `@[]` to splice an expression that evaluates to a list:
|
297 |
|
298 | write -- @[split('ale bean')]
|
299 | # =>
|
300 | # ale
|
301 | # bean
|
302 |
|
303 | Each item will be converted to a string.
|
304 |
|
305 | #### Split Command Sub / Split Builtin Sub
|
306 |
|
307 | There's also a variant of *command sub* that splits first:
|
308 |
|
309 | write @(seq 3) # write gets 3 arguments
|
310 | # =>
|
311 | # 1
|
312 | # 2
|
313 | # 3
|
314 |
|
315 | <!-- TODO: This should decode J8 notation, which includes "" j"" and b"" -->
|
316 |
|
317 | ## Command Language: I/O, Control Flow, Abstraction
|
318 |
|
319 | ### Simple Commands and Redirects
|
320 |
|
321 | A simple command is a space-separated list of words, which are often unquoted.
|
322 | YSH looks up the first word to determine if it's a `proc` or shell builtin.
|
323 |
|
324 | echo 'hello world' # The shell builtin 'echo'
|
325 |
|
326 | proc greet (name) { # A proc is like a procedure or process
|
327 | echo "hello $name"
|
328 | }
|
329 |
|
330 | # Now the first word will resolve to the proc
|
331 | greet alice # => hello alice
|
332 |
|
333 | If it's neither, then it's assumed to be an external command:
|
334 |
|
335 | ls -l /tmp # The external 'ls' command
|
336 |
|
337 | Commands accept traditional string arguments, as well as typed arguments in
|
338 | parentheses:
|
339 |
|
340 | # 'write' is a string arg; 'x' is a typed expression arg
|
341 | json write (x)
|
342 |
|
343 | You can **redirect** `stdin` and `stdout` of simple commands:
|
344 |
|
345 | echo hi > tmp.txt # write to a file
|
346 | sort < tmp.txt
|
347 |
|
348 | Idioms for using stderr (identical to shell):
|
349 |
|
350 | ls /tmp 2>errors.txt
|
351 | echo 'fatal error' 1>&2
|
352 |
|
353 | "Simple" commands in YSH can also have typed `()` and block `{}` args, which
|
354 | we'll see in the section on "procs".
|
355 |
|
356 | ### Pipelines
|
357 |
|
358 | Pipelines are a powerful method manipulating data streams:
|
359 |
|
360 | ls | wc -l # count files in this directory
|
361 | find /bin -type f | xargs wc -l # count files in a subtree
|
362 |
|
363 | The stream may contain (lines of) text, binary data, JSON, TSV, and more.
|
364 | Details below.
|
365 |
|
366 | ### Multi-line Commands
|
367 |
|
368 | The YSH `...` prefix lets you write long commands, pipelines, and `&&` chains
|
369 | without `\` line continuations.
|
370 |
|
371 | ... find /bin # traverse this directory and
|
372 | -type f -a -executable # print executable files
|
373 | | sort -r # reverse sort
|
374 | | head -n 30 # limit to 30 files
|
375 | ;
|
376 |
|
377 | When this mode is active:
|
378 |
|
379 | - A single newline behaves like a space
|
380 | - A blank line (two newlines in a row) is illegal, but a line that has only a
|
381 | comment is allowed. This prevents confusion if you forget the `;`
|
382 | terminator.
|
383 |
|
384 | ### `var`, `setvar`, `const` to Declare and Mutate
|
385 |
|
386 | Constants can't be modified:
|
387 |
|
388 | const myconst = 'mystr'
|
389 | # setvar myconst = 'foo' would be an error
|
390 |
|
391 | Modify variables with the `setvar` keyword:
|
392 |
|
393 | var num_beans = 12
|
394 | setvar num_beans = 13
|
395 |
|
396 | A more complex example:
|
397 |
|
398 | var d = {name: 'bob', age: 42} # dict literal
|
399 | setvar d.name = 'alice' # d.name is a synonym for d['name']
|
400 | echo $[d.name] # => alice
|
401 |
|
402 | That's most of what you need to know about assignments. Advanced users may
|
403 | want to use `setglobal` or `call myplace->setValue(42)` in certain situations.
|
404 |
|
405 | <!--
|
406 | var g = 1
|
407 | var h = 2
|
408 | proc demo(:out) {
|
409 | setglobal g = 42
|
410 | setref out = 43
|
411 | }
|
412 | demo :h # pass a reference to h
|
413 | echo "$g $h" # => 42 43
|
414 | -->
|
415 |
|
416 | More details: [Variable Declaration and Mutation](variables.html).
|
417 |
|
418 | ### `for` Loop
|
419 |
|
420 | Shell-style for loops iterate over **words**:
|
421 |
|
422 | for word in 'oils' $num_beans {pea,coco}nut {
|
423 | echo $word
|
424 | }
|
425 | # =>
|
426 | # oils
|
427 | # 13
|
428 | # peanut
|
429 | # coconut
|
430 |
|
431 | You can also request the loop index:
|
432 |
|
433 | for i, word in README.md *.py {
|
434 | echo "$i - $word"
|
435 | }
|
436 | # =>
|
437 | # 0 - README.md
|
438 | # 1 - __init__.py
|
439 |
|
440 | To iterate over lines of `stdin`, use:
|
441 |
|
442 | for line in (stdin) {
|
443 | echo $line
|
444 | }
|
445 | # lines are buffered, so it's much faster than `while read --rawline`
|
446 |
|
447 | Ask for the loop index:
|
448 |
|
449 | for i, line in (stdin) {
|
450 | echo "$i $line"
|
451 | }
|
452 |
|
453 | To iterate over a typed data, use parentheses around an **expression**. The
|
454 | expression should evaluate to an integer `Range`, `List`, or `Dict`:
|
455 |
|
456 | for i in (3 .. 5) { # range operator ..
|
457 | echo "i = $i"
|
458 | }
|
459 | # =>
|
460 | # i = 3
|
461 | # i = 4
|
462 |
|
463 | List:
|
464 |
|
465 | var foods = ['ale', 'bean']
|
466 | for item in (foods) {
|
467 | echo $item
|
468 | }
|
469 | # =>
|
470 | # ale
|
471 | # bean
|
472 |
|
473 | Again, you can request the index with `for i, item in ...`.
|
474 |
|
475 | Here's the most general form of the loop over `Dict`:
|
476 |
|
477 | var mydict = {pea: 42, nut: 10}
|
478 | for i, k, v in (mydict) {
|
479 | echo "$i - $k - $v"
|
480 | }
|
481 | # =>
|
482 | # 0 - pea - 42
|
483 | # 1 - nut - 10
|
484 |
|
485 | There are two simpler forms:
|
486 |
|
487 | - One variable gives you the key: `for k in (mydict)`
|
488 | - Two variables gives you the key and value: `for k, v in (mydict)`
|
489 |
|
490 | (One way to think of it: `for` loops in YSH have the functionality Python's
|
491 | `enumerate()`, `items()`, `keys()`, and `values()`.)
|
492 |
|
493 | <!--
|
494 | TODO: Str loop should give you the (UTF-8 offset, rune)
|
495 | Or maybe just UTF-8 offset? Decoding errors could be exceptions, or Unicode
|
496 | replacement.
|
497 | -->
|
498 |
|
499 | ### `while` Loop
|
500 |
|
501 | While loops can use a **command** as the termination condition:
|
502 |
|
503 | while test --file lock {
|
504 | sleep 1
|
505 | }
|
506 |
|
507 | Or an **expression**, which is surrounded in `()`:
|
508 |
|
509 | var i = 3
|
510 | while (i < 6) {
|
511 | echo "i = $i"
|
512 | setvar i += 1
|
513 | }
|
514 | # =>
|
515 | # i = 3
|
516 | # i = 4
|
517 | # i = 5
|
518 |
|
519 | ### `if elif` Conditional
|
520 |
|
521 | If statements test the exit code of a command, and have optional `elif` and
|
522 | `else` clauses:
|
523 |
|
524 | if test --file foo {
|
525 | echo 'foo is a file'
|
526 | rm --verbose foo # delete it
|
527 | } elif test --dir foo {
|
528 | echo 'foo is a directory'
|
529 | } else {
|
530 | echo 'neither'
|
531 | }
|
532 |
|
533 | Invert the exit code with `!`:
|
534 |
|
535 | if ! grep alice /etc/passwd {
|
536 | echo 'alice is not a user'
|
537 | }
|
538 |
|
539 | As with `while` loops, the condition can also be an **expression** wrapped in
|
540 | `()`:
|
541 |
|
542 | if (num_beans > 0) {
|
543 | echo 'so many beans'
|
544 | }
|
545 |
|
546 | var done = false
|
547 | if (not done) { # negate with 'not' operator (contrast with !)
|
548 | echo "we aren't done"
|
549 | }
|
550 |
|
551 | ### `case` Conditional
|
552 |
|
553 | The case statement is a series of conditionals and executable blocks. The
|
554 | condition can be either an unquoted glob pattern like `*.py`, an eggex pattern
|
555 | like `/d+/`, or a typed expression like `(42)`:
|
556 |
|
557 | var s = 'README.md'
|
558 | case (s) {
|
559 | *.py { echo 'Python' }
|
560 | *.cc | *.h { echo 'C++' }
|
561 | * { echo 'Other' }
|
562 | }
|
563 | # => Other
|
564 |
|
565 | case (s) {
|
566 | / dot* '.md' / { echo 'Markdown' }
|
567 | (30 + 12) { echo 'the integer 42' }
|
568 | (else) { echo 'neither' }
|
569 | }
|
570 | # => Markdown
|
571 |
|
572 | <!-- TODO: document case on typed data -->
|
573 |
|
574 | (Shell style like `if foo; then ... fi` and `case $x in ... esac` is also legal,
|
575 | but discouraged in YSH code.)
|
576 |
|
577 | ### Error Handling
|
578 |
|
579 | If statements are also used for **error handling**. Builtins and external
|
580 | commands use this style:
|
581 |
|
582 | if ! test -d /bin {
|
583 | echo 'not a directory'
|
584 | }
|
585 |
|
586 | if ! cp foo /tmp {
|
587 | echo 'error copying' # any non-zero status
|
588 | }
|
589 |
|
590 | Procs use this style (because of shell's *disabled `errexit` quirk*):
|
591 |
|
592 | try {
|
593 | myproc
|
594 | }
|
595 | if failed {
|
596 | echo 'failed'
|
597 | }
|
598 |
|
599 | For a complete list of examples, see [YSH Error
|
600 | Handling](ysh-error.html). For design goals and a reference, see [YSH
|
601 | Fixes Shell's Error Handling](error-handling.html).
|
602 |
|
603 | #### `break`, `continue`, `return`, `exit`
|
604 |
|
605 | The `exit` **keyword** exits a process (it's not a shell builtin.) The other 3
|
606 | control flow keywords behave like they do in Python and JavaScript.
|
607 |
|
608 | ### Ruby-like Blocks
|
609 |
|
610 | Here's a builtin command that takes a literal block argument:
|
611 |
|
612 | shopt --unset errexit { # ignore errors
|
613 | cp ale /tmp
|
614 | cp bean /bin
|
615 | }
|
616 |
|
617 | Blocks are a special kind of typed argument passed to commands like `shopt`.
|
618 | Their type is `value.Command`.
|
619 |
|
620 | ### Shell-like `proc`
|
621 |
|
622 | You can define units of code with the `proc` keyword.
|
623 |
|
624 | proc mycopy (src, dest) {
|
625 | ### Copy verbosely
|
626 |
|
627 | mkdir -p $dest
|
628 | cp --verbose $src $dest
|
629 | }
|
630 |
|
631 | The `###` line is a "doc comment", and can be retrieved with `pp proc`. Simple
|
632 | procs like this are invoked like a shell command:
|
633 |
|
634 | touch log.txt
|
635 | mycopy log.txt /tmp # first word 'mycopy' is a proc
|
636 |
|
637 | Procs have more features, including **four** kinds of arguments:
|
638 |
|
639 | 1. Word args (which are always strings)
|
640 | 1. Typed, positional args (aka positional args)
|
641 | 1. Typed, named args (aka named args)
|
642 | 1. A final block argument, which may be written with `{ }`.
|
643 |
|
644 | At the call site, they can look like any of these forms:
|
645 |
|
646 | cd /tmp # word arg
|
647 |
|
648 | json write (d) # word arg, then positional arg
|
649 |
|
650 | # error 'failed' (status=9) # word arg, then named arg
|
651 |
|
652 | cd /tmp { echo $PWD } # word arg, then block arg
|
653 |
|
654 | var mycmd = ^(echo hi) # expression for a value.Command
|
655 | eval (mycmd) # positional arg
|
656 |
|
657 | <!-- TODO: lazy arg list: ls8 | where [age > 10] -->
|
658 |
|
659 | At the definition site, the kinds of parameters are separated with `;`, similar
|
660 | to the Julia language:
|
661 |
|
662 | proc p2 (word1, word2; pos1, pos2, ...rest_pos) {
|
663 | echo "$word1 $word2 $[pos1 + pos2]"
|
664 | json write (rest_pos)
|
665 | }
|
666 |
|
667 | proc p3 (w ; ; named1, named2, ...rest_named; block) {
|
668 | echo "$w $[named1 + named2]"
|
669 | eval (block)
|
670 | json write (rest_named)
|
671 | }
|
672 |
|
673 | proc p4 (; ; ; block) {
|
674 | eval (block)
|
675 | }
|
676 |
|
677 | YSH also has Python-like functions defined with `func`. These are part of the
|
678 | expression language, which we'll see later.
|
679 |
|
680 | For more info, see the [Informal Guide to Procs and Funcs](proc-func.html)
|
681 | (under construction).
|
682 |
|
683 | #### Builtin Commands
|
684 |
|
685 | **Shell builtins** like `cd` and `read` are the "standard library" of the
|
686 | command language. Each one takes various flags:
|
687 |
|
688 | cd -L . # follow symlinks
|
689 |
|
690 | echo foo | read --all # read all of stdin
|
691 |
|
692 | Here are some categories of builtin:
|
693 |
|
694 | - I/O: `echo write read`
|
695 | - File system: `cd test`
|
696 | - Processes: `fork wait forkwait exec`
|
697 | - Interpreter settings: `shopt shvar`
|
698 | - Meta: `command builtin runproc type eval`
|
699 |
|
700 | <!-- TODO: Link to a comprehensive list of builtins -->
|
701 |
|
702 | ## Expression Language: Python-like Types
|
703 |
|
704 | YSH expressions look and behave more like Python or JavaScript than shell. For
|
705 | example, we write `if (x < y)` instead of `if [ $x -lt $y ]`. Expressions are
|
706 | usually surrounded by `( )`.
|
707 |
|
708 | At runtime, variables like `x` and `y` are bounded to **typed data**, like
|
709 | integers, floats, strings, lists, and dicts.
|
710 |
|
711 | <!--
|
712 | [Command vs. Expression Mode](command-vs-expression-mode.html) may help you
|
713 | understand how YSH is parsed.
|
714 | -->
|
715 |
|
716 | ### Python-like `func`
|
717 |
|
718 | At the end of the *Command Language*, we saw that procs are shell-like units of
|
719 | code. Now let's talk about Python-like **functions** in YSH, which are
|
720 | different than `procs`:
|
721 |
|
722 | - They're defined with the `func` keyword.
|
723 | - They're called in expressions, not in commands.
|
724 | - They're **pure**, and live in the **interior** of a process.
|
725 | - In contrast, procs usually perform I/O, and have **exterior** boundaries.
|
726 |
|
727 | Here's a function that mutates its argument:
|
728 |
|
729 | func popTwice(mylist) {
|
730 | call mylist->pop()
|
731 | call mylist->pop()
|
732 | }
|
733 |
|
734 | var mylist = [3, 4]
|
735 |
|
736 | # The call keyword is an "adapter" between commands and expressions,
|
737 | # like the = keyword.
|
738 | call popTwice(mylist)
|
739 |
|
740 | Here's a pure function:
|
741 |
|
742 | func myRepeat(s, n; special=false) { # positional; named params
|
743 | var parts = []
|
744 | for i in (0 .. n) {
|
745 | append $s (parts)
|
746 | }
|
747 | var result = join(parts)
|
748 |
|
749 | if (special) {
|
750 | return ("$result !!") # parens required for typed return
|
751 | } else {
|
752 | return (result)
|
753 | }
|
754 | }
|
755 |
|
756 | echo $[myRepeat('z', 3)] # => zzz
|
757 |
|
758 | echo $[myRepeat('z', 3, special=true)] # => zzz !!
|
759 |
|
760 | Funcs are named using `camelCase`, while procs use `kebab-case`. See the
|
761 | [Style Guide](style-guide.html) for more conventions.
|
762 |
|
763 | #### Builtin Functions
|
764 |
|
765 | In addition, to builtin commands, YSH has Python-like builtin **functions**.
|
766 | These are like the "standard library" for the expression language. Examples:
|
767 |
|
768 | - Functions that take multiple types: `len() type()`
|
769 | - Conversions: `bool() int() float() str() list() ...`
|
770 | - Explicit word evaluation: `split() join() glob() maybe()`
|
771 |
|
772 | <!-- TODO: Make a comprehensive list of func builtins. -->
|
773 |
|
774 |
|
775 | ### Data Types: `Int`, `Str`, `List`, `Dict`, ...
|
776 |
|
777 | YSH has data types, each with an expression syntax and associated methods.
|
778 |
|
779 | ### Methods
|
780 |
|
781 | Mutating methods are looked up with a thin arrow `->`:
|
782 |
|
783 | var foods = ['ale', 'bean']
|
784 | var last = foods->pop() # bean
|
785 | write @foods # => ale
|
786 |
|
787 | You can ignore the return value with the `call` keyword:
|
788 |
|
789 | call foods->pop()
|
790 |
|
791 | Transforming methods use a fat arrow `=>`:
|
792 |
|
793 | var line = ' ale bean '
|
794 | var trimmed = line => trim() => upper() # 'ALE BEAN'
|
795 |
|
796 | If the `=>` operator doesn't find a method with the given name in the object's
|
797 | type, it looks for free functions:
|
798 |
|
799 | # list() is a free function taking one arg
|
800 | # join() is a free function taking two args
|
801 | var x = {k1: 42, k2: 43} => list() => join('/') # 'K1/K2'
|
802 |
|
803 | This allows a left-to-right "method chaining" style.
|
804 |
|
805 | ---
|
806 |
|
807 | Now let's go through the data types in YSH. We'll show the syntax for
|
808 | literals, and what **methods** they have.
|
809 |
|
810 | #### Null and Bool
|
811 |
|
812 | YSH uses JavaScript-like spellings these three "atoms":
|
813 |
|
814 | var x = null
|
815 |
|
816 | var b1, b2 = true, false
|
817 |
|
818 | if (b1) {
|
819 | echo 'yes'
|
820 | } # => yes
|
821 |
|
822 |
|
823 | #### Int
|
824 |
|
825 | There are many ways to write integers:
|
826 |
|
827 | var small, big = 42, 65_536
|
828 | echo "$small $big" # => 42 65536
|
829 |
|
830 | var hex, octal, binary = 0x0001_0000, 0o755, 0b0001_0101
|
831 | echo "$hex $octal $binary" # => 65536 493 21
|
832 |
|
833 | <!--
|
834 | "Runes" are integers that represent Unicode code points. They're not common in
|
835 | YSH code, but can make certain string algorithms more readable.
|
836 |
|
837 | # Pound rune literals are similar to ord('A')
|
838 | const a = #'A'
|
839 |
|
840 | # Backslash rune literals can appear outside of quotes
|
841 | const newline = \n # Remember this is an integer
|
842 | const backslash = \\ # ditto
|
843 |
|
844 | # Unicode rune literal is syntactic sugar for 0x3bc
|
845 | const mu = \u{3bc}
|
846 |
|
847 | echo "chars $a $newline $backslash $mu" # => chars 65 10 92 956
|
848 | -->
|
849 |
|
850 | #### Float
|
851 |
|
852 | Floats are written like you'd expect:
|
853 |
|
854 | var small = 1.5e-10
|
855 | var big = 3.14
|
856 |
|
857 | #### Str
|
858 |
|
859 | See the section above called *Three Kinds of String Literals*. It described
|
860 | `'single quoted'`, `"double ${quoted}"`, and `u'J8-style\n'` strings; as well
|
861 | as their multiline variants.
|
862 |
|
863 | Strings are UTF-8 encoded in memory, like strings in the [Go
|
864 | language](https://golang.org). There isn't a separate string and unicode type,
|
865 | as in Python.
|
866 |
|
867 | Strings are **immutable**, as in Python and JavaScript. This means they only
|
868 | have **transforming** methods:
|
869 |
|
870 | var x = s => trim()
|
871 |
|
872 | Other methods:
|
873 |
|
874 | - `trimLeft() trimRight()`
|
875 | - `trimPrefix() trimSuffix()`
|
876 | - `upper() lower()` (not implemented)
|
877 |
|
878 | <!--
|
879 | The syntax `:symbol` could be an interned string.
|
880 | -->
|
881 |
|
882 | #### List (and Arrays)
|
883 |
|
884 | All lists can be expressed with Python-like literals:
|
885 |
|
886 | var foods = ['ale', 'bean', 'corn']
|
887 | var recursive = [1, [2, 3]]
|
888 |
|
889 | As a special case, list of strings are called **arrays**. It's often more
|
890 | convenient to write them with shell-like literals:
|
891 |
|
892 | # No quotes or commas
|
893 | var foods = :| ale bean corn |
|
894 |
|
895 | # You can use the word language here
|
896 | var other = :| foo $s *.py {alice,bob}@example.com |
|
897 |
|
898 | Lists are **mutable**, as in Python and JavaScript. So they mainly have
|
899 | mutating methods:
|
900 |
|
901 | call foods->reverse()
|
902 | write -- @foods
|
903 | # =>
|
904 | # corn
|
905 | # bean
|
906 | # ale
|
907 |
|
908 | #### Dict
|
909 |
|
910 | Dicts use syntax that's more like JavaScript than Python. Here's a dict
|
911 | literal:
|
912 |
|
913 | var d = {
|
914 | name: 'bob', # unquoted keys are allowed
|
915 | age: 42,
|
916 | 'key with spaces': 'val'
|
917 | }
|
918 |
|
919 | There are two syntaxes for key lookup. If the key doesn't exist, it's a fatal
|
920 | error.
|
921 |
|
922 | var v1 = d['name']
|
923 | var v2 = d.name # shorthand for the above
|
924 | var v3 = d['key with spaces'] # no shorthand for this
|
925 |
|
926 | Keys names can be computed with expressions in `[]`:
|
927 |
|
928 | var key = 'alice'
|
929 | var d2 = {[key ++ '_z']: 'ZZZ'} # Computed key name
|
930 | echo $[d2.alice_z] # => ZZZ # Reminder: expression sub
|
931 |
|
932 | Omitting the value causes it to be taken from a variable of the same name:
|
933 |
|
934 | var d3 = {key} # value is taken from the environment
|
935 | echo "name is $[d3.key]" # => name is alice
|
936 |
|
937 | More:
|
938 |
|
939 | var empty = {}
|
940 | echo $[len(empty)] # => 0
|
941 |
|
942 | Dicts are **mutable**, as in Python and JavaScript. But the `keys()` and `values()`
|
943 | methods return new `List` objects:
|
944 |
|
945 | var keys = d2 => keys() # => alice_z
|
946 | # var vals = d3 => values() # => alice
|
947 |
|
948 | ### `Place` type / "out params"
|
949 |
|
950 | The `read` builtin can either set an implicit variable `_reply`:
|
951 |
|
952 | whoami | read --all # sets _reply
|
953 |
|
954 | Or you can pass a `value.Place`, created with `&`
|
955 |
|
956 | var x # implicitly initialized to null
|
957 | whoami | read --all (&x) # mutate this "place"
|
958 | echo who=$x # => who=andy
|
959 |
|
960 | #### Quotation Types: value.Command (Block) and value.Expr
|
961 |
|
962 | These types are for reflection on YSH code. Most YSH programs won't use them
|
963 | directly.
|
964 |
|
965 | - `Command`: an unevaluated code block.
|
966 | - rarely-used literal: `^(ls | wc -l)`
|
967 | - `Expr`: an unevaluated expression.
|
968 | - rarely-used literal: `^[42 + a[i]]`
|
969 |
|
970 | <!-- TODO: implement Block, Expr, ArgList types (variants of value) -->
|
971 |
|
972 | ### Operators
|
973 |
|
974 | Operators are generally the same as in Python:
|
975 |
|
976 | if (10 <= num_beans and num_beans < 20) {
|
977 | echo 'enough'
|
978 | } # => enough
|
979 |
|
980 | YSH has a few operators that aren't in Python. Equality can be approximate or
|
981 | exact:
|
982 |
|
983 | var n = ' 42 '
|
984 | if (n ~== 42) {
|
985 | echo 'equal after stripping whitespace and type conversion'
|
986 | } # => equal after stripping whitespace type conversion
|
987 |
|
988 | if (n === 42) {
|
989 | echo "not reached because strings and ints aren't equal"
|
990 | }
|
991 |
|
992 | <!-- TODO: is n === 42 a type error? -->
|
993 |
|
994 | Pattern matching can be done with globs (`~~` and `!~~`)
|
995 |
|
996 | const filename = 'foo.py'
|
997 | if (filename ~~ '*.py') {
|
998 | echo 'Python'
|
999 | } # => Python
|
1000 |
|
1001 | if (filename !~~ '*.sh') {
|
1002 | echo 'not shell'
|
1003 | } # => not shell
|
1004 |
|
1005 | or regular expressions (`~` and `!~`). See the Eggex section below for an
|
1006 | example of the latter.
|
1007 |
|
1008 | Concatenation is `++` rather than `+` because it avoids confusion in the
|
1009 | presence of type conversion:
|
1010 |
|
1011 | var n = 42 + 1 # string plus int does implicit conversion
|
1012 | echo $n # => 43
|
1013 |
|
1014 | var y = 'ale ' ++ "bean $n" # concatenation
|
1015 | echo $y # => ale bean 43
|
1016 |
|
1017 | <!--
|
1018 | TODO: change example above
|
1019 | var n = '42' + 1 # string plus int does implicit conversion
|
1020 | -->
|
1021 |
|
1022 | <!--
|
1023 |
|
1024 | #### Summary of Operators
|
1025 |
|
1026 | - Arithmetic: `+ - * / // %` and `**` for exponentatiation
|
1027 | - `/` always yields a float, and `//` is integer division
|
1028 | - Bitwise: `& | ^ ~`
|
1029 | - Logical: `and or not`
|
1030 | - Comparison: `== < > <= >= in 'not in'`
|
1031 | - Approximate equality: `~==`
|
1032 | - Eggex and glob match: `~ !~ ~~ !~~`
|
1033 | - Ternary: `1 if x else 0`
|
1034 | - Index and slice: `mylist[3]` and `mylist[1:3]`
|
1035 | - `mydict->key` is a shortcut for `mydict['key']`
|
1036 | - Function calls
|
1037 | - free: `f(x, y)`
|
1038 | - transformations and chaining: `s => startWith('prefix')`
|
1039 | - mutating methods: `mylist->pop()`
|
1040 | - String and List: `++` for concatenation
|
1041 | - This is a separate operator because the addition operator `+` does
|
1042 | string-to-int conversion
|
1043 |
|
1044 | TODO: What about list comprehensions?
|
1045 | -->
|
1046 |
|
1047 | ### Egg Expressions (YSH Regexes)
|
1048 |
|
1049 | An *Eggex* is a type of YSH expression that denote regular expressions. They
|
1050 | translate to POSIX ERE syntax, for use with tools like `egrep`, `awk`, and `sed
|
1051 | --regexp-extended` (GNU only).
|
1052 |
|
1053 | They're designed to be readable and composable. Example:
|
1054 |
|
1055 | var D = / digit{1,3} /
|
1056 | var ip_pattern = / D '.' D '.' D '.' D'.' /
|
1057 |
|
1058 | var z = '192.168.0.1'
|
1059 | if (z ~ ip_pattern) { # Use the ~ operator to match
|
1060 | echo "$z looks like an IP address"
|
1061 | } # => 192.168.0.1 looks like an IP address
|
1062 |
|
1063 | if (z !~ / '.255' %end /) {
|
1064 | echo "doesn't end with .255"
|
1065 | } # => doesn't end with .255"
|
1066 |
|
1067 | See the [Egg Expressions doc](eggex.html) for details.
|
1068 |
|
1069 | ## Interlude
|
1070 |
|
1071 | Let's review what we've seen before moving onto other YSH features.
|
1072 |
|
1073 | ### Three Interleaved Languages
|
1074 |
|
1075 | Here are the languages we saw in the last 3 sections:
|
1076 |
|
1077 | 1. **Words** evaluate to a string, or list of strings. This includes:
|
1078 | - literals like `'mystr'`
|
1079 | - substitutions like `${x}` and `$(hostname)`
|
1080 | - globs like `*.sh`
|
1081 | 2. **Commands** are used for
|
1082 | - I/O: pipelines, builtins like `read`
|
1083 | - control flow: `if`, `for`
|
1084 | - abstraction: `proc`
|
1085 | 3. **Expressions** on typed data are borrowed from Python, with some JavaScript
|
1086 | influence.
|
1087 | - Lists: `['ale', 'bean']` or `:| ale bean |`
|
1088 | - Dicts: `{name: 'bob', age: 42}`
|
1089 | - Functions: `split('ale bean')` and `join(['pea', 'nut'])`
|
1090 |
|
1091 | ### How Do They Work Together?
|
1092 |
|
1093 | Here are two examples:
|
1094 |
|
1095 | (1) In this this *command*, there are **four** *words*. The fourth word is an
|
1096 | *expression sub* `$[]`.
|
1097 |
|
1098 | write hello $name $[d['age'] + 1]
|
1099 | # =>
|
1100 | # hello
|
1101 | # world
|
1102 | # 43
|
1103 |
|
1104 | (2) In this assignment, the *expression* on the right hand side of `=`
|
1105 | concatenates two strings. The first string is a literal, and the second is a
|
1106 | *command sub*.
|
1107 |
|
1108 | var food = 'ale ' ++ $(echo bean | tr a-z A-Z)
|
1109 | write $food # => ale BEAN
|
1110 |
|
1111 | So words, commands, and expressions are **mutually recursive**. If you're a
|
1112 | conceptual person, skimming [Syntactic Concepts](syntactic-concepts.html) may
|
1113 | help you understand this on a deeper level.
|
1114 |
|
1115 | <!--
|
1116 | One way to think about these sublanguages is to note that the `|` character
|
1117 | means something different in each context:
|
1118 |
|
1119 | - In the command language, it's the pipeline operator, as in `ls | wc -l`
|
1120 | - In the word language, it's only valid in a literal string like `'|'`, `"|"`,
|
1121 | or `\|`. (It's also used in `${x|html}`, which formats a string.)
|
1122 | - In the expression language, it's the bitwise OR operator, as in Python and
|
1123 | JavaScript.
|
1124 | -->
|
1125 |
|
1126 | ## Languages for Data (Interchange Formats)
|
1127 |
|
1128 | In addition to languages for **code**, YSH also deals with languages for
|
1129 | **data**. [JSON]($xref) is a prominent example of the latter.
|
1130 |
|
1131 | <!-- TODO: Link to slogans, fallacies, and concepts -->
|
1132 |
|
1133 | ### UTF-8
|
1134 |
|
1135 | UTF-8 is the foundation of our textual data languages.
|
1136 |
|
1137 | <!-- TODO: there's a runes() iterator which gives integer offsets, usable for
|
1138 | slicing -->
|
1139 |
|
1140 | <!-- TODO: write about J8 notation -->
|
1141 |
|
1142 | ### Lines of Text (traditional), and JSON/J8 Strings
|
1143 |
|
1144 | Traditional Unix tools like `grep` and `awk` operate on streams of lines. YSH
|
1145 | supports this style, just like any other shell.
|
1146 |
|
1147 | But YSH also has [J8 Notation][], a data format based on [JSON][].
|
1148 |
|
1149 | [J8 Notation]: j8-notation.html
|
1150 |
|
1151 | It lets you encode arbitrary byte strings into a single (readable) line,
|
1152 | including those with newlines and terminal escape sequences.
|
1153 |
|
1154 | Example:
|
1155 |
|
1156 | # A line with a tab char in the middle
|
1157 | var mystr = u'pea\t' ++ u'42\n'
|
1158 |
|
1159 | # Print it as JSON
|
1160 | write $[toJson(mystr)] # => "pea\t42\n"
|
1161 |
|
1162 | # JSON8 is the same, but it's not lossy for binary data
|
1163 | write $[toJson8(mystr)] # => "pea\t42\n"
|
1164 |
|
1165 | ### Structured: JSON8, TSV8
|
1166 |
|
1167 | You can write and read **tree-shaped** as [JSON][]:
|
1168 |
|
1169 | var d = {key: 'value'}
|
1170 | json write (d) # dump variable d as JSON
|
1171 | # =>
|
1172 | # {
|
1173 | # "key": "value"
|
1174 | # }
|
1175 |
|
1176 | echo '["ale", 42]' > example.json
|
1177 |
|
1178 | json read (&d2) < example.json # parse JSON into var d2
|
1179 | pp cell d2 # inspect the in-memory value
|
1180 | # =>
|
1181 | # ['ale', 42]
|
1182 |
|
1183 | [JSON][] will lose information when strings have binary data, but the slight
|
1184 | [JSON8]($xref) upgrade won't:
|
1185 |
|
1186 | var b = {binary: $'\xff'}
|
1187 | json8 write (b)
|
1188 | # =>
|
1189 | # {
|
1190 | # "binary": b'\yff'
|
1191 | # }
|
1192 |
|
1193 | [JSON]: $xref
|
1194 |
|
1195 | <!--
|
1196 | TODO:
|
1197 | - Fix pp cell output
|
1198 | - Use json write (d) syntax
|
1199 | -->
|
1200 |
|
1201 | **Table-shaped** data can be read and written as [TSV8]($xref). (TODO: not yet
|
1202 | implemented.)
|
1203 |
|
1204 | <!-- Figure out the API. Does it work like JSON?
|
1205 |
|
1206 | Or I think we just implement
|
1207 | - rows: 'where' or 'filter' (dplyr)
|
1208 | - cols: 'select' conflicts with shell builtin; call it 'cols'?
|
1209 | - sort: 'sort-by' or 'arrange' (dplyr)
|
1210 | - TSV8 <=> sqlite conversion. Are these drivers or what?
|
1211 | - and then let you pipe output?
|
1212 |
|
1213 | Do we also need TSV8 space2tab or something? For writing TSV8 inline.
|
1214 |
|
1215 | More later:
|
1216 | - MessagePack (e.g. for shared library extension modules)
|
1217 | - msgpack read, write? I think user-defined function could be like this?
|
1218 | - SASH: Simple and Strict HTML? For easy processing
|
1219 | -->
|
1220 |
|
1221 | ## The Runtime Shared by OSH and YSH
|
1222 |
|
1223 | Although we describe OSH and YSH as different languages, they use the **same**
|
1224 | interpreter under the hood. This interpreter has various `shopt` flags that
|
1225 | are flipped for different behavior, e.g. with `shopt --set ysh:all`.
|
1226 |
|
1227 | Understanding this interpreter and its interface to the Unix kernel will help
|
1228 | you understand **both** languages!
|
1229 |
|
1230 | ### Interpreter Data Model
|
1231 |
|
1232 | The [Interpreter State](interpreter-state.html) doc is **under construction**.
|
1233 | It will cover:
|
1234 |
|
1235 | - Two separate namespaces (like Lisp 1 vs. 2):
|
1236 | - **proc** namespace for procs as the first word
|
1237 | - **variable** namespace
|
1238 | - The variable namespace has a **call stack**, for the local variables of a
|
1239 | proc.
|
1240 | - Each **stack frame** is a `{name -> cell}` mapping.
|
1241 | - A **cell** has one of the above data types: `Bool`, `Int`, `Str`, etc.
|
1242 | - A cell has `readonly`, `export`, and `nameref` **flags**.
|
1243 | - Boolean shell options with `shopt`: `parse_paren`, `simple_word_eval`, etc.
|
1244 | - String shell options with `shvar`: `IFS`, `PATH`
|
1245 | - **Registers** that are silently modified by the interpreter
|
1246 | - `$?` and `_error`
|
1247 | - `$!` for the last PID
|
1248 | - `_this_dir`
|
1249 | - `_reply`
|
1250 |
|
1251 | ### Process Model (the kernel)
|
1252 |
|
1253 | The [Process Model](process-model.html) doc is **under construction**. It will cover:
|
1254 |
|
1255 | - Simple Commands, `exec`
|
1256 | - Pipelines. #[shell-the-good-parts](#blog-tag)
|
1257 | - `fork`, `forkwait`
|
1258 | - Command and process substitution.
|
1259 | - Related links:
|
1260 | - [Tracing execution in Oils](xtrace.html) (xtrace), which divides
|
1261 | process-based concurrency into **synchronous** and **async** constructs.
|
1262 | - [Three Comics For Understanding Unix
|
1263 | Shell](http://www.oilshell.org/blog/2020/04/comics.html) (blog)
|
1264 |
|
1265 |
|
1266 | <!--
|
1267 | Process model additions: Capers, Headless shell
|
1268 |
|
1269 | some optimizations: See YSH starts fewer processes than other shells.
|
1270 | -->
|
1271 |
|
1272 | ## Summary
|
1273 |
|
1274 | YSH is a large language that evolved from Unix shell. It has shell-like
|
1275 | commands, Python-like expressions on typed data, and Ruby-like command blocks.
|
1276 |
|
1277 | Even though it's large, you can "forget" the bad parts of shell like `[ $x -lt
|
1278 | $y ]`.
|
1279 |
|
1280 | These concepts are central to YSH:
|
1281 |
|
1282 | 1. Interleaved *word*, *command*, and *expression* languages.
|
1283 | 2. A standard library of *shell builtins*, as well as *builtin functions*
|
1284 | 3. Languages for *data*: J8 Notation, including JSON8 and TSV8
|
1285 | 4. A *runtime* shared by OSH and YSH
|
1286 |
|
1287 | ## Related Docs
|
1288 |
|
1289 | - [YSH vs. Shell Idioms](idioms.html) - YSH side-by-side with shell.
|
1290 | - [YSH Language Influences](language-influences.html) - In addition to shell,
|
1291 | Python, and JavaScript, YSH is influenced by Ruby, Perl, Awk, PHP, and more.
|
1292 | - [A Feel For YSH Syntax](syntax-feelings.html) - Some thoughts that may help
|
1293 | you remember the syntax.
|
1294 | - [YSH Language Warts](warts.html) documents syntax that may be surprising.
|
1295 |
|
1296 | ## Appendix: Features Not Shown
|
1297 |
|
1298 | ### Advanced
|
1299 |
|
1300 | These shell features are part of YSH, but aren't shown for brevity.
|
1301 |
|
1302 | - The `fork` and `forkwait` builtins, for concurrent execution and subshells.
|
1303 | - Process Substitution: `diff <(sort left.txt) <(sort right.txt)`
|
1304 |
|
1305 | ### Deprecated Shell Constructs
|
1306 |
|
1307 | The shared interpreter supports many shell constructs that are deprecated:
|
1308 |
|
1309 | - YSH code uses shell's `||` and `&&` in limited circumstances, since `errexit`
|
1310 | is on by default.
|
1311 | - Assignment builtins like `local` and `declare`. Use YSH keywords.
|
1312 | - Boolean expressions like `[[ x =~ $pat ]]`. Use YSH expressions.
|
1313 | - Shell arithmetic like `$(( x + 1 ))` and `(( y = x ))`. Use YSH expressions.
|
1314 | - The `until` loop can always be replaced with a `while` loop
|
1315 | - Most of what's in `${}` can be written in other ways. For example
|
1316 | `${s#/tmp}` could be `s => removePrefix('/tmp')` (TODO).
|
1317 |
|
1318 | ### Not Yet Implemented
|
1319 |
|
1320 | This document mentions a few constructs that aren't yet implemented. Here's a
|
1321 | summary:
|
1322 |
|
1323 | ```none
|
1324 | # Unimplemented syntax:
|
1325 |
|
1326 | echo ${x|html} # formatters
|
1327 |
|
1328 | echo ${x %.2f} # statically-parsed printf
|
1329 |
|
1330 | var x = j"line\n"
|
1331 | echo j"line\n" # JSON-style string literal
|
1332 |
|
1333 | var x = "<p>$x</p>"html
|
1334 | echo "<p>$x</p>"html # tagged string
|
1335 |
|
1336 | var x = 15 Mi # units suffix
|
1337 | ```
|
1338 |
|
1339 | Important builtins that aren't implemented:
|
1340 |
|
1341 | - `describe` for testing
|
1342 | - `parseArgs()` to parse flags
|
1343 | - Builtins for [TSV8]($xref) - selection, projection, sorting
|
1344 |
|
1345 | <!--
|
1346 |
|
1347 | - To document: Method calls
|
1348 | - To implement: Capers: stateless coprocesses
|
1349 | -->
|
1350 |
|
1351 | ## Appendix: Example of an YSH Module
|
1352 |
|
1353 | YSH can be used to write simple "shell scripts" or longer programs. It has
|
1354 | *procs* and *modules* to help with the latter.
|
1355 |
|
1356 | A module is just a file, like this:
|
1357 |
|
1358 | ```
|
1359 | #!/usr/bin/env ysh
|
1360 | ### Deploy script
|
1361 |
|
1362 | source-guard main || return 0 # declaration, "include guard"
|
1363 |
|
1364 | source $_this_dir/lib/util.ysh # defines 'log' helper
|
1365 |
|
1366 | const DEST = '/tmp/ysh-tour'
|
1367 |
|
1368 | proc my-sync(...files) {
|
1369 | ### Sync files and show which ones
|
1370 |
|
1371 | cp --verbose @files $DEST
|
1372 | }
|
1373 |
|
1374 | proc main {
|
1375 | mkdir -p $DEST
|
1376 |
|
1377 | touch {foo,bar}.py {build,test}.sh
|
1378 |
|
1379 | log "Copying source files"
|
1380 | my-sync *.py *.sh
|
1381 |
|
1382 | if test --dir /tmp/logs {
|
1383 | cd /tmp/logs
|
1384 |
|
1385 | log "Copying logs"
|
1386 | my-sync *.log
|
1387 | }
|
1388 | }
|
1389 |
|
1390 | if is-main { # The only top-level statement
|
1391 | main @ARGV
|
1392 | }
|
1393 | ```
|
1394 |
|
1395 | <!--
|
1396 | TODO:
|
1397 | - Also show flags parsing?
|
1398 | - Show longer examples where it isn't boilerplate
|
1399 | -->
|
1400 |
|
1401 | You wouldn't bother with the boilerplate for something this small. But this
|
1402 | example illustrates the idea, which is that the top level often contains these
|
1403 | words: `proc`, `const`, `module`, `source`, and `use`.
|
1404 |
|