OILS / doc / options.md View on Github | oilshell.org

318 lines, 219 significant
1---
2in_progress: yes
3default_highlighter: oils-sh
4---
5
6Global Shell Options: Turning OSH into YSH
7==========================================
8
9
10This document describes global shell options, which look like this:
11
12 shopt --set strict_backslash # YSH style
13 shopt --set ysh:upgrade # A whole group of options
14 set -o errexit # Bourne shell style
15
16They can affect parsing or execution, and are used to gradually turn the
17[OSH]($xref:OSH) into the [YSH]($xref:YSH).
18
19For example, YSH doesn't have word splitting on whitespace. Instead, it use
20[Simple Word Evaluation](simple-word-eval.html). (Blog: [Oil Doesn't Require
21Quoting
22Everywhere](https://www.oilshell.org/blog/2021/04/simple-word-eval.html)). (Until 2023, YSH was called the "Oil language".)
23
24This isn't the **only** use for options, but it's an important one.
25
26<div id="toc">
27</div>
28
29## What Every User Should Know (2 minutes)
30
31When you run `bin/osh`, the **option groups** `strict:all` and `ysh:upgrade` are
32"canned settings" that relieve you of having to know about dozens of shell
33options.
34
35Running `bin/ysh` is equivalent to using `shopt --set ysh:all` in `bin/osh`.
36
37Let's look at three examples.
38
39### Strict
40
41If you put this line at the top of your shell script, it will still **run under
42other shells**, but OSH will act as sort of a "runtime linter":
43
44 # Abort on more errors, but fixes will still be compatible
45 shopt -s strict:all 2>/dev/null || true
46
47### Upgrade
48
49If you want to upgrade a script, and don't care about running under other
50shells, use this:
51
52 # Start enabling YSH syntax and semantics
53 shopt --set ysh:upgrade
54
55This second line may break a few things, but is designed to be an easy upgrade.
56See [What Breaks When You Upgrade to YSH](upgrade-breakage.html).
57
58### YSH
59
60If you're writing a new script, you can use `bin/ysh` to get **all**
61enhancements. Typically you use a shebang line like this:
62
63 #!/usr/bin/env ysh
64
65That's all most users need to know. For more details, see the wiki page:
66[Gradually Upgrading Shell to Oil]($wiki).
67
68## Using Shell Options
69
70There are several different ways of using shell options.
71
72### Preferred Style
73
74YSH has **long flags** for readability, which are preferred:
75
76 shopt --set errexit
77 shopt --unset errexit
78
79It also allows **scoped** options:
80
81 shopt --unset errexit {
82 false # non-zero status ignored
83 ls /bad
84 }
85 false # original setting restored
86
87### Bourne Shell Style
88
89For compatibility, these styles works in YSH:
90
91 set -e # abort script on non-zero exit exit code
92 set +e # turn it off
93
94 set -o errexit # a more readable version of the above
95 set +o errexit
96
97[Bash]($xref:bash)-style option with `shopt`:
98
99 shopt -s nullglob # turn it on
100 shopt -u nullglob # turn it off
101
102### Setting Options Via Command Line Flags
103
104You typically invoke the `shopt` builtin at the top of a script, but you
105can also set options at the command line:
106
107 osh -O errexit -c 'shopt -p -o' # turn on Bourne option
108 osh +O errexit -c 'shopt -p -o' # turn off Bourne option
109
110 osh -O strict_tilde -c 'shopt -p' # turn on YSH option
111 osh +O strict_tilde -c 'shopt -p' # turn off YSH option
112
113### Inspecting Option State
114
115Shell has many ways to do this, like:
116
117 set -o # print all Bourne shell options
118 shopt -p # print all bash options
119 shopt -p nullglob failglob # print selected options
120 shopt -p ysh:upgrade # print options in the given group
121
122TODO: YSH should enable `shopt --print` for all options. It should have a flat
123list.
124
125## Kinds of Options, With Examples
126
127*Option groups* like `ysh:upgrade` are baked into the interpreter. What follows
128is an informal list of *kinds* of options, which are different categorization:
129
130- Groups: How much of YSH do you want to use?
131- Kinds: Does this option affect parsing behavior, runtime behavior, or
132 something else?
133
134### Naming Conventions
135
136- `parse_*`: Change parsing.
137 - enable new features: `parse_at`, `parse_equals`.
138 - turn off to reject bad or old code: `parse_backticks`, `parse_backslash`,
139 `parse_dollar`.
140- `strict_*`: Fail at runtime instead of ignoring the bug like bash.
141 - `${#s}` on invalid unicode is a runtime error.
142 - `~typo` is a runtime error.
143- `simple_*`: Break things to improve style.
144 - `simple_eval_builtin`, `simple_echo`.
145 - `simple_word_eval` is the most aggressive
146
147### Strict Options Produce More Errors
148
149These options produce more **programming errors**. Importantly, the resulting
150program is still compatible with other shells.
151
152For example, `shopt -s strict_array` produces runtime errors when you confuse
153strings and arrays. After you fix these problems, your program will still run
154correctly under `bash`.
155
156In contrast, if you set `shopt -s simple_word_eval` (an option that doesn't
157start with `strict_`), the semantics of your program have changed, and you can
158**no longer** run it under other shells. It's considered an "YSH option": by
159setting it, you're using parts of YSH.
160
161### Parse Options Change Syntax
162
163Options that affect parsing start with `parse_`. For example, `shopt -s
164parse_at` enables **splicing** with the `@` character:
165
166 var words = :| ale bean |
167 write -- @words
168 # =>
169 # ale
170 # bean
171
172and inline function calls:
173
174 write -- @[split('ale bean')]
175 # =>
176 # ale
177 # bean
178
179As another example, `shopt --set parse_brace` takes over the `{ }` characters.
180Specifically, it does three things:
181
1821. Allow builtins like `cd` to take a block (discussed in a [Zulip
183 thread](https://oilshell.zulipchat.com/#narrow/stream/121540-oil-discuss/topic/cd.20now.20takes.20a.20Ruby-like.20block))
1842. Control flow like `if`, `case`, `for`, and `while/until`, use curly brace
185 delimiters instead of `then/fi`, `do/done`, etc. See below.
1863. To remove confusion, braces must be balanced inside a word. echo `foo{` is
187 an error. It has to be `echo foo\{` or `echo 'foo{'`.
188 - In a correct brace expansion, they're always balanced: `{pea,coco}nut`
189 - This is so that the syntax errors are better when you forget a space.
190
191<!--
192Test cases start here: <https://github.com/oilshell/oil/blob/master/spec/oil-options.test.sh#L257>
193-->
194
195Here's idiomatic YSH syntax after `parse_brace`:
196
197 cd /tmp {
198 echo $PWD
199 }
200
201 if test -d foo {
202 echo 'dir'
203 } elif test -f foo {
204 echo 'file'
205 } else {
206 echo 'neither'
207 }
208
209 # Single line statements are supported:
210 if test -d / { echo 'dir' } else { echo 'nope' }
211
212 while true {
213 echo hi
214 break
215 }
216
217 # Loop over words
218 for x in ale bean *.sh {
219 echo $x
220 }
221
222 # Replace 'in' with {, and 'esac' with }
223 case $x {
224 *.py)
225 echo python
226 ;;
227 *.sh)
228 echo shell
229 ;;
230 }
231
232What's the motivation for this? Mainly familiarity: I hear a lot of feedback
233that nobody can remember how to write if statements in shell. See [The
234Simplest Explanation of
235Oil](//www.oilshell.org/blog/2020/01/simplest-explanation.html).
236
237<!--
238
239There are also **expression** variants of these constructs:
240
241 if (x > 0) {
242 echo hi
243 }
244
245 while (x > 0) {
246 echo hi
247 }
248
249(`for` and `case` to come later.)
250
251-->
252
253
254### Runtime Options Change Behavior
255
256- `simple_echo`. Changes the flags accepted by the `echo` builtin, and style of flag parsing.
257 See the `Builtins > echo` below.
258- `simple_word_eval`. Word evaluation consists of one stage rather than three:
259 - No word splitting or empty elision. (In other words, arity isn't data-dependent.)
260 - Static globbing, but no dynamic globbing. (In other words, data isn't re-parsed as code.)
261 - This option is intended to be implemented by other shells.
262
263TODO: copy examples from spec tests
264
265 echo $dir/*.py
266
267- `command_sub_errexit`. A error in a command sub can cause the **parent
268 shell** to exit fatally. Also see `inherit_errexit` and `strict_errexit`.
269
270## List of Options
271
272### Selected Options
273
274`strict_arith`. Strings that don't look like integers cause a fatal error in
275arithmetic expressions.
276
277`strict_argv`. Empty `argv` arrays are disallowed (because there's no
278practical use for them). For example, the second statement in `x=''; $x`
279results in a fatal error.
280
281`strict_array`. No implicit conversions between string an array. In other
282words, turning this on gives you a "real" array type.
283
284`strict_control_flow`. `break` and `continue` outside of a loop are fatal
285errors.
286
287`simple_eval_builtin`. The `eval` builtin takes exactly **one** argument. It
288doesn't concatenate its arguments with spaces, or accept zero arguments.
289
290`strict_word_eval`. More word evaluation errors are fatal.
291
292- String slices with negative arguments like `${s: -1}` and `${s: 1 : -1}`
293 result in a fatal error. (NOTE: In array slices, negative start indices are
294 allowed, but negative lengths are always fatal, regardless of
295 `strict_word_eval`.)
296- UTF-8 decoding errors are fatal when computing lengths (`${#s}`) and slices.
297
298For options affecting exit codes, see the [error handling
299doc](error-handling.html).
300
301### Complete List
302
303See the [Chapter on Global Shell Options](ref/chap-option.html) in the
304reference.
305
306## FAQ: Aren't Global Variables Bad?
307
308Options are technically globals, but YSH controls them in 2 ways:
309
3101. It has scoped mutation with Ruby-like [blocks](proc-block-func.html).
311 - Example: `shopt --unset errexit { false }`
3122. Like all Bourne shells, YSH uses process-based concurrency. It doesn't have
313 shared memory.
314
315## Related Documents
316
317- Up: [Interpreter State](interpreter-state.html), which is under construction
318