Why Sponsor Oils? | source | all docs for version 0.22.0 | all versions | oilshell.org
This document is for sophisticated shell users.
You're unlikely to encounter these incompatibilities in everyday shell usage. If you do, there's almost always a simple workaround, like adding a space or a backslash.
OSH is meant to run all POSIX shell programs, and most bash programs.
In other shells, printf %d invalid_integer
prints 0
and a warning. OSH
gives you a runtime error.
shopt -s eval_unsafe_arith
In shell, array locations are often dynamically parsed, and the index can have command subs, which execute arbitrary code.
For example, if you have code='a[$(echo 42 | tee PWNED)]'
, shells will parse
this data and execute it in many situations:
echo $(( code )) # dynamic parsing and evaluation in bash, mksh, zsh
unset $code
printf -v $code hi
echo ${!code}
OSH disallows this by default. If you want this behavior, you can turn on
shopt -s eval_unsafe_arith
.
Related: A 30-year-old security problem
This section describes differences related to static parsing. OSH avoids the dynamic parsing of most shells.
(Note: This section should encompass all the failures from the wild tests and spec tests.
Strings should be quoted inside array indices:
No:
"${SETUP_STATE[$err.cmd]}"
Yes:
"${SETUP_STATE["$err.cmd"]}"
When unquoted, the period causes an ambiguity with respect to regular arrays vs. associative arrays. See Parsing Bash is Undecidable (2016).
You can have a subshell in a command sub, but it usually doesn't make sense.
In OSH you need a space after $(
. The characters $((
always start an
arith sub.
No:
$((cd / && ls))
Yes:
$( (cd / && ls) ) # Valid but usually doesn't make sense.
$({ cd / && ls; }) # Use {} for grouping, not (). Note trailing ;
$(cd / && ls) # Even better
The OSH parser distinguishes these two constructs with a space:
[[ !(a == a) ]]
is an extended glob.[[ ! (a == a) ]]
is the negation of an equality test.In bash, the parsing of such expressions depends on shopt -s extglob
. In
OSH, shopt -s extglob
is accepted, but doesn't affect parsing.
Lines like EOF]
or EOF)
don't end here docs. The delimiter must be on its
own line.
No:
a=$(cat <<EOF
abc
EOF)
a=$(cat <<EOF
abc
EOF # this is not a comment; it makes the EOF delimiter invalid
)
Yes:
a=$(cat <<EOF
abc
EOF
) # this is actually a comment
Bash allows:
a[1 + 2 * 3]=value
OSH only allows:
a[1+2*3]=value
because it parses with limited lookahead. The first line would result in the
execution of a command named a[1
.
This means that they aren't "dynamic":
b=break
while true; do
$b # doesn't break in OSH
done
Static control flow will allow static analysis of shell scripts.
(Test cases are in spec/loop).
For example, append
is a builtin in OSH, but not in bash
. Use env append
or /path/to/append
if you want to run an external command.
(Note that a user-defined proc append
takes priority over the builtin
append
.)
In contrast with builtins, keywords affect shell parsing.
For example, func
is a keyword in OSH, but not in bash
. To run a command
named func
, use command func arg1
.
Note that all shells have extensions that cause this issue. For example, [[
is a keyword in bash
but not in POSIX shell.
These differences occur in subsequent stages of parsing, or in runtime parsing.
No:
{a,b}{ # what does the second { mean?
{a,b}{1...3} # 3 dots instead of 2
Yes:
{a,b}\{
{a,b}\{1...3\}
bash will do a partial expansion in the former cases, giving you a{ b{
and a{1...3} b{1...3}
.
OSH considers them syntax errors and aborts all brace expansion, giving you
the same thing back: {a,b}{
and {a,b}{1...3}
.
Don't use ambiguous syntax for a character class consisting of a single bracket character.
No:
echo [[]
echo []]
Yes:
echo [\[]
echo [\]]
The ambiguous syntax is allowed when we pass globs through to libc
, but it's
good practice to be explicit.
In bash, you can use [[
with -v
to test whether an array contains an entry:
declare -a array=('' foo)
if [[ -v array[1] ]]; then
echo 'exists'
fi # => exists
Likewise for an associative array:
declare -A assoc=([key]=value)
if [[ -v assoc['key'] ]]
echo 'exists'
fi # => exists
OSH currently treats these expressions as a string, which means the status will
be 1 (false
).
Workaround:
if [[ "${assoc['key']:+exists}" ]]; then
echo 'exists'
fi # => exists
In ysh, you can use:
var d = { key: 42 }
if ('key' in d) {
echo 'exists'
} # => exists
Most shells split the entries of arrays like "$@"
and "${a[@]}"
here:
echo ${undef:-"$@"}
In OSH, omit the quotes if you want splitting:
echo ${undef:-$@}
I think OSH is more consistent, but it disagrees with other shells.
declare -i -a -A
)Even though there's a large common subset, OSH and bash have a different model for typed data.
In particular,
-i
flag is a no-op in OSH. See Shell Idioms > Remove Dynamic
Parsing for alternatives to -i
.-a
and -A
flags behave differently. They pertain to the value, not
the location.For example, these two statements are different in bash, but the same in OSH:
declare -A assoc # unset cell that will LATER be an assoc array
declare -A assoc=() # empty associative array
In bash, you can tell the difference with set -u
, but there's no difference
in OSH.
Here is how you can create arrays in OSH, in a bash-compatible way:
local indexed=(foo bar)
local -a indexed=(foo bar) # -a is redundant
echo ${indexed[1]} # bar
local assoc=(['one']=1 ['two']=2)
local -A assoc=(['one']=1 ['two']=2) # -A is redundant
echo ${assoc['one']} # 1
In bash, the distinction between the two is blurry, with cases like this:
local -A x=(foo bar) # -A disagrees with literal
local -a y=(['one']=1 ['two']=2) # -a disagrees with literal
These are disallowed in OSH.
Notes:
=
keyword is useful for gaining an understanding of the data model.The assignment builtins are export
, readonly
, local
, and
declare
/typeset
. They're parsed in 2 ways:
declare x=$y
when $y
contains
spaces. bash and other shells behave this way.declare $1
where $1
is a=b
builtin declare x=$y
is a runtime errorThis is because the special parsing of x=$y
depends on the first word
declare
.
In bash, you can do unusual things with args to assignment builtins:
vars='a=b x=y'
touch foo=bar.py spam=eggs.py
declare $vars *.py # assigns at least 4 variables
echo $a # b
echo $x # y
echo $foo # bar.py
echo $spam # eggs.py
In contrast, OSH doesn't split or glob args to assignment builtins. This is more like the behavior of zsh.
shopt -s lastpipe
)In this pipeline, the builtin read
is run in the shell process, not a child
process:
$ echo hi | read x
$ echo x=$x
x=hi # empty in bash unless shopt -s lastpipe
If the last part is an external command, there is no difference:
$ ls | wc -l
42
This is how zsh behaves, and how bash (sometimes) behaves with shopt -s lastpipe
.
Because the last part may be the current shell process, the entire pipeline can't be suspended.
OSH and zsh share this consequence of the lastpipe
semantics.
In contrast, bash's shopt -s lastpipe
is ignored in interactive shells.
${PIPESTATUS[@]}
is only set after an actual pipelineThis makes it easier to check compound status codes without worrying about them being "clobbered".
Bash will set ${PIPESTATUS[@]}
on every command, regardless of whether its a
pipeline.
Almost all "real" aliases should work in OSH. But these don't work:
alias left='{'
left echo hi; }
(cases #33-#34 in spec/alias)
or
alias a=
a (( var = 0 ))
Details on the OSH parsing model:
SimpleCommand
are the only ones that are further alias-expanded.For example, these result in SimpleCommand
nodes:
ls -l
read -n 1
(normally a builtin)myfunc foo
These don't:
x=42
declare -r x=42
break
, continue
, return
, exit
— as explained above, these are
keywords and not builtins.{ echo one; echo two; }
for
, while
, case
, functions, etc.mksh
, and have other differencesThat is, in OSH and mksh, something like echo *.@(cc|h)
is an extended glob.
But echo $x
, where $x
contains the pattern, is not.
For more details and differences, see the Extended Glob section of the Word Language doc.
The OSH completion API is mostly compatible with the bash completion API,
except that it moves the responsibility for quoting out of plugins and onto
the shell itself. Plugins should return candidates as argv
entries, not
shell words.
See the completion doc for details.
The rules for history substitution like !echo
are simpler. There are no
special cases to avoid clashes with ${!indirect}
and so forth.
TODO: Link to the history lexer.
External:
set -o posix
does in bash.