run-file () { local spec_name=$1; shift; local spec_file=spec/$spec_name.test.sh; local suite; suite=$(test/sh_spec.py --print-spec-suite $spec_file); local spec_subdir; case $suite in osh) spec_subdir='osh-cpp' ;; ysh) spec_subdir='ysh-cpp' ;; *) die "Invalid suite $suite" ;; esac; local base_dir=_tmp/spec/$spec_subdir; mkdir -v -p $base_dir; sh-spec $spec_file --timeout 10 --oils-bin-dir $PWD/bin --oils-cpp-bin-dir $REPO_ROOT/_bin/cxx-asan --tsv-output $base_dir/${spec_name}.tsv "$@" } ysh-stdlib-args: spec test case results

Results for ysh-stdlib-args.test.sh

status	ysh	ysh-cpp
pass	8	8
FAIL	1	1
total	9	9

case	ysh	ysh-cpp	description
0	pass	pass	args.ysh example usage
1	pass	pass	Bool flag, positional args, more positional
2	pass	pass	Test multiple ARGVs against a parser
3	FAIL	FAIL	Basic help message
	details	details
4	pass	pass	Compare parseArgs() vs Python argparse
5	pass	pass	Define spec and print it
6	pass	pass	Default values
7	pass	pass	Duplicate argument/flag names
8	pass	pass	Error cases

16 passed, 0 OK, 0 not implemented, 0 BUG, 1 failed, 0 timeouts, 0 cases skipped
1 failed under osh

Details on runs that didn't PASS

ysh

3 Basic help message

[ysh stdout] Expected 'usage: program-name [-h] [-v] src dst\n\nReference Implementation\n\npositional arguments:\n src\n dst\n\noptions:\n -h, --help show this help message and exit\n -v, --verbose Verbose\n' Got ''
[ysh status] Expected 0, got 127

stdout:

stderr:

    description '''
    ^~~~~~~~~~~
[ stdin ]:4: 'description' not found (OILS-ERR-100)
[ stdin ]:4: errexit PID 31986: command.Simple failed with status 127

ysh-cpp

3 Basic help message

[ysh-cpp stdout] Expected 'usage: program-name [-h] [-v] src dst\n\nReference Implementation\n\npositional arguments:\n src\n dst\n\noptions:\n -h, --help show this help message and exit\n -v, --verbose Verbose\n' Got ''
[ysh-cpp status] Expected 0, got 127

stdout:

stderr:

    description '''
    ^~~~~~~~~~~
[ stdin ]:4: 'description' not found (OILS-ERR-100)
[ stdin ]:4: errexit PID 32003: command.Simple failed with status 127