Overview of Oils Code

category total lines num files
OSH (and common libraries) 42,507 81
YSH 9,050 24
Data Languages 3,033 11
Tools 1,330 5
YSH stdlib 214 6
Code Borrowed from Python's stdlib 429 5
Spec Tests 52,830 182
Gold Tests 2,775 27
Python Unit Tests 11,708 76
Other Shell Tests 17,051 65
Test Automation 1,412 11
mycpp Translator 6,930 15
mycpp Test Data 4,065 46
Zephyr ASDL 3,273 11
pgen2 Parser Generator 1,854 8
Other Code Generators 3,030 11
Yaks 510 4
Hand-written C++ Code 3,435 29
Garbage-Collected Runtime 5,070 24
Unit tests in C++ 7,625 25
Incremental C++ Build 3,435 19
Build Automation 7,751 38
Release Automation 2,530 8
Soil: Multi-cloud CI with containers 3,676 23
Benchmarks 9,373 47
Metrics 1,619 8
Generated Python Code 27,358 24
Doc Tools 4,689 15
Web 1,086 5

Related Documents

The README for oilshell/oil has another overview of the repository.

OSH (and common libraries)

This is the input to the translators, written in statically-typed Python. Note that bash is at least 140K lines of code, and OSH implements a large part of bash and more.

bin/oils_for_unix.py                            198
builtin/assign_osh.py                           575
builtin/bracket_osh.py                          285
builtin/completion_osh.py                       514
builtin/dirs_osh.py                             349
builtin/io_osh.py                               183
builtin/meta_osh.py                             479
builtin/misc_osh.py                             115
builtin/printf_osh.py                           541
builtin/process_osh.py                          598
builtin/pure_osh.py                             490
builtin/read_osh.py                             522
builtin/readline_osh.py                         142
builtin/trap_osh.py                             274
core/alloc.py                                   320
core/ansi.py                                     15
core/comp_ui.py                                 578
core/completion.py                            1,516
core/dev.py                                     766
core/error.py                                   344
core/executor.py                                713
core/main_loop.py                               402
core/num.py                                      99
core/optview.py                                  58
core/process.py                               1,952
core/pyos.py                                    397
core/pyutil.py                                  228
core/runtime.asdl                               173
core/shell.py                                 1,146
core/state.py                                 2,512
core/ui.py                                      539
core/util.py                                    168
core/value.asdl                                 175
core/vm.py                                      338
frontend/args.py                                669
frontend/builtin_def.py                         164
frontend/consts.py                              383
frontend/flag_def.py                            540
frontend/flag_spec.py                           349
frontend/flag_util.py                            69
frontend/id_kind_def.py                         798
frontend/lexer.py                               451
frontend/lexer_def.py                         1,119
frontend/location.py                            571
frontend/match.py                               292
frontend/option_def.py                          410
frontend/parse_lib.py                           405
frontend/py_readline.py                         104
frontend/reader.py                              256
frontend/signal_def.py                           94
frontend/syntax.asdl                            653
frontend/syntax_abbrev.py                       140
frontend/typed_args.py                          589
frontend/types.asdl                              61
osh/arith_parse.py                              197
osh/bool_parse.py                               272
osh/bool_stat.py                                135
osh/braces.py                                   539
osh/cmd_eval.py                               2,173
osh/cmd_parse.py                              2,824
osh/glob_.py                                    494
osh/history.py                                  193
osh/prompt.py                                   373
osh/sh_expr_eval.py                           1,205
osh/split.py                                    311
osh/string_ops.py                               545
osh/tdop.py                                     342
osh/word_.py                                    805
osh/word_compile.py                             310
osh/word_eval.py                              2,436
osh/word_parse.py                             2,218
pyext/fanos.c                                   119
pyext/fastfunc.c                                133
pyext/fastfunc.pyi                               11
pyext/fastlex.c                                 351
pyext/fastlex.pyi                                19
pyext/libc.c                                    421
pyext/libc.pyi                                   18
pyext/line_input.pyi                             32
pyext/posix_.pyi                                210

42,507 lines in 81 files  

YSH

Expression grammar, parser, evaluator, etc.

builtin/completion_ysh.py                        69
builtin/error_ysh.py                            301
builtin/func_eggex.py                           186
builtin/func_hay.py                             150
builtin/func_misc.py                            709
builtin/hay_ysh.py                              431
builtin/io_ysh.py                               257
builtin/json_ysh.py                             139
builtin/meta_ysh.py                             101
builtin/method_dict.py                           45
builtin/method_io.py                             87
builtin/method_list.py                          100
builtin/method_other.py                          32
builtin/method_str.py                           514
builtin/module_ysh.py                           120
builtin/pure_ysh.py                             270
ysh/expr_eval.py                              1,418
ysh/expr_parse.py                               378
ysh/expr_to_ast.py                            1,708
ysh/func_proc.py                                571
ysh/grammar.pgen2                               538
ysh/regex_translate.py                          393
ysh/val_ops.py                                  533

9,050 lines in 24 files  

Data Languages

JSON, J8 Notation, ...

data_lang/nil8.asdl                              31
data_lang/pretty.asdl                            30
data_lang/j8.py                               1,367
data_lang/j8_lite.py                             54
data_lang/pretty.py                             791
data_lang/pyj8.py                                27
data_lang/j8.h                                  314
data_lang/j8_libc.c                             221
data_lang/j8_libc.h                              28
data_lang/utf8.h                                170

3,033 lines in 11 files  

Tools

tools/deps.py                                   149
tools/fmt.py                                     25
tools/readlink.py                                31
tools/ysh_ify.py                              1,125

1,330 lines in 5 files  

YSH stdlib

stdlib/draft-synch.ysh                           51
stdlib/funcs.ysh                                  5
stdlib/methods.ysh                               18
stdlib/prelude.ysh                               25
stdlib/testing.ysh                              115

214 lines in 6 files  

Code Borrowed from Python's stdlib

pylib/cgi.py                                     19
pylib/collections_.py                           214
pylib/os_path.py                                169
pylib/path_stat.py                               27

429 lines in 5 files  

Spec Tests

A comprehensive test suite that compares OSH against other shells. If OSH passes these tests in BOTH Python and C++, it means that the translation works.

spec/TODO-deprecate.test.sh                      80
spec/alias.test.sh                              571
spec/append.test.sh                             303
spec/arith-context.test.sh                      216
spec/arith.test.sh                              875
spec/array-basic.test.sh                         45
spec/array-compat.test.sh                       189
spec/array.test.sh                              803
spec/assign-deferred.test.sh                    105
spec/assign-dialects.test.sh                     61
spec/assign-extended.test.sh                    881
spec/assign.test.sh                             674
spec/assoc-zsh.test.sh                           53
spec/assoc.test.sh                              797
spec/background.test.sh                         184
spec/ble-features.test.sh                       645
spec/ble-idioms.test.sh                         334
spec/blog-other1.test.sh                         57
spec/blog1.test.sh                               95
spec/blog2.test.sh                               60
spec/brace-expansion.test.sh                    502
spec/bugs.test.sh                               390
spec/builtin-bash.test.sh                       192
spec/builtin-bracket.test.sh                    581
spec/builtin-cd.test.sh                         304
spec/builtin-completion.test.sh                 596
spec/builtin-dirs.test.sh                       238
spec/builtin-echo.test.sh                       248
spec/builtin-eval-source.test.sh                249
spec/builtin-getopts.test.sh                    347
spec/builtin-history.test.sh                    198
spec/builtin-meta.test.sh                       607
spec/builtin-misc.test.sh                       161
spec/builtin-printf.test.sh                   1,100
spec/builtin-process.test.sh                    588
spec/builtin-read.test.sh                       770
spec/builtin-special.test.sh                    141
spec/builtin-times.test.sh                       15
spec/builtin-trap-bash.test.sh                  657
spec/builtin-trap-err.test.sh                   647
spec/builtin-trap.test.sh                       247
spec/builtin-type-bash.test.sh                  326
spec/builtin-type.test.sh                       117
spec/builtin-vars.test.sh                       707
spec/case_.test.sh                              199
spec/command-parsing.test.sh                     63
spec/command-sub.test.sh                        285
spec/command_.test.sh                           194
spec/comments.test.sh                            11
spec/dbracket.test.sh                           438
spec/dparen.test.sh                             205
spec/empty-bodies.test.sh                        24
spec/errexit-osh.test.sh                        758
spec/errexit.test.sh                            532
spec/exit-status.test.sh                        280
spec/explore-parsing.test.sh                     42
spec/extglob-files.test.sh                      384
spec/extglob-match.test.sh                      400
spec/fatal-errors.test.sh                       186
spec/for-expr.test.sh                           173
spec/func-parsing.test.sh                       105
spec/glob.test.sh                               388
spec/globignore.test.sh                         159
spec/hay-isolation.test.sh                      169
spec/hay-meta.test.sh                           200
spec/hay.test.sh                                699
spec/here-doc.test.sh                           370
spec/if_.test.sh                                 56
spec/interactive-parse.test.sh                   37
spec/interactive.test.sh                        343
spec/introspect.test.sh                         287
spec/let.test.sh                                 23
spec/loop.test.sh                               474
spec/nameref.test.sh                            637
spec/nix-idioms.test.sh                         196
spec/nocasematch-match.test.sh                   71
spec/nul-bytes.test.sh                          158
spec/parse-errors.test.sh                       226
spec/pipeline.test.sh                           197
spec/posix.test.sh                              153
spec/process-sub.test.sh                        204
spec/prompt.test.sh                             253
spec/quote.test.sh                              230
spec/redirect-command.test.sh                   306
spec/redirect-multi.test.sh                     308
spec/redirect.test.sh                           485
spec/regex.test.sh                              631
spec/serialize.test.sh                          230
spec/sh-func.test.sh                            152
spec/sh-options.test.sh                         746
spec/sh-usage.test.sh                            94
spec/shell-grammar.test.sh                      206
spec/smoke.test.sh                              126
spec/strict-options.test.sh                     277
spec/subshell.test.sh                            23
spec/tilde.test.sh                              184
spec/toysh-posix.test.sh                        344
spec/toysh.test.sh                              143
spec/type-compat.test.sh                        159
spec/unicode.test.sh                            196
spec/var-num.test.sh                             43
spec/var-op-bash.test.sh                        397
spec/var-op-len.test.sh                         219
spec/var-op-patsub.test.sh                      413
spec/var-op-slice.test.sh                       403
spec/var-op-strip.test.sh                       357
spec/var-op-test.test.sh                        520
spec/var-ref.test.sh                            509
spec/var-sub-quote.test.sh                      312
spec/var-sub.test.sh                             60
spec/vars-bash.test.sh                           30
spec/vars-special.test.sh                       634
spec/whitespace.test.sh                         114
spec/word-eval.test.sh                           62
spec/word-split.test.sh                         421
spec/xtrace.test.sh                             361
spec/ysh-assign.test.sh                         402
spec/ysh-augmented.test.sh                      245
spec/ysh-bin.test.sh                             19
spec/ysh-blocks.test.sh                         407
spec/ysh-bugs.test.sh                           199
spec/ysh-builtin-ctx.test.sh                    168
spec/ysh-builtin-error.test.sh                  413
spec/ysh-builtin-eval.test.sh                    99
spec/ysh-builtin-help.test.sh                    99
spec/ysh-builtin-meta.test.sh                   254
spec/ysh-builtin-module.test.sh                  38
spec/ysh-builtin-process.test.sh                 85
spec/ysh-builtin-shopt.test.sh                  180
spec/ysh-builtins.test.sh                       551
spec/ysh-case.test.sh                           245
spec/ysh-cmd-lang.test.sh                        54
spec/ysh-command-sub.test.sh                    147
spec/ysh-completion.test.sh                     204
spec/ysh-convert.test.sh                        171
spec/ysh-demo.test.sh                            59
spec/ysh-dev.test.sh                            163
spec/ysh-dict.test.sh                            91
spec/ysh-expr-arith.test.sh                     369
spec/ysh-expr-bool.test.sh                      185
spec/ysh-expr-compare.test.sh                   389
spec/ysh-expr-sub.test.sh                        42
spec/ysh-expr.test.sh                           697
spec/ysh-for.test.sh                            192
spec/ysh-func-builtin.test.sh                   103
spec/ysh-func.test.sh                           535
spec/ysh-funcs-external.test.sh                  86
spec/ysh-int-float.test.sh                      200
spec/ysh-interactive.test.sh                     37
spec/ysh-json.test.sh                         1,259
spec/ysh-keywords.test.sh                        37
spec/ysh-list.test.sh                            97
spec/ysh-methods.test.sh                        490
spec/ysh-multiline.test.sh                      179
spec/ysh-options-assign.test.sh                 106
spec/ysh-options.test.sh                        811
spec/ysh-place.test.sh                          136
spec/ysh-printing.test.sh                       267
spec/ysh-proc.test.sh                           526
spec/ysh-prompt.test.sh                         133
spec/ysh-regex-api.test.sh                      829
spec/ysh-regex.test.sh                          711
spec/ysh-reserved.test.sh                        48
spec/ysh-scope.test.sh                          766
spec/ysh-slice-range.test.sh                    209
spec/ysh-source.test.sh                          29
spec/ysh-special-vars.test.sh                    50
spec/ysh-stdlib-2.test.sh                       158
spec/ysh-stdlib-args.test.sh                    333
spec/ysh-stdlib-testing.test.sh                 125
spec/ysh-stdlib.test.sh                         222
spec/ysh-string.test.sh                         548
spec/ysh-tuple.test.sh                           28
spec/ysh-unicode.test.sh                        184
spec/ysh-usage.test.sh                           47
spec/ysh-user-feedback.test.sh                  141
spec/ysh-var-sub.test.sh                         45
spec/ysh-with-sh.test.sh                        221
spec/ysh-word-eval.test.sh                      145
spec/ysh-xtrace.test.sh                         547
spec/zsh-idioms.test.sh                          45

52,830 lines in 182 files  

Gold Tests

Another suite that tests shells "from the outside". Instead of making explicit assertions, we verify that OSH behaves like bash.

test/gold/abuild.sh                               9
test/gold/and-or.sh                              27
test/gold/autoconf-backtick.sh                   50
test/gold/big-here-doc.sh                     1,531
test/gold/case-in-subshell.sh                    37
test/gold/char-class.sh                          33
test/gold/command-sub-2.sh                       12
test/gold/command-sub.sh                         39
test/gold/comments.sh                            22
test/gold/complex-here-docs.sh                  192
test/gold/configure-bug.sh                       13
test/gold/de1.sh                                 24
test/gold/declare.sh                             28
test/gold/dollar-sq.sh                           42
test/gold/echo-e.sh                              36
test/gold/errexit-confusion.sh                  139
test/gold/errexit.sh                            121
test/gold/export.sh                              65
test/gold/glob.sh                                26
test/gold/lineno.sh                              23
test/gold/nix.sh                                 32
test/gold/readlink.sh                            61
test/gold/readonly.sh                            24
test/gold/scope.sh                              126
test/gold/strip-op-char-class.sh                  6
test/gold/word-eval.sh                           57

2,775 lines in 27 files  

Python Unit Tests

asdl/examples/typed_arith_parse_test.py         227
asdl/format_test.py                              46
asdl/front_end_test.py                          184
asdl/gen_python_test.py                         180
benchmarks/uftrace_allocs_test.py                20
build/dynamic_deps_test.py                       23
build/ninja_lib_test.py                         253
builtin/bracket_osh_test.py                      26
builtin/completion_osh_test.py                   16
builtin/func_misc_test.py                        27
builtin/misc_osh_test.py                         28
builtin/read_osh_test.py                         35
builtin/readline_osh_test.py                    100
client/py_fanos_test.py                         108
core/alloc_test.py                               54
core/comp_ui_test.py                            177
core/completion_test.py                         823
core/process_test.py                            267
core/pyutil_test.py                              17
core/state_test.py                              352
core/ui_test.py                                  30
core/util_test.py                                64
data_lang/j8_test.py                            229
data_lang/pretty_test.py                        113
data_lang/pyj8_test.py                           52
doctools/cmark_test.py                          203
doctools/help_gen_test.py                       116
doctools/html_lib_test.py                        49
doctools/oils_doc_test.py                        79
doctools/spelling_test.py                        49
doctools/split_doc_test.py                      104
frontend/args_test.py                           237
frontend/id_kind_def_test.py                     97
frontend/lexer_def_test.py                      454
frontend/lexer_gen_test.py                       53
frontend/lexer_test.py                          216
frontend/match_test.py                          128
frontend/reader_test.py                          74
frontend/typed_args_test.py                     158
lazylex/html_test.py                             78
mycpp/format_strings_test.py                     43
mycpp/mops_test.py                               32
mycpp/mylib_test.py                              56
mycpp/pass_state_test.py                        596
osh/arith_parse_test.py                         183
osh/bool_parse_test.py                          122
osh/braces_test.py                              186
osh/cmd_eval_test.py                             81
osh/cmd_parse_test.py                         1,491
osh/glob_test.py                                160
osh/history_test.py                             126
osh/prompt_test.py                               60
osh/split_test.py                               164
osh/string_ops_test.py                          203
osh/word_compile_test.py                         16
osh/word_eval_test.py                           155
osh/word_parse_test.py                          607
osh/word_test.py                                176
pyext/fanos_test.py                             274
pyext/fastfunc_test.py                           83
pyext/fastlex_test.py                            85
pyext/libc_test.py                              310
pyext/line_input_test.py                         27
pyext/posix_test.py                             184
pylib/os_path_test.py                            59
pylib/path_stat_test.py                          24
soil/web_test.py                                 52
spec/stateful/harness_test.py                    46
test/sh_spec_test.py                            182
test/wild_report_test.py                         29
tools/fmt_test.py                                18
tools/ysh_ify_test.py                            78
ysh/expr_parse_test.py                          163
ysh/regex_translate_test.py                      51
ysh/val_ops_test.py                              40

11,708 lines in 76 files  

Other Shell Tests

test/all.sh                                      92
test/alpine.sh                                  307
test/asan.sh                                     41
test/ble.sh                                     139
test/bug-1986.sh                                122
test/bugs.sh                                     43
test/common-test.sh                              22
test/common.sh                                  271
test/coverage.sh                                180
test/cpp-unit.sh                                 45
test/fedora.sh                                   22
test/gentoo.sh                                   83
test/gold.sh                                    168
test/lint.sh                                    312
test/lossless.sh                                148
test/ltrace.sh                                   49
test/nix.sh                                      24
test/nohup.sh                                    89
test/opyc.sh                                    118
test/osh-usage.sh                               222
test/other.sh                                    34
test/parse-err-compare.sh                       138
test/parse-errors.sh                            833
test/process-table-portable.sh                  164
test/process-table.sh                           171
test/process_table.py                           223
test/py2_lint.py                                107
test/py3_lint.py                                 19
test/py_stdlib.sh                                32
test/report.R                                    54
test/runtime-errors.sh                        1,239
test/sh-assert.sh                               164
test/sh_spec.py                               1,428
test/shebang.sh                                  26
test/shell-vs-shell.sh                          314
test/signal-state.sh                             96
test/smoosh.sh                                  220
test/smoosh_import.py                            31
test/souffle-smoke.sh                            19
test/spec-alpine.sh                             154
test/spec-bin.sh                                 41
test/spec-common.sh                              93
test/spec-cpp.sh                                360
test/spec-py.sh                                 167
test/spec-runner.sh                             375
test/spec-util.sh                                39
test/spec-version.sh                            127
test/spec.sh                                    969
test/spec_lib.py                                233
test/stateful.sh                                229
test/syscall.py                                 215
test/syscall.sh                                 411
test/tools-deps.sh                               76
test/tsv-lib-test.sh                             71
test/tsv-lib.sh                                 111
test/unit.sh                                    292
test/vagrant.sh                                  54
test/wild-runner.sh                             204
test/wild.sh                                    729
test/wild_report.py                             813
test/ysh-every-string.sh                        192
test/ysh-ify.sh                                 657
test/ysh-parse-errors.sh                      1,689
test/ysh-runtime-errors.sh                      941

17,051 lines in 65 files  

Test Automation

asdl/TEST.sh                                     86
core/TEST.sh                                     21
cpp/TEST.sh                                     110
data_lang/TEST.sh                                35
frontend/TEST.sh                                 22
mycpp/TEST.sh                                   356
opy/TEST.sh                                     270
pea/TEST.sh                                     336
stdlib/TEST.sh                                   55
yaks/TEST.sh                                    121

1,412 lines in 11 files  

mycpp Translator

This prototype uses the MyPy frontend to translate statically-typed Python to C++. The generated code calls a small runtime which implements things like List[T], Dict[K, V], and Python's len().

mycpp/compare_pairs.py                           60
mycpp/const_pass.py                             532
mycpp/control_flow_pass.py                      519
mycpp/cppgen_pass.py                          3,068
mycpp/crash.py                                   38
mycpp/debug_pass.py                             465
mycpp/format_strings.py                         114
mycpp/ir_pass.py                                 99
mycpp/mops.py                                   203
mycpp/mycpp_main.py                             401
mycpp/mylib.py                                  529
mycpp/pass_state.py                             487
mycpp/util.py                                   177
mycpp/visitor.py                                238

6,930 lines in 15 files  

mycpp Test Data

Small Python examples that translate to C++, compile, and run.

mycpp/examples/__init__.py                        0
mycpp/examples/asdl_generated.py                 65
mycpp/examples/cartesian.py                      78
mycpp/examples/cgi.py                            62
mycpp/examples/classes.py                       298
mycpp/examples/containers.py                    254
mycpp/examples/control_flow.py                  113
mycpp/examples/escape.py                         70
mycpp/examples/fib_iter.py                       53
mycpp/examples/fib_recursive.py                  49
mycpp/examples/files.py                          64
mycpp/examples/invalid_condition.py              31
mycpp/examples/invalid_default_args.py           44
mycpp/examples/invalid_except.py                 38
mycpp/examples/invalid_format_args.py            15
mycpp/examples/invalid_format_strings.py         20
mycpp/examples/invalid_global.py                 18
mycpp/examples/invalid_print.py                  13
mycpp/examples/invalid_python.py                 61
mycpp/examples/invalid_switch.py                 84
mycpp/examples/invalid_try_else.py               38
mycpp/examples/invalid_types.py                  31
mycpp/examples/invalid_types_2.py                31
mycpp/examples/length.py                        109
mycpp/examples/lexer_main.py                     47
mycpp/examples/loops.py                         177
mycpp/examples/modules.py                        81
mycpp/examples/parse.py                         271
mycpp/examples/pgen2_demo.py                    106
mycpp/examples/scoped_resource.py               221
mycpp/examples/test_arith_ops.py                 45
mycpp/examples/test_cast.py                     208
mycpp/examples/test_classes_gc.py               212
mycpp/examples/test_conditional.py               66
mycpp/examples/test_default_args.py              75
mycpp/examples/test_globals.py                   62
mycpp/examples/test_hoist.py                     61
mycpp/examples/test_integers.py                  74
mycpp/examples/test_io_os_error.py              131
mycpp/examples/test_iterators.py                101
mycpp/examples/test_small_str.py                 33
mycpp/examples/test_strings.py                  272
mycpp/examples/test_switch.py                    80
mycpp/examples/tuple_return_value.py             54
mycpp/examples/varargs.py                        49

4,065 lines in 46 files  

Zephyr ASDL

A DSL for algebraic data types, borrowed from Python. Oils is the most strongly typed Bourne shell implementation!

asdl/asdl_main.py                               299
asdl/ast.py                                     238
asdl/format.py                                  518
asdl/front_end.py                               516
asdl/gen_cpp.py                                 826
asdl/gen_python.py                              611
asdl/pybase.py                                   51
asdl/runtime.py                                  67
asdl/util.py                                     11
asdl/visitor.py                                 136

3,273 lines in 11 files  

pgen2 Parser Generator

An LL(1) parser generator used to parse YSH expressions. Also borrowed from CPython.

pgen2/driver.py                                 101
pgen2/grammar.py                                303
pgen2/parse.py                                  225
pgen2/pgen.py                                   430
pgen2/pnode.py                                   67
pgen2/token.py                                  149
pgen2/tokenize.py                               579

1,854 lines in 8 files  

Other Code Generators

In order to make Oils statically typed, we had to abandon Python reflection and use C++ source code generation instead. The lexer, flag definitions, and constants can be easily compiled to C++.

core/optview_gen.py                              94
cpp/embedded_file_gen.py                         66
doctools/help_gen.py                            658
frontend/consts_gen.py                          641
frontend/flag_gen.py                            532
frontend/lexer_gen.py                           496
frontend/option_gen.py                           94
frontend/signal_gen.py                          105
osh/arith_parse_gen.py                          100
ysh/grammar_gen.py                              244

3,030 lines in 11 files  

Yaks

Experimental replacement for mycpp

yaks/gen_cpp.py                                 104
yaks/transform.py                               250
yaks/yaks_main.py                               156

510 lines in 4 files  

Hand-written C++ Code

Includes OS bindings. Small C++ files like cpp/osh_arith_parse.{cc,h} correspond to larger Python files like osh/arith_parse.py.

cpp/core.cc                                     427
cpp/core.h                                      277
cpp/data_lang.cc                                271
cpp/data_lang.h                                  34
cpp/embedded_file.h                              18
cpp/fanos.cc                                     57
cpp/fanos.h                                      18
cpp/fanos_shared.h                               49
cpp/frontend_flag_spec.cc                       231
cpp/frontend_flag_spec.h                        151
cpp/frontend_match.cc                           142
cpp/frontend_match.h                             90
cpp/frontend_pyreadline.cc                      336
cpp/frontend_pyreadline.h                        73
cpp/libc.cc                                     241
cpp/libc.h                                       38
cpp/osh.cc                                      161
cpp/osh.h                                        50
cpp/osh_tdop.cc                                  16
cpp/osh_tdop.h                                   52
cpp/pgen2.cc                                     44
cpp/pgen2.h                                      85
cpp/preamble.h                                   48
cpp/pylib.cc                                     58
cpp/pylib.h                                      22
cpp/stdlib.cc                                   258
cpp/stdlib.h                                    177
cpp/translation_stubs.h                          11

3,435 lines in 29 files  

Garbage-Collected Runtime

Uses a fork-friendly Mark-Sweep collector.

mycpp/common.h                                   92
mycpp/comparators.h                              90
mycpp/gc_alloc.h                                256
mycpp/gc_builtins.cc                            461
mycpp/gc_builtins.h                             175
mycpp/gc_dict.h                                 471
mycpp/gc_list.h                                 524
mycpp/gc_mops.cc                                 58
mycpp/gc_mops.h                                 123
mycpp/gc_mylib.cc                               303
mycpp/gc_mylib.h                                381
mycpp/gc_obj.h                                  179
mycpp/gc_slab.h                                  58
mycpp/gc_str.cc                                 645
mycpp/gc_str.h                                  219
mycpp/gc_tuple.h                                149
mycpp/hash.cc                                    45
mycpp/hash.h                                     22
mycpp/mark_sweep_heap.cc                        438
mycpp/mark_sweep_heap.h                         294
mycpp/probes.h                                   39
mycpp/runtime.h                                  24
mycpp/test_common.h                              24

5,070 lines in 24 files  

Unit tests in C++

The goal is to make the spec tests pass, but unit tests are helpful too.

cpp/core_test.cc                                417
cpp/data_lang_test.cc                           178
cpp/data_race_test.cc                           206
cpp/frontend_flag_spec_test.cc                  154
cpp/frontend_match_test.cc                       87
cpp/libc_test.cc                                403
cpp/obj_layout_test.cc                          212
cpp/osh_test.cc                                  50
cpp/pylib_test.cc                                52
cpp/stdlib_test.cc                              158
cpp/unicode_test.cc                              71
mycpp/bump_leak_heap_test.cc                     62
mycpp/float_test.cc                             150
mycpp/gc_builtins_test.cc                       447
mycpp/gc_dict_test.cc                           745
mycpp/gc_heap_test.cc                           479
mycpp/gc_list_test.cc                           508
mycpp/gc_mops_test.cc                           128
mycpp/gc_mylib_test.cc                          327
mycpp/gc_str_test.cc                          1,558
mycpp/gc_stress_test.cc                         222
mycpp/gc_tuple_test.cc                          139
mycpp/mark_sweep_heap_test.cc                   370
mycpp/small_str_test.cc                         502

7,625 lines in 25 files  

Incremental C++ Build

NINJA-config.sh                                 137
asdl/NINJA_subgraph.py                           56
bin/NINJA_subgraph.py                           125
build/ninja-rules-cpp.sh                        367
build/ninja-rules-py.sh                         349
build/ninja_lib.py                              518
build/ninja_lib_test.py                         253
build/ninja_main.py                             431
core/NINJA_subgraph.py                           58
cpp/NINJA_subgraph.py                           223
data_lang/NINJA_subgraph.py                      67
frontend/NINJA_subgraph.py                      175
mycpp/NINJA_subgraph.py                         418
osh/NINJA_subgraph.py                            32
pea/NINJA_subgraph.py                            19
prebuilt/NINJA_subgraph.py                       75
yaks/NINJA_subgraph.py                           93
ysh/NINJA_subgraph.py                            39

3,435 lines in 19 files  

Build Automation

Makefile                                        196
build/c_module_srcs.py                           98
build/c_module_toc.py                            51
build/clean.sh                                   65
build/common.sh                                  78
build/cpp.sh                                     40
build/cpython-defs.sh                           354
build/cpython_defs.py                           515
build/deps.sh                                 1,123
build/detect-cc.c                                 4
build/detect-pwent.c                              9
build/detect-readline.c                          19
build/detect-systemtap-sdt.c                      6
build/dev-setup-test.sh                          30
build/dev-shell-test.sh                          95
build/dev-shell.sh                              153
build/dev.sh                                      5
build/doc.sh                                    694
build/dynamic-deps.sh                           223
build/dynamic_deps.py                           155
build/hello.mk                                   51
build/make_zip.py                                53
build/native.sh                                 116
build/ninja-rules-cpp.sh                        367
build/ninja-rules-py.sh                         349
build/ninja_lib.py                              518
build/ninja_main.py                             431
build/oil.mk                                     61
build/old-ovm-test.sh                            42
build/ovm-actions.sh                            190
build/ovm-compile.sh                            430
build/portable-rules.mk                          96
build/py.sh                                     437
build/py2.sh                                     14
build/stamp.sh                                   76
configure                                       487
install                                         120

7,751 lines in 38 files  

Release Automation

devtools/release-history.sh                     207
devtools/release-native.sh                      128
devtools/release-note.sh                        280
devtools/release-patch.sh                        26
devtools/release-test.sh                         36
devtools/release-version.sh                   1,041
devtools/release.sh                             812

2,530 lines in 8 files  

Soil: Multi-cloud CI with containers

soil/__init__.py                                  0
soil/admin.sh                                    19
soil/collect_json.py                             35
soil/common.sh                                   92
soil/cpp-tarball.sh                              68
soil/diagnose.sh                                139
soil/github-actions.sh                          139
soil/github-tool.sh                              43
soil/history.sh                                  78
soil/host-shim.sh                               389
soil/maybe-merge.sh                             153
soil/sourcehut.sh                                81
soil/tests.sh                                    20
soil/timeout-test.sh                             50
soil/wait.sh                                    142
soil/web-init.sh                                174
soil/web-remote-test.sh                          43
soil/web-worker.sh                              352
soil/web.py                                     782
soil/web.sh                                     187
soil/web_test.py                                 52
soil/worker.sh                                  638

3,676 lines in 23 files  

Benchmarks

benchmarks/__init__.py                            0
benchmarks/auto.sh                              110
benchmarks/autoconf.sh                          599
benchmarks/awk-python.sh                         97
benchmarks/builtin-io.sh                         60
benchmarks/cachegrind.sh                         30
benchmarks/cachegrind_to_tsv.py                  54
benchmarks/callgrind.sh                          47
benchmarks/common.R                              98
benchmarks/common.sh                            101
benchmarks/compute.sh                           614
benchmarks/fake_libc.py                          14
benchmarks/gc-cachegrind.sh                      86
benchmarks/gc.sh                                725
benchmarks/gc_stats_to_tsv.py                    51
benchmarks/gperftools.sh                         60
benchmarks/id-test.sh                            41
benchmarks/id.sh                                448
benchmarks/micro.sh                              45
benchmarks/mimalloc.sh                          107
benchmarks/mycpp.sh                             117
benchmarks/osh-parser.sh                        508
benchmarks/osh-runtime.sh                       578
benchmarks/osh_runtime.py                        64
benchmarks/ovm-build.sh                         479
benchmarks/parse-help.sh                         65
benchmarks/perf.sh                              482
benchmarks/pypy.sh                               42
benchmarks/pytrace.py                            92
benchmarks/pytrace.sh                           184
benchmarks/report.R                           1,351
benchmarks/report.sh                            112
benchmarks/report_test.R                         32
benchmarks/startup.sh                           165
benchmarks/systemtap.sh                          75
benchmarks/time-test.sh                         324
benchmarks/time_.py                             267
benchmarks/tokens.sh                             24
benchmarks/uftrace.sh                           349
benchmarks/uftrace_allocs.py                    180
benchmarks/uftrace_allocs_test.py                20
benchmarks/uftrace_plugin.py                     99
benchmarks/virtual_memory.py                     59
benchmarks/vm-baseline.sh                       180
benchmarks/wedge.R                               97
benchmarks/wedge.sh                              41

9,373 lines in 47 files  

Metrics

metrics/bytecode.R                              388
metrics/bytecode.sh                             143
metrics/cpython-defs.R                           72
metrics/native-code.R                            88
metrics/native-code.sh                          191
metrics/source-code.sh                          583
metrics/tarball.sh                              154

1,619 lines in 8 files  

Generated Python Code

For the Python App Bundle.

_devbuild/gen/__init__.py                         0
_devbuild/gen/arg_types.py                      368
_devbuild/gen/arith_nt.py                         4
_devbuild/gen/demo_lib_asdl.py                  277
_devbuild/gen/expr_asdl.py                      219
_devbuild/gen/find_asdl.py                    1,250
_devbuild/gen/find_nt.py                         10
_devbuild/gen/grammar_nt.py                      73
_devbuild/gen/help_meta.py                      423
_devbuild/gen/hnode_asdl.py                     132
_devbuild/gen/id_kind.py                        512
_devbuild/gen/id_kind_asdl.py                   906
_devbuild/gen/nil8_asdl.py                      450
_devbuild/gen/option_asdl.py                    394
_devbuild/gen/pretty_asdl.py                    582
_devbuild/gen/runtime_asdl.py                 2,078
_devbuild/gen/shared_variant_asdl.py            651
_devbuild/gen/syntax_asdl.py                 12,193
_devbuild/gen/typed_arith_asdl.py               706
_devbuild/gen/typed_demo_asdl.py              1,402
_devbuild/gen/types_asdl.py                     163
_devbuild/gen/value_asdl.py                   2,306
_devbuild/gen/yaks_asdl.py                    2,259

27,358 lines in 24 files  

Doc Tools

doctools/cmark.py                               498
doctools/doc_html.py                            106
doctools/fmt_check.py                            91
doctools/html_head.py                            98
doctools/html_lib.py                             48
doctools/micro_syntax.cc                      1,122
doctools/micro_syntax.re2c.h                    688
doctools/oils_doc.py                            671
doctools/ref_check.py                           182
doctools/spelling.py                            141
doctools/split_doc.py                           157
doctools/src_tree.py                            470
doctools/util.py                                 11
lazylex/html.py                                 406

4,689 lines in 15 files  

Web

web/ajax.js                                     149
web/osh-to-oil.js                                79
web/table/csv2html.py                           425
web/table/table-sort.js                         433

1,086 lines in 5 files