1 | ---
|
2 | default_highlighter: oils-sh
|
3 | ---
|
4 |
|
5 | BYO - Protocols for Test Discovery, Shell Completion
|
6 | ===========
|
7 |
|
8 | BYO is a simple mechanism to turn CLI processes into "servers" which respond to
|
9 | requests encoded in environment variables.
|
10 |
|
11 | Points of reference:
|
12 |
|
13 | - [Test Anything Protocol][TAP]
|
14 | - e.g. Perl scripts parse stdout of test processes in any language
|
15 | - [Shellac Protocol Proposal V2]($wiki) (wiki, 2019)
|
16 |
|
17 | (About the name: It originally stood for Bash YSH OSH. But "bring your own" is
|
18 | a good acronym!)
|
19 |
|
20 | [TAP]: https://testanything.org/
|
21 |
|
22 | <div id="toc">
|
23 | </div>
|
24 |
|
25 | ## The General Idea
|
26 |
|
27 | Executables should respond to the `BYO_COMMAND` environment variable:
|
28 |
|
29 | BYO_COMMAND=foo
|
30 |
|
31 | And `BYO_ARG=bar` varies based on the command.
|
32 |
|
33 | A library that implements this is:
|
34 |
|
35 | source $LIB_OSH/byo-server.sh
|
36 |
|
37 | But it's designed to be implemented in Python, C++, etc.
|
38 |
|
39 | A client is:
|
40 |
|
41 | test/byo-client.sh detect myscript.sh
|
42 |
|
43 |
|
44 | ## Protocol
|
45 |
|
46 | ### Detecting BYO Servers
|
47 |
|
48 | Here's how you detect if an executable supports BYO:
|
49 |
|
50 | $ BYO_COMMAND=detect ./any-executable </dev/null
|
51 | list-tests
|
52 | run-tests
|
53 |
|
54 | # must exit with code 66, which is ASCII 'B'
|
55 |
|
56 | ### Testing - discover and run
|
57 |
|
58 | List tests first:
|
59 |
|
60 | BYO_COMMAND=list-tests
|
61 |
|
62 | Then run them one at a time:
|
63 |
|
64 | BYO_COMMAND=run-test
|
65 | BYO_ARG=foo
|
66 |
|
67 | ### Shell completion - use these primitives
|
68 |
|
69 | TODO:
|
70 |
|
71 | BYO_COMMAND=list-tasks # related to task-five
|
72 | BYO_COMMAND=list-flags # only some binaries have flags
|
73 |
|
74 | <!--
|
75 | Note: Look at Clang and npm completion?
|
76 | -->
|
77 |
|
78 | ## Client Tool
|
79 |
|
80 | The tool should work like this:
|
81 |
|
82 | $ byo detect myscript.sh
|
83 | $ byo test myscript.sh
|
84 |
|
85 | (Right now it's [test/byo-client.sh]($oils-src))
|
86 |
|
87 | ## Appendix: Future Work
|
88 |
|
89 | ### Other Applications
|
90 |
|
91 | - Benchmarking with TSV output
|
92 | - Building tests first, with Ninja
|
93 | - Deployment
|
94 |
|
95 | Runtime:
|
96 |
|
97 | - Gateway Interface / Coprocess
|
98 | - like FastCGI / CGI
|
99 | - Logs
|
100 |
|
101 | Points of reference:
|
102 |
|
103 | - 12 factor app for Hosting (Heroku)
|
104 | - CGI / FastCGI
|
105 |
|
106 | ### Coprocesses
|
107 |
|
108 | Instead of a fresh process env variables, we might want to detect coprocesses.
|
109 |
|