1 | `deps/` Directory
|
2 | =================
|
3 |
|
4 | Scripts to build various dependencies. They can used on the host machine, or
|
5 | in a Dockerfile to build a container. The container is used in our CI build,
|
6 | and can also be used locally.
|
7 |
|
8 | ## Docker Files
|
9 |
|
10 | Our images:
|
11 |
|
12 | - <https://hub.docker.com/u/oilshell>
|
13 |
|
14 | Base image maintained by:
|
15 |
|
16 | - <https://hub.docker.com/_/debian>
|
17 | - <https://github.com/debuerreotype/debuerreotype>
|
18 |
|
19 | Note: we could use names like `buster-20220912-slim` instead of `buster-slim`.
|
20 |
|
21 | - How can we get notified when the image is updated?
|
22 | - <https://crazymax.dev/diun/> has many notification mechanisms
|
23 |
|
24 |
|
25 | ## Container Dir Structure
|
26 |
|
27 | The file system in our images looks like this:
|
28 |
|
29 | /home/uke/
|
30 | tmp/ # Dockerfiles copy build scripts here
|
31 | oil/ # soil/host-shim.sh mounts the repo here
|
32 | _tmp/ # published as .wwz files to CI
|
33 | wedge/
|
34 | oils-for-unix.org # should be oils.pub
|
35 | pkg/
|
36 | bash/
|
37 | 4.4/
|
38 | 5.2/
|
39 | R-libs/
|
40 | ...
|
41 |
|
42 | Note that `build/dev-shell.sh` puts some executables in `$PATH`.
|
43 |
|
44 | TODO:
|
45 |
|
46 | - We should `--mount type=bind` "wedges" into say `/wedge/foo`
|
47 | - These can be either data or executable tools
|
48 | - I suppose `/home/uke/{oil,oil_DEPS}` are also wedges? They are meant not to conflict
|
49 | - But we want wedges to work OUTSIDE containers too.
|
50 | - We should also be able to COPY wedges into an image, for easier deployment on
|
51 | cloud services like Github Actions and fly.io
|
52 | - Then run an arbitrary shell command that uses the code and data
|
53 | - Outputs can go back into `/wedge`
|
54 | - Problems: absolute paths, root, `make install`, etc.
|
55 |
|
56 | Dir structure:
|
57 |
|
58 | /wedge/ # or you can build your own /home/andy/wedge
|
59 | oilshell.org/ # built for this distro, e.g. ubuntu-18.04
|
60 | DEPS/
|
61 | Python-2.7.13/
|
62 | Python-3.10.4/
|
63 | bash-4.4/
|
64 | bash-5.1/
|
65 |
|
66 | And then we'll need some symlinks like `python3`, etc.
|
67 |
|
68 | ## Kinds of Tarballs / "Dir Values"
|
69 |
|
70 | - Source vs. Intermediate - what repo stores it?
|
71 | - Absolute vs. Relative - where is it mounted?
|
72 | - Layer vs. Wedge - how is it mounted?
|
73 |
|
74 | ## Soil C++ Tarball Notes
|
75 |
|
76 | cpp-tarball is consumed by these jobs now:
|
77 |
|
78 | - raw-vm
|
79 | - wild
|
80 | - app-tests for ble.sh
|
81 |
|
82 | If we use it in more tasks, we could try to remove MyPy/Python 3 from Docker
|
83 | images. It should especially speed up the pull on sourcehut -- 30-40 seconds
|
84 | for pea/other-tests, vs. 2:13 for benchmarks2
|
85 |
|
86 | So these can also consume the tarball:
|
87 |
|
88 | - benchmarks
|
89 | - benchmarks2
|
90 | - interactive uses the 'benchmarks' image for some reason
|
91 | - cpp-spec
|
92 | - build ASAN binary from tarball
|
93 | - needs both osh and ysh
|
94 | - cpp-coverage -- would need to include C++ unit tests and Ninja in tarball,
|
95 | which is not a bad idea
|
96 |
|
97 | More:
|
98 |
|
99 | - Alpine testing
|
100 | - Smoosh testing with OCaml, similar to spec tests
|
101 |
|