OILS / deps / README.md View on Github | oilshell.org

101 lines, 72 significant
1`deps/` Directory
2=================
3
4Scripts to build various dependencies. They can used on the host machine, or
5in a Dockerfile to build a container. The container is used in our CI build,
6and can also be used locally.
7
8## Docker Files
9
10Our images:
11
12- <https://hub.docker.com/u/oilshell>
13
14Base image maintained by:
15
16- <https://hub.docker.com/_/debian>
17- <https://github.com/debuerreotype/debuerreotype>
18
19Note: we could use names like `buster-20220912-slim` instead of `buster-slim`.
20
21- How can we get notified when the image is updated?
22 - <https://crazymax.dev/diun/> has many notification mechanisms
23
24
25## Container Dir Structure
26
27The file system in our images looks like this:
28
29 /home/uke/
30 tmp/ # Dockerfiles copy build scripts here
31 oil/ # soil/host-shim.sh mounts the repo here
32 _tmp/ # published as .wwz files to CI
33 wedge/
34 oils-for-unix.org # should be oils.pub
35 pkg/
36 bash/
37 4.4/
38 5.2/
39 R-libs/
40 ...
41
42Note that `build/dev-shell.sh` puts some executables in `$PATH`.
43
44TODO:
45
46- We should `--mount type=bind` "wedges" into say `/wedge/foo`
47 - These can be either data or executable tools
48 - I suppose `/home/uke/{oil,oil_DEPS}` are also wedges? They are meant not to conflict
49 - But we want wedges to work OUTSIDE containers too.
50- We should also be able to COPY wedges into an image, for easier deployment on
51 cloud services like Github Actions and fly.io
52- Then run an arbitrary shell command that uses the code and data
53- Outputs can go back into `/wedge`
54 - Problems: absolute paths, root, `make install`, etc.
55
56Dir structure:
57
58 /wedge/ # or you can build your own /home/andy/wedge
59 oilshell.org/ # built for this distro, e.g. ubuntu-18.04
60 DEPS/
61 Python-2.7.13/
62 Python-3.10.4/
63 bash-4.4/
64 bash-5.1/
65
66And then we'll need some symlinks like `python3`, etc.
67
68## Kinds of Tarballs / "Dir Values"
69
70- Source vs. Intermediate - what repo stores it?
71- Absolute vs. Relative - where is it mounted?
72- Layer vs. Wedge - how is it mounted?
73
74## Soil C++ Tarball Notes
75
76cpp-tarball is consumed by these jobs now:
77
78- raw-vm
79- wild
80- app-tests for ble.sh
81
82If we use it in more tasks, we could try to remove MyPy/Python 3 from Docker
83images. It should especially speed up the pull on sourcehut -- 30-40 seconds
84for pea/other-tests, vs. 2:13 for benchmarks2
85
86So these can also consume the tarball:
87
88- benchmarks
89- benchmarks2
90- interactive uses the 'benchmarks' image for some reason
91- cpp-spec
92 - build ASAN binary from tarball
93 - needs both osh and ysh
94- cpp-coverage -- would need to include C++ unit tests and Ninja in tarball,
95 which is not a bad idea
96
97More:
98
99- Alpine testing
100- Smoosh testing with OCaml, similar to spec tests
101