stargazing/README.md

226 lines
7.9 KiB
Markdown
Raw Normal View History

2022-09-23 12:00:15 -07:00
# stargazing
this repository contains experiments
in building third-party software against
[cosmopolitan libc.](https://justine.lol/cosmopolitan/)
the provided scripts produce APEs
(Actually Portable Executables) that run on
Linux, Mac, Windows, FreeBSD, OpenBSD, and NetBSD.
**however,** despite *running* on all platforms,
not every feature is available on every platform.
cosmopolitan already does a tremendous job at
polyfilling missing features across platforms,
2022-10-14 16:54:03 -07:00
but pre-existing software may take a lot of
2022-09-23 15:16:58 -07:00
patching before becoming fully functional.
2022-09-23 12:04:33 -07:00
2022-10-14 16:35:09 -07:00
## procedures
in order of dependence, then alphabetical order.
### Phase 1
* **notwa-util:** convenient shell scripts for
augmenting common tasks relating to building.
this also builds a busybox rescue shell,
configured specifically for containers.
### Phase 2
<!--
the base cosmopolitan library and binaries,
with some minor patches and a custom distrib.
-->
2022-10-14 16:44:57 -07:00
* **cosmo-bootstrap:** builds the basic libraries (i.e.
2022-10-14 16:35:09 -07:00
cosmopolitan libc) and binaries to bootstrap itself
2022-10-14 16:54:03 -07:00
and build other software against.
2022-10-14 16:35:09 -07:00
### Phase 3
2022-10-14 16:44:57 -07:00
* **cosmo-all:** builds almost everything else from
the cosmopolitan repository (i.e. first-class third-party support)
2022-10-14 16:49:33 -07:00
the "flavor" used can be different than that of cosmo-bootstrap.
2022-10-14 16:44:57 -07:00
2022-10-14 16:49:33 -07:00
* **cosmo-dist:** only contains the essentials from cosmo-bootstrap.
2022-10-14 16:35:09 -07:00
2022-10-14 19:48:02 -07:00
* **rescue:** a minimal container used as a base
for many of the simple containers in phase 4.
2022-10-14 16:35:09 -07:00
### Phase 4
2022-10-14 19:48:02 -07:00
* **assimilate:** converts Actually Portable Executables
into native binaries. typically unnecessary, but sometimes nice.
* **cosmo-awk:** Actually Portable awk (fork of One True Awk)
2022-10-14 16:35:09 -07:00
* **cosmo-base:** combines cosmo-dist, cosmo-assimilate, cosmo-awk,
and cosmo-sed. this is used for building other containers.
* **cosmo-chibicc:** chibicc (not sure which commit this is forked from)
with many extensions.
2022-10-14 17:54:35 -07:00
* **cosmo-lua:** Actually Portable Lua
2022-10-14 19:48:02 -07:00
* **cosmo-make:** Landlocked Make (fork of GNU Make)
2022-10-14 16:35:09 -07:00
* **cosmo-python:** Actually Portable Python 3.6.14
* **cosmo-quickjs:** quickjs (2021-03-27)
2022-10-14 19:48:02 -07:00
* **cosmo-sed:** Actually Portable sed (fork of NetBSD sed)
2022-10-14 16:35:09 -07:00
* **cosmo-sqlite:** sqlite3 (3350500) binaries and libraries.
2022-10-14 19:48:02 -07:00
* **unbourne:** a fork of dash (v0.5.11.5) with extended interactivity.
2022-10-14 16:35:09 -07:00
### Phase 5
* **cosmo-kuroko**: kuroko (bleeding edge), a dialect of Python 3.
kuroko combines Python's syntax with Lua's level of embeddability,
and it can be used standalone as well.
* **cosmo-muon**: muon (bleeding edge), a reimplementation of mesen, written in C.
* **cosmo-perl**: Actually Portable Perl, thanks to Gavin Hayes
* **cosmo-yices**: yices (bleeding edge), SAT + SMT (SMTLIB, SMTLIB2) solvers.
also includes **cosmo-kissat** (bleeding edge), one of the best SAT solvers.
2022-09-23 14:24:39 -07:00
## building
on most Linux distros with podman properly installed and configured,
2022-09-23 14:25:08 -07:00
it should suffice to run [`./build-all`][ba] in the root of the repository.
2022-09-23 17:08:55 -07:00
binaries are copied to the `out/` directory upon success.
2022-09-23 14:24:39 -07:00
2022-09-23 14:25:08 -07:00
[ba]: https://eaguru.guru/git/notwa/stargazing/src/branch/galaxy/build-all
2022-09-23 14:55:15 -07:00
## installing
no installation is necessary — everything in the `out/` directory is self-contained.
however, if you'd like to use create
your own pseudo-release of cosmopolitan to build against,
you can run `install-cosmo` to install an amalgamation to `/opt/cosmopolitan/`.
2022-09-23 15:14:46 -07:00
this includes the amalgamation (in `dist/`),
the gcc toolchain (in `gcc/`),
some essential APEs (in `bin/`),
and all the necessary headers.
2022-09-23 14:55:15 -07:00
2022-09-23 12:04:33 -07:00
## notes
### Docker
2022-09-23 12:09:14 -07:00
this project uses [Dockerfiles][df] to allow for *mostly-reproducible* builds.
2022-09-23 15:15:36 -07:00
[podman is used][podman] instead of docker, because the official docker software
2022-09-23 12:09:14 -07:00
is commercial and obnoxious.
2022-09-23 12:04:33 -07:00
**neither docker nor podman is required for executing the binaries.**
2022-09-23 12:09:14 -07:00
binaries are to be isolated and extracted from the resulting containers.
those binaries can then be transferred across platforms, like any other APE.
[df]: https://docs.docker.com/engine/reference/builder/
2022-09-23 12:25:04 -07:00
[podman]: https://docs.podman.io/en/latest/markdown/podman-build.1.html
2022-09-23 12:04:33 -07:00
in the future, it would be nice to use something
2022-09-23 12:25:04 -07:00
like [what Void Linux does for its packages,][void]
2022-09-23 12:04:33 -07:00
which would entirely sidestep the clunkiness
of Dockerfiles and working with containers.
2022-09-23 12:09:14 -07:00
[void]: https://github.com/void-linux/void-packages
make no mistake, this project does *not* strive for reproducible builds
[in any way that nix does.](https://nixos.org/guides/how-nix-works.html)
2022-09-23 12:13:52 -07:00
### Host OS
this project must be built on a host running Linux.
if you are not running Linux, use a VM of some sort.
i personally use VirtualBox running Alpine Linux.
i have not tried building with WSL2.
2022-09-23 12:14:00 -07:00
### Code Style
this project encourages a heretical and horrifying style of code
for its shell scripts. the intent is to chain `&&`s while facilitating
trivial re-ordering of lines. for instance, a `RUN` command in a Dockerfile
(which just executes a line of shell) might look like this:
```sh
RUN : \
&& echo hello \
&& note() \
{ : \
&& printf '\033[1m%s\033[m: %s\n' >&2 \
noted "$*" \
;} \
2022-09-23 12:26:41 -07:00
# this is an inline comment! it should end with a semicolon, \
# because vim's syntax highlighting is bad. \
2022-09-23 12:14:00 -07:00
&& note '\\( o _ o )//' \
;
```
2022-09-23 12:56:29 -07:00
a good example of a script in this style is
[the `acquire` script.](https://eaguru.guru/git/notwa/stargazing/src/branch/galaxy/notwa-util/acquire)
2022-09-23 12:22:51 -07:00
#### gratuitous explanation
2022-09-23 12:14:00 -07:00
* 4-space indentation is a must, since at least three characters
are needed to contain the operators on the left of each line.
* `:` is akin to the `true` command; it's used as a no-op
that only sets the most recent exit code (`$?`) to `0`.
2022-09-23 12:22:51 -07:00
* `\` continues the shell command to the next line;
2022-09-23 12:14:00 -07:00
the proceeding newline is removed and ignored.
* `&&` is a short-circuiting operator that only evaluates
its right-hand side when `$?` is `0`. `&&` is preferred
over a simple semicolon because most programs use a
non-zero exit code to indicate that an error occurred,
whereas `;` will continue executing code,
even in the presence of an error.
2022-09-23 12:22:51 -07:00
**note:** the `diff` command breaks the convention
and uses both `0` and `1` as normal exit codes.
2022-09-23 12:14:00 -07:00
* `||` is like `&&` but only executes when `$?` is *not* `0`.
**important:** `||` has a lower precedence than `&&`!
make sure to wrap your `||`s in braces unless you want to
2022-09-23 12:22:51 -07:00
capture *everything* before it. for example, an excerpt
of a shell script might look like this:
2022-09-23 12:14:00 -07:00
` && { test -e myfile || touch myfile ;} \`
* leftmost `&&`s and `||`s are always centered within their 4-space indent.
* avoid using pipes (`|`) when possible, because only the
exit code of the final command in any pipe is respected.
2022-09-23 12:22:51 -07:00
instead, write intermediate data to `/tmp` when possible.
remember to mount `/tmp` as a `tmpfs` at the start of the `RUN` command!
2022-09-23 12:14:00 -07:00
* functions are declared (`()`) on their own line
without any braces or other commands.
* `{ :` is used to begin a multi-line block,
similar to how `RUN :` begins a shell command.
sadly, the space between the brace and the colon is not optional.
2022-09-23 12:22:51 -07:00
* `;}` is used to finish a multi-line block.
2022-09-23 12:14:00 -07:00
this is aligned with the preceeding `&&` and `||` operators.
thankfully, the space between `;` and `}` is optional.
* a final `;` is used to finish off the line-continutation.
this means that every line before it can end with a `\`,
which simplifies editing. when podman gives you strange errors,
2022-09-23 12:22:51 -07:00
double-check that you aren't forgetting a `\` or the final semicolon.
2022-09-23 12:14:00 -07:00
* redirections are usually specified *before* a final list of arguments.
this makes skimming code easier when arguments extend across many lines.
2022-09-23 12:22:51 -07:00
* **TODO:** explain `if cmd \␤ ;then : \␤ && cmd \␤ ;fi`.
2022-09-23 12:14:00 -07:00
2022-09-23 12:22:51 -07:00
* **TODO:** explain `for cmd \␤ ;do : \␤ && cmd \␤ ;done`.
2022-09-23 12:14:00 -07:00
2022-09-23 12:22:51 -07:00
* **TODO:** explain `while cmd \␤ ;do : \␤ && cmd \␤ ;done`.
2022-09-23 12:14:00 -07:00
2022-09-23 12:26:41 -07:00
* **TODO:** explain `test`s?
* **TODO:** explain the two different kinds of commends.