stargazing/README.md

stargazing

this repository contains experiments in building third-party software against cosmopolitan libc. the provided scripts produce APEs (Actually Portable Executables) that run on Linux, Mac, Windows, FreeBSD, OpenBSD, and NetBSD. however, despite running on all platforms, not every feature is available on every platform. cosmopolitan already does a tremendous job at polyfilling missing features across platforms, but pre-existing software may take a lot of patching before becoming fully functional.

containers

in order of dependence, then alphabetical order.

Phase 1

• notwa-util: convenient shell scripts for augmenting common tasks relating to building. this also builds a busybox rescue shell, configured specifically for containers.

Phase 2

• cosmo-bootstrap: builds the basic libraries (i.e. cosmopolitan libc) and binaries to bootstrap itself and build other software against.

Phase 3

• cosmo-all: builds almost everything else from the cosmopolitan repository (i.e. first-class third-party support) the "flavor" used can be different than that of cosmo-bootstrap.

• cosmo-dist: only contains the essentials from cosmo-bootstrap.

• rescue: a minimal container used as a base for many of the simple containers in phase 4.

Phase 4

• assimilate: converts Actually Portable Executables into native binaries. typically unnecessary, but sometimes nice.

• bd: (TODO)

• bing: (TODO)

• blinkenlights: (TODO)

• cosmo-awk: Actually Portable awk (fork of One True Awk)

• cosmo-base: used to build other containers in phase 5.

• cosmo-chibicc: chibicc (TODO: version?) with many extensions.

• cosmo-lua: Actually Portable Lua

• cosmo-make: Landlocked Make (fork of GNU Make)

• cosmo-python: Actually Portable Python 3.6.14

• cosmo-quickjs: quickjs (2021-03-27)

• cosmo-sed: Actually Portable sed (fork of NetBSD sed)

• cosmo-sqlite: sqlite3 (3350500)

• fold: (TODO)

• greenbean: (TODO)

• hello: (TODO)

• life: (TODO)

• memzoom: (TODO)

• mkdeps: (TODO)

• pledge: (TODO)

• printimage: (TODO)

• printvideo: (TODO)

• redbean: (TODO)

• tinyemu: (TODO)

• unbourne: a fork of dash (v0.5.11.5) with extended interactivity.

• unveil: (TODO)

Phase 5

• cosmo-kuroko: kuroko (bleeding edge), a dialect of Python 3. kuroko combines Python's syntax with Lua's level of embeddability, and it can be used standalone as well.

• cosmo-muon: muon (bleeding edge), a reimplementation of mesen, written in C.

• cosmo-perl: Actually Portable Perl, thanks to Gavin Hayes

• cosmo-yices: yices (bleeding edge), SAT + SMT (SMTLIB, SMTLIB2) solvers. also includes cosmo-kissat (bleeding edge), one of the best SAT solvers.

building

on most Linux distros with podman properly installed and configured, it should suffice to run ./build-all in the root of the repository. binaries are copied to the out/ directory upon success.

installing

no installation is necessary — everything in the out/ directory is self-contained. however, if you'd like to use create your own pseudo-release of cosmopolitan to build against, you can run install-cosmo to install an amalgamation to /opt/cosmopolitan/. this includes the amalgamation (in dist/), the gcc toolchain (in gcc/), some essential APEs (in bin/), and all the necessary headers.

notes

Docker

this project uses Dockerfiles to allow for mostly-reproducible builds. podman is used instead of docker, because the official docker software is commercial and obnoxious.

neither docker nor podman is required for executing the binaries. binaries are to be isolated and extracted from the resulting containers. those binaries can then be transferred across platforms, like any other APE.

in the future, it would be nice to use something like what Void Linux does for its packages, which would entirely sidestep the clunkiness of Dockerfiles and working with containers.

make no mistake, this project does not strive for reproducible builds in any way that nix does.

Host OS

this project must be built on a host running Linux. if you are not running Linux, use a VM of some sort. i personally use VirtualBox running Alpine Linux. i have not tried building with WSL2.

Code Style

this project encourages a heretical and horrifying style of code for its shell scripts. the intent is to chain &&s while facilitating trivial re-ordering of lines. for instance, a RUN command in a Dockerfile (which just executes a line of shell) might look like this:

RUN : \
 && echo hello \
 && note() \
    { : \
     && printf '\033[1m%s\033[m: %s\n' >&2 \
        noted "$*" \
     ;} \
# this is an inline comment! it should end with a semicolon, \
# because vim's syntax highlighting is bad. \
 && note '\\( o _ o )//' \
;

a good example of a script in this style is the acquire script.

gratuitous explanation

• 4-space indentation is a must, since at least three characters are needed to contain the operators on the left of each line.

• : is akin to the true command; it's used as a no-op that only sets the most recent exit code ($?) to 0.

• \ continues the shell command to the next line; the proceeding newline is removed and ignored.

• && is a short-circuiting operator that only evaluates its right-hand side when $? is 0. && is preferred over a simple semicolon because most programs use a non-zero exit code to indicate that an error occurred, whereas ; will continue executing code, even in the presence of an error. note: the diff command breaks the convention and uses both 0 and 1 as normal exit codes.

• || is like && but only executes when $? is not 0. important: || has a lower precedence than &&! make sure to wrap your ||s in braces unless you want to capture everything before it. for example, an excerpt of a shell script might look like this:
  && { test -e myfile || touch myfile ;} \

• leftmost &&s and ||s are always centered within their 4-space indent.

• avoid using pipes (|) when possible, because only the exit code of the final command in any pipe is respected. instead, write intermediate data to /tmp when possible. remember to mount /tmp as a tmpfs at the start of the RUN command!

• functions are declared (()) on their own line without any braces or other commands.

• { : is used to begin a multi-line block, similar to how RUN : begins a shell command. sadly, the space between the brace and the colon is not optional.

• ;} is used to finish a multi-line block. this is aligned with the preceeding && and || operators. thankfully, the space between ; and } is optional.

• a final ; is used to finish off the line-continutation. this means that every line before it can end with a \, which simplifies editing. when podman gives you strange errors, double-check that you aren't forgetting a \ or the final semicolon.

• redirections are usually specified before a final list of arguments. this makes skimming code easier when arguments extend across many lines.

• TODO: explain if cmd \␤ ;then : \␤ && cmd \␤ ;fi.

• TODO: explain for cmd \␤ ;do : \␤ && cmd \␤ ;done.

• TODO: explain while cmd \␤ ;do : \␤ && cmd \␤ ;done.

• TODO: explain tests?

• TODO: explain the two different kinds of commends.