notwa/stargazing
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

wip

Browse source
This commit is contained in:
Connor Olding 2022-10-14 16:35:09 -07:00
parent 99a99ae08a
commit 8356e07b42
15 changed files with 268 additions and 72 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

68
README.md
View file

@ -13,6 +13,74 @@ polyfilling missing features across platforms,
but pre-existing software still takes a lot of
patching before becoming fully functional.
## procedures
in order of dependence, then alphabetical order.
### Phase 1
* **notwa-util:** convenient shell scripts for
augmenting common tasks relating to building.
this also builds a busybox rescue shell,
configured specifically for containers.
### Phase 2
<!--
the base cosmopolitan library and binaries,
with some minor patches and a custom distrib.
-->
* **cosmo-build:** builds the basic libraries (i.e.
cosmopolitan libc) and binaries to bootstrap itself
and build other software against. also includes `unzip.com`.
### Phase 3
* **cosmo-assimilate:** converts Actually Portable Executables
into native binaries.
* **cosmo-awk:** Actually Portable awk (fork of One True Awk)
* **cosmo-build-all:** builds almost everything else from
the cosmopolitan repository (i.e. first-class third-party support)
the "flavor" used can be different than that of cosmo-build.
* **cosmo-dist:** only contains the essentials from cosmo-build.
* **cosmo-make:** Landlocked Make (fork of GNU Make)
* **cosmo-sed:** Actually Portable sed (fork of NetBSD sed)
### Phase 4
* **cosmo-base:** combines cosmo-dist, cosmo-assimilate, cosmo-awk,
and cosmo-sed. this is used for building other containers.
* **cosmo-chibicc:** chibicc (not sure which commit this is forked from)
with many extensions.
* **cosmo-python:** Actually Portable Python 3.6.14
* **cosmo-quickjs:** quickjs (2021-03-27)
* **cosmo-sqlite:** sqlite3 (3350500) binaries and libraries.
* **cosmo-unbourne:** a fork of dash (v0.5.11.5) with extended interactivity.
### Phase 5
* **cosmo-kuroko**: kuroko (bleeding edge), a dialect of Python 3.
kuroko combines Python's syntax with Lua's level of embeddability,
and it can be used standalone as well.
* **cosmo-muon**: muon (bleeding edge), a reimplementation of mesen, written in C.
* **cosmo-perl**: Actually Portable Perl, thanks to Gavin Hayes
* **cosmo-yices**: yices (bleeding edge), SAT + SMT (SMTLIB, SMTLIB2) solvers.
also includes **cosmo-kissat** (bleeding edge), one of the best SAT solvers.
## building
on most Linux distros with podman properly installed and configured,

46
build-all
View file

@ -24,22 +24,40 @@ dbg_retrieve() { : \
&& yes_retrieve "$container" "$@" \
;}
build() { : \
&& local t="${1?missing argument}" && shift \
&& printf >&2 ' \033[1m\033[7m <-- \033[0m\033[1m ' \
&& printf >&2 '%22sbuilding %-19s%13s' '' "$t" '' \
&& printf >&2 ' \033[1m\033[7m --> \033[0m\n' \
&& podman build -t "$t" "$t" "$@" \
;}
: \
&& me="$(readlink -f "$0")" && cd "${me%/*}" && { [ -d out ] || mkdir out ;} \
\
&& podman build -t notwa-util notwa-util \
&& podman build -t cosmo cosmo \
&& podman build -t cosmo-kuroko cosmo-kuroko \
&& podman build -t cosmo-muon cosmo-muon \
&& podman build -t cosmo-perl cosmo-perl \
&& podman build -t cosmo-yices cosmo-yices \
;: \
`# phase 1` \
&& build notwa-util \
`# phase 2` \
&& build cosmo-build \
`# phase 3` \
&& build cosmo-assimilate \
&& build cosmo-awk \
&& build cosmo-build-all \
&& build cosmo-dist \
&& build cosmo-make \
&& build cosmo-sed \
`# phase 4` \
&& build cosmo-base \
&& echo build cosmo-chibicc \
&& echo build cosmo-python \
&& echo build cosmo-quickjs \
&& echo build cosmo-sqlite \
&& echo build cosmo-unbourne \
`# phase 5` \
&& build cosmo-kuroko \
&& build cosmo-muon \
&& build cosmo-perl \
&& build cosmo-yices \
\
&& bin=/cosmopolitan/bin \
&& yes_retrieve cosmo $bin/ape $bin/awk.com $bin/gzip.com $bin/make.com \
$bin/sed.com $bin/unbourne.com $bin/unzip.com $bin/zip.com \
&& dbg_retrieve cosmo-kuroko /bin/kuroko.com \
&& dbg_retrieve cosmo-muon /bin/muon.com \
&& dbg_retrieve cosmo-perl /bin/perl.com \
&& dbg_retrieve cosmo-yices /bin/kissat.com /bin/yices-sat.com \
/bin/yices-smt.com /bin/yices-smt2.com /bin/yices.com \
;

12
cosmo-assimilate/Dockerfile Normal file
View file

@ -0,0 +1,12 @@
FROM cosmo-build AS builder
RUN --mount=type=tmpfs,target=/tmp : \
&& . ./env \
&& ./expand ./makeit b//assimilate.com \
&& ./expand cp -t /usr/bin b//assimilate.com b//assimilate.com.dbg \
;
FROM scratch AS runner
COPY --chmod=0755 --from=localhost/notwa-util /nu/busybox /bin/busybox
COPY --chmod=0755 --from=builder /cosmopolitan/build/bootstrap/ape.elf /bin/ape
COPY --chmod=0755 --from=builder /usr/bin/assimilate.com /usr/bin/assimilate.com.dbg /bin/
ENTRYPOINT ["/bin/ape", "/bin/assimilate.com"]

12
cosmo-awk/Dockerfile Normal file
View file

@ -0,0 +1,12 @@
FROM cosmo-build AS builder
RUN --mount=type=tmpfs,target=/tmp : \
&& . ./env \
&& ./expand ./makeit t//awk/awk.com \
&& ./expand cp -t /usr/bin t//awk/awk.com t//awk/awk.com.dbg \
;
FROM scratch AS runner
COPY --chmod=0755 --from=localhost/notwa-util /nu/busybox /bin/busybox
COPY --chmod=0755 --from=builder /cosmopolitan/build/bootstrap/ape.elf /bin/ape
COPY --chmod=0755 --from=builder /usr/bin/awk.com /usr/bin/awk.com.dbg /bin/
ENTRYPOINT ["/bin/ape", "/bin/awk.com"]

4
cosmo-base/Dockerfile Normal file
View file

@ -0,0 +1,4 @@
FROM cosmo-dist AS runner
COPY --chmod=0755 --from=localhost/cosmo-assimilate /bin/assimilate.com /cosmopolitan/bin/
COPY --chmod=0755 --from=localhost/cosmo-awk /bin/awk.com /cosmopolitan/bin/
COPY --chmod=0755 --from=localhost/cosmo-sed /bin/sed.com /cosmopolitan/bin/

34
cosmo-build-all/Dockerfile Normal file
View file

@ -0,0 +1,34 @@
FROM localhost/cosmo-build AS builder
ARG COSMO_FLAVOR=def
ARG COSMO_REBUILD=1
RUN : \
&& . ./env \
&& if [ "$COSMO_REBUILD" = 1 ] || [ "$flavor" != "$COSMO_FLAVOR" ] \
;then : \
&& rm -r o \
;else : \
&& mv o o.old \
;fi \
# create a mapping from flavor to mode (possibly an empty string) \
&& { [ "$COSMO_FLAVOR" != def ] && mode="$COSMO_FLAVOR" || mode= ;} \
&& printf '%s="%s"\n' >env \
commit "$commit" \
flavor "$COSMO_FLAVOR" \
mode "$mode" \
;
RUN --mount=type=cache,id=cosmo-build-all,target=/cosmopolitan/o,sharing=locked \
--mount=type=tmpfs,target=/tmp : \
&& . ./env \
&& if [ -e o.old ] \
;then : \
&& cp o.old/* o/ \
&& rm -r o.old \
;fi \
\
# skip some checks that are currently failing: \
&& touch o//test/libc/calls/getgroups_test.com.runs \
&& touch o//test/libc/calls/getpriority_test.com.runs \
\
&& ./makeit MODE="$mode" all \
;

75
cosmo/Dockerfile → cosmo-build/Dockerfile
View file

@ -3,6 +3,7 @@
ARG COSMO_FLAVOR=def
ARG COSMO_COMMIT=f52f65b2e351085250b743a5319dab418f2d48be
ARG COSMO_SHA256=8a13f4be3d689b8895af6e1178d455db2f82a69e33ce5848e2f2f14f96760d4c
ARG COSMO_REBUILD=0
FROM localhost/notwa-util AS downloader
ARG COSMO_COMMIT COSMO_SHA256
@ -26,6 +27,7 @@ RUN : \
&& shed build/config.mk 's/-march=native/-march=haswell -mno-pclmul -mtune=generic/' \
# fix execution on alpine (although installing the loader mitigates this) \
&& sed -i 's/skip=" 76" count=" 128"/skip=76 count=128 /' build/bootstrap/*.com \
\
&& install -m0755 build/bootstrap/ape.elf /usr/bin/ape \
&& shed Makefile \
# TODO: apply changes to COSMOPOLITAN_HEADERS as to include them in cosmopolitan.h \
@ -59,7 +61,7 @@ RUN : \
'(T//*) f="third_party/$g";;' \
'esac; $once && set -- "$@" "$f" || set -- "$f"; once=true; done' \
'"$@"' \
# don't forget to do this \
# i always forget to do this \
&& chmod +x makeit expand \
;
@ -79,63 +81,32 @@ RUN --mount=type=tmpfs,target=/tmp : \
b//mkdir.com b//package.com b//pwd.com \
b//rm.com b//touch.com b//unbundle.com \
b//zipobj.com \
# these are essential too! \
&& ./expand ./makeit \
e//unbourne.com t//awk/awk.com t//sed/sed.com \
t//unzip/unzip.com t//zip/zip.com b//assimilate.com \
;
# install the basics (no executable binaries besides ape.elf)
RUN : \
&& . ./env \
&& mkdir dist "dist/$flavor" "dist/$flavor/public" \
&& echo /dist >>.gitignore \
&& ./expand cp -t "dist/$flavor" \
# optionally use the newly-built bootstrapped binaries to build themselves again.
ARG COSMO_REBUILD
RUN --mount=type=tmpfs,target=/tmp : \
&& { [ "$COSMO_REBUILD" = 1 ] || exit 0 ;} \
&& rm -r o \
\
&& ./expand ./makeit \
a//ape-copy-self.o a//ape-no-modify-self.o a//ape.elf \
a//ape.lds a//ape.macho a//ape.o \
o//cosmopolitan.a o//libc/crt/crt.o o/cosmopolitan.h \
&& cp -t "dist/$flavor/public" "o/$mode/ape/public/ape.lds" \
\
&& install -m0755 "dist/$flavor/ape.elf" /usr/bin/ape \
&& find libc -name '*.h' >headers.txt \
&& ./expand tar zcf dist/headers.tar.gz -T headers.txt \
ape/config.h libc/integral T//gdtoa/gdtoa.h \
T//getopt/getopt.h T//linenoise/linenoise.h T//musl/crypt.h \
T//musl/fnmatch.h T//musl/lockf.h T//musl/passwd.h \
T//musl/rand48.h T//musl/tempnam.h T//regex/regex.h \
T//zlib/zconf.h T//zlib/zlib.h \
;
# install the rest
FROM builder AS distrib
RUN --mount=type=tmpfs,target=/tmp : \
&& . ./env \
&& find o -name '*.sym' -delete \
&& dedupe o/third_party/gcc /distrib/gcc/ \
bin/x86_64-linux-musl-gcc \
bin/x86_64-linux-musl-g++ \
bin/x86_64-linux-musl-ld.bfd \
&& dedupe dist /distrib/dist/ "$flavor/cosmopolitan.h" \
&& mkdir /tmp/bin \
# discover .com files by uncommenting this command: \
#&& find o -name '*.com' ! -type d -exec printf '%s\n' cp -t /tmp/bin {} + | sort && exit 1 \
&& ./expand cp -t /tmp/bin \
e//unbourne.com t//awk/awk.com t//make/make.com \
t//sed/sed.com t//unzip/unzip.com t//zip/zip.com \
b//ar.com b//assimilate.com b//cocmd.com \
a//public/ape.lds o//cosmopolitan.a o//libc/crt/crt.o \
o/cosmopolitan.h \
# build some extra tools for other containers to stem off of: \
&& rebuild() { ./expand ./makeit "$@" && ./expand cp -t build/bootstrap "$@" ;} \
&& rebuild \
t//make/make.com b//ar.com b//cocmd.com \
b//compile.com b//cp.com b//echo.com \
b//fixupobj.com b//gzip.com b//mkdeps.com \
b//mkdir.com b//package.com b//pwd.com \
b//rm.com b//rollup.com b//symtab.com \
b//touch.com b//unbundle.com b//zipobj.com \
&& dedupe /tmp/bin /distrib/bin/ \
#&& ln -s "../dist/$flavor/ape.elf" /distrib/bin/ape \
&& cp "dist/$flavor/ape.elf" /distrib/bin/ape \
&& cp /cosmopolitan/env /distrib/env \
b//rm.com b//touch.com b//unbundle.com \
b//zipobj.com \
;
FROM alpine:3.16 AS runner
COPY --from=distrib /distrib /cosmopolitan
ENV PATH="/cosmopolitan/bin:/cosmopolitan/gcc/bin/:/cosmopolitan/gcc/x86_64-linux-musl/bin/:$PATH"
ARG COSMO_FLAVOR
ENV COSMO_DIST="/cosmopolitan/dist/$COSMO_FLAVOR"
# these are essential too! \
RUN --mount=type=tmpfs,target=/tmp : \
&& ./expand ./makeit \
t//unzip/unzip.com t//zip/zip.com \
;

0
cosmo/make.com → cosmo-build/make.com
View file

56
cosmo-dist/Dockerfile Normal file
View file

@ -0,0 +1,56 @@
FROM localhost/cosmo-build AS builder
# install the basics (no executable binaries besides ape.elf)
RUN : \
&& . ./env \
&& mkdir dist "dist/$flavor" "dist/$flavor/public" \
&& echo /dist >>.gitignore \
&& ./expand cp -t "dist/$flavor" \
a//ape-copy-self.o a//ape-no-modify-self.o a//ape.elf \
a//ape.lds a//ape.macho a//ape.o \
o//cosmopolitan.a o//libc/crt/crt.o o/cosmopolitan.h \
&& cp -t "dist/$flavor/public" "o/$mode/ape/public/ape.lds" \
\
&& install -m0755 "dist/$flavor/ape.elf" /usr/bin/ape \
&& find libc -name '*.h' >headers.txt \
&& ./expand tar zcf dist/headers.tar.gz -T headers.txt \
ape/config.h libc/integral T//gdtoa/gdtoa.h \
T//getopt/getopt.h T//linenoise/linenoise.h T//musl/crypt.h \
T//musl/fnmatch.h T//musl/lockf.h T//musl/passwd.h \
T//musl/rand48.h T//musl/tempnam.h T//regex/regex.h \
T//zlib/zconf.h T//zlib/zlib.h \
;
# install the rest
FROM builder AS distrib
RUN --mount=type=tmpfs,target=/tmp : \
&& . ./env \
&& find o -name '*.sym' -delete \
&& dedupe o/third_party/gcc /distrib/gcc/ \
bin/x86_64-linux-musl-gcc \
bin/x86_64-linux-musl-g++ \
bin/x86_64-linux-musl-ld.bfd \
&& dedupe dist /distrib/dist/ "$flavor/cosmopolitan.h" \
&& mkdir /tmp/bin \
# discover .com files by uncommenting this command: \
#&& find o -name '*.com' ! -type d -exec printf '%s\n' cp -t /tmp/bin {} + | sort && exit 1 \
&& ./expand cp -t /tmp/bin \
t//make/make.com t//unzip/unzip.com t//zip/zip.com \
b//ar.com b//cocmd.com b//compile.com \
b//cp.com b//echo.com b//fixupobj.com \
b//gzip.com b//mkdeps.com b//mkdir.com \
b//package.com b//pwd.com b//rm.com \
b//rollup.com b//symtab.com b//touch.com \
b//unbundle.com b//zipobj.com \
&& dedupe /tmp/bin /distrib/bin/ \
#&& ln -s "../dist/$flavor/ape.elf" /distrib/bin/ape \
&& cp "dist/$flavor/ape.elf" /distrib/bin/ape \
&& cp /cosmopolitan/env /distrib/env \
;
FROM alpine:3.16 AS runner
COPY --from=distrib /distrib /cosmopolitan
ENV PATH="/cosmopolitan/bin:/cosmopolitan/gcc/bin/:/cosmopolitan/gcc/x86_64-linux-musl/bin/:$PATH"
# FIXME:
#ARG COSMO_FLAVOR
#ENV COSMO_DIST="/cosmopolitan/dist/$COSMO_FLAVOR"

2
cosmo-kuroko/Dockerfile
View file

@ -11,7 +11,7 @@ RUN --mount=type=cache,id=common,target=/media/common,sharing=locked \
&& acquire from=github repo=kuroko-lang/kuroko dest=/kuroko env=KUROKO \
;
FROM localhost/cosmo AS builder
FROM localhost/cosmo-base AS builder
COPY --from=downloader /kuroko /kuroko
WORKDIR /kuroko

5
cosmo-make/Dockerfile Normal file
View file

@ -0,0 +1,5 @@
FROM scratch AS runner
COPY --chmod=0755 --from=localhost/notwa-util /nu/busybox /bin/busybox
COPY --chmod=0755 --from=localhost/cosmo-dist /cosmopolitan/bin/ape /bin/
COPY --chmod=0755 --from=localhost/cosmo-dist /cosmopolitan/bin/make.com /bin/
ENTRYPOINT ["/bin/ape", "/bin/make.com"]

2
cosmo-muon/Dockerfile
View file

@ -8,7 +8,7 @@ RUN --mount=type=cache,id=common,target=/media/common,sharing=locked \
&& acquire from=sourcehut repo=lattis/muon dest=/muon env=MUON \
;
FROM localhost/cosmo AS builder
FROM localhost/cosmo-base AS builder
RUN apk add --no-cache ninja
COPY --chmod=0755 --from=localhost/notwa-util /nu/cosmocc /usr/bin/

10
cosmo-perl/Dockerfile
View file

@ -1,11 +1,15 @@
FROM localhost/notwa-util AS downloader
# MAIN: https://www.perl.org/
# REPO: https://github.com/Perl/perl5
# FORK: https://github.com/G4Vi/perl5/tree/cosmo
FROM localhost/notwa-util AS downloader
ARG PERL_COMMIT=66baebd8320a90ddf2c90de14b83f7eee4f6b143
ARG PERL_SHA256=0e6731efb5f25589116340b4e861abd43fb47520669fe903569b1489339dcddb
# MAIN: https://computoid.com/APPerl/
# REPO: https://github.com/G4Vi/Perl-Dist-APPerl
# MAIN: https://metacpan.org/pod/perltidy
# REPO: https://github.com/perltidy/perltidy
ARG PERLTIDY_VERSION=20220613
ARG PERLTIDY_SHA256=50496a6952904ef28f495919fc0a67801a63c87779c61308ce1ca5b32467c5d4
@ -22,7 +26,7 @@ RUN --mount=type=cache,id=common,target=/media/common,sharing=locked \
&& acquire \
;
FROM localhost/cosmo AS builder
FROM localhost/cosmo-base AS builder
COPY --from=downloader /perl /perl
RUN : \

12
cosmo-sed/Dockerfile Normal file
View file

@ -0,0 +1,12 @@
FROM cosmo-build AS builder
RUN --mount=type=tmpfs,target=/tmp : \
&& . ./env \
&& ./expand ./makeit t//sed/sed.com \
&& ./expand cp -t /usr/bin t//sed/sed.com t//sed/sed.com.dbg \
;
FROM scratch AS runner
COPY --chmod=0755 --from=localhost/notwa-util /nu/busybox /bin/busybox
COPY --chmod=0755 --from=builder /cosmopolitan/build/bootstrap/ape.elf /bin/ape
COPY --chmod=0755 --from=builder /usr/bin/sed.com /usr/bin/sed.com.dbg /bin/
ENTRYPOINT ["/bin/ape", "/bin/sed.com"]

2
cosmo-yices/Dockerfile
View file

@ -28,7 +28,7 @@ RUN --mount=type=cache,id=common,target=/media/common,sharing=locked \
&& acquire from=github repo=arminbiere/kissat dest=/kissat env=KISSAT \
;
FROM localhost/cosmo AS builder
FROM localhost/cosmo-base AS builder
RUN apk add --no-cache autoconf gperf