The new macro automatically takes care of inserting question marks in
the SQL queries, which in turn guarantees that there are always as many
question marks and arguments.

* src/cuirass/database.scm (sqlite-exec): Rename to...
(%sqlite-exec): ... this.
(sqlite-exec/bind, sqlite-exec): New macros.
(assq-refs): Remove.
(db-add-specification): Use the new 'sqlite-exec' form.
(db-get-specifications): Correctly deal with REV or TAG being #f.
(db-add-derivation, db-get-derivation, db-add-evaluation)
(db-add-build, db-update-build-status!, db-get-outputs)
(db-get-build, db-get-stamp, db-add-stamp): Adjust to the new
'sqlite-exec' form.

* examples/random-jobs.scm (random-derivation): Tweak the computation of
SEED.  Actually use the new random state.

Previously any derivation not directly built by Cuirass would be
considered as failed because 'handle-build-event' wouldn't see any build
event.  Here we just make sure the build status recorded in the database
corresponds to reality.

* src/cuirass/base.scm (update-build-statuses!): New procedure.
(spawn-builds): Call it after 'build-derivations&'.

* src/cuirass/base.scm (build-packages)[register]: Make 'db-add-build' a
tail call.
Fix computation of 'outs'.

* examples/random-jobs.scm (make-job): Add '.' in #:job-name.

* src/cuirass/database.scm (db-update-build-status!): And "AND status != ?"
in SQL queries.
* tests/database.scm ("database")["db-update-build-status!"]: Add call
to 'db-update-build-status!'.

This is purely "defensive programming" since the default value for
'print-build-trace' is currently #t.

* src/cuirass/base.scm (with-store): Pass #:print-build-trace to
'set-build-options'.

Suggested by Danny Milosavljevic.

* src/cuirass/database.scm (sqlite-exec): Pass #:cache? to
'sqlite-prepare'.

* src/cuirass/database.scm (%sqlite-exec): Remove.
(sqlite-exec): Turn back into a procedure.  Use 'sqlite-bind'.  Add
'normalize' procedure and use it.
(db-add-specification, db-add-derivation, db-get-derivation)
(db-add-evaluation, db-add-build, db-update-build-status!)
(db-get-build, db-get-stamp, db-add-stamp): Use question marks in SQL
queries.
* src/cuirass/base.scm (build-packages)[register]: Make #:log
non-false.
* tests/database.scm (make-dummy-job): Add #:job-name, #:system,
 #:nix-name, and #:eval-id.  This is necessary because 'sqlite-bind'
would now translate #f to a real NULL (before it would translate to the
string "#f"...), and would thus report violations of the non-NULL
constraint.

Co-authored-by: Ludovic Courtès <ludo@gnu.org>

* src/cuirass/base.scm (build-packages): Remove useless 'log-message'
calls.

Reported by Danny Milosavljevic.

* src/cuirass/http.scm (url-handler) <"jobsets"> Return the result of
'db-get-specifications' as-is, not just the car.

* src/cuirass/base.scm (spawn-builds): Log the number of builds
performed, not the number of remaining builds.

* bin/cuirass.in (show-help, %options): Add "--threads".
(main): Honor it.  Pass #:parallelism to 'run-fibers'.

Works around <https://github.com/wingo/fibers/issues/19>.

The effect in practice would be that we'd usually not reach the
'close-pipe' call in 'evaluate', leaving zombie processes behind us,
never executing the continuation, and additionally spinning fast on a
sequence of epoll_wait/epoll_ctl calls.

* src/cuirass/base.scm <top level>: Monkey-patch (fibers internal).

* examples/random-jobs.scm, examples/random.scm: New files.
* Makefile.am (nobase_dist_pkgdata_DATA): Add them.

We used to have 'build-derivations' write to the custom binary port
returned by 'build-event-output-port'.  However, custom binary ports
constitute continuation barriers, thereby preventing fibers from being
suspended.

To make build log processing non-blocking, we therefore invert this
inversion of control and use a suspendable I/O procedure,
'read-line/non-blocking', when reading the build log.

* src/cuirass/base.scm (read-line/non-blocking, process-build-log)
(build-derivations&): New procedures.
(%newline, build-event-output-port): Remove.
(spawn-builds): Use 'build-derivations&' instead of 'build-derivations'
with 'build-event-output-port'.

* src/cuirass/http.scm (handle-builds-request): Wrap 'db-get-builds'
into 'non-blocking'.

* src/cuirass/logging.scm (log-monitoring-stats): New procedure.
* bin/cuirass.in (main): Add a fiber that calls it regularly.

* src/cuirass/database.scm (db-get-builds)[format-order-clause]: Add
'status+submission-time'.
* src/cuirass/http.scm (url-handler) <"queue">: Use it.

* src/cuirass/base.scm (shuffle-jobs): New procedure.
(spawn-builds): Use it.

* src/cuirass/base.scm (spawn-builds): Add 'log-message' call.

* build-aux/guix.scm (inputs): Add guile-fibers.

* examples/guix-track-git.scm (package->git-tracked): Add STORE parameter.
(guix-jobs): Pass value for STORE parameter.

* bin/cuirass.in (main): Pass #:parallel? to 'spawn-fiber' for the web
server.

* src/cuirass/utils.scm (essential-task): New procedure.
* bin/cuirass.in (main): Wrap each fiber in 'essential-task'.

* src/cuirass/http.scm (url-handler): Check whether REQUEST's method is
'GET, and return 405 if not.

This should placate 'guix-hydra-queued-builds', for instance, which
checks these values to choose between "Running" and "Scheduled".

* src/cuirass/http.scm (build->hydra-build): Provide the correct value
for #:finished and #:busy.

Suggested by Ricardo Wurmus.

* src/cuirass/database.scm (wal-mode): New procedure.
(db-open): Use it.

Fixes a bug whereby 'build-event-output-port' would sometimes read more
than COUNT from BV, which would usually result in a 'decoding-error'
exception from 'utf8->string'.

* src/cuirass/base.scm (build-event-output-port)[write!]: Use
'bytevector-range' to honor COUNT and OFFSET.

* src/cuirass/utils.scm (%weak-references): New variable.
(bytevector-range): New procedure.

Previously these calls would effectively suspend execution of all the
fibers until they had completed.

* src/cuirass/base.scm (process-specs): Wrap 'fetch-repository' and
'compile' calls in 'non-blocking'.

* src/cuirass/utils.scm (%non-blocking): New procedure.
(non-blocking): New macro.

* bin/cuirass.in (main): Pass #:hz to 'run-fibers'.

* bin/cuirass.in (main): Add 'exit-channel' and read from it.
Catch exceptions in the main fiber and write to that channel upon
error.

* src/cuirass/logging.scm (call-with-time-logging): New procedure.
(with-time-logging): New macro.
* src/cuirass/http.scm (handle-builds-request): Use it.

* bin/cuirass.in (main): Print message while fetching pending builds.

* tests/http.scm ("http")["cuirass-run"]: Run server with 'run-fibers'.

* src/cuirass/base.scm (restart-builds): Mark as succeeded the subset of
VALID with at least one valid output.

* src/cuirass/base.scm (handle-build-event)[valid?]: New procedure.
Use it when handling 'build-started', 'build-succeeded', and
'build-failed' events.

* src/web/server/fiberized.scm (socket-loop): Add 'log-message' call.
* src/cuirass/http.scm (url-handler): Likewise.