Previously we'd systematically add the first argument to the search
path.  When that first argument was the empty string (as with the
'guix-modular' jobset), we'd thus add the current directory to the
search path, even if the intent was to leave the load path unchanged.

Furthermore, a3a7c09b changed load-path
handling to be explicit in Scheme.

* bin/evaluate.in: Remove GUILE_LOAD_PATH assignment.

* bin/evaluate.in (with-directory-excursion): Remove.
(main): chdir to SOURCE.

This avoids a situation whereby, when evaluating from a Guix checkout,
we'd have already loaded slightly different and incompatible (guix …)
modules.

Hydra's 'hydra-eval-guile-jobs' implemented the same solution as in this
patch already.

* bin/evaluate.in: Remove use of (cuirass …) and (guix …) modules.
(ref): New procedure.
(with-directory-excursion): New macro.
(main): Use 'ref'.  Remove uses of Guix or Cuirass modules.

The previous load path was potentially incorrect since commit
2fe7ff87.

* src/cuirass/base.scm (evaluate)[tokenize, load-path]: New variables.
Assume #:load-path is colon-separated.  Pass LOAD-PATH as the second
argument to 'evaluate'.
* doc/cuirass.texi (Database): Adjust documentation.

* bin/evaluate.in (main): Prepend LOAD-PATH to '%load-path' for the
dynamic extend of the 'primitive-load' call.

Fixes a thinko introduced in
1872dd95.

* bin/evaluate.in (main): Make sure 'real-build-things' is bound.

* bin/evaluate.in (main): Use SPEC's #:name as a key in ARGS.
* examples/random-jobs.scm (make-random-jobs): Adjust accordingly.

Previously we would make a SQL query that would return many build jobs,
and then call 'delete-duplicates' on that.  This was extremely wasteful
because the list of returned by the query was huge leading to a heap of
several tens of GiB on a big database, and 'delete-duplicates' would
lead to more GC and it would take ages.

Furthermore, since 'delete-duplicates' is written in C as of Guile
2.2.3, it is uninterruptible from Fiber's viewpoint.  Consequently, the
kernel thread running the 'restart-builds' fiber would never schedule
other fibers, which could lead to deadlocks--e.g., since fibers are
scheduled on a circular shuffled list of kernel threads, once every N
times, a web server fiber would be sent to that kernel thread and not be
serviced.

* src/cuirass/base.scm (shuffle-jobs): Remove.
(shuffle-derivations): New procedure.
(spawn-builds): Take a list of derivations instead of a list of jobs.
(restart-builds): Remove 'builds' parameter.  Remove 'delete-duplicates'
call.  Remove done/remaining partitioning.
(build-packages): Adjust to pass 'spawn-builds' a list of derivations.
* bin/cuirass.in (main): Remove computation of PENDING.  Remove second
parameter in call to 'restart-builds'.

* src/cuirass/database.scm (db-get-pending-derivations): New procedure.
* tests/database.scm ("database")["db-get-pending-derivations"]: New test.

* bin/evaluate.in (main): Always pass an alist as the arguments to PROC,
containing at least 'file-name' and 'revision'.
* examples/random-jobs.scm (make-random-jobs): Display 'file-name' and
'revision' from ARGUMENTS.

* src/cuirass/utils.scm (make-critical-section)
(call-with-critical-section): New procedures.
(with-critical-section): New macro.
* src/cuirass/http.scm (with-database-access): Remove.
(handle-build-request, handle-builds-request, url-handler): Use
'with-critical-section' instead of 'with-database-access'.
(run-cuirass-server): Remove 'spawn-fiber' call.  Use
'make-critical-section' instead.

* src/cuirass/base.scm (make-writable-copy)[chmod+w]: New procedure.
Replace 'system*' call with 'file-system-fold' call.

This avoids copying things back and forth.

* src/cuirass/base.scm (fetch-repository): Add #:writable-copy?
parameter.  Call 'make-writable-copy' when it's true.
(copy-repository-cache): Remove.
(make-writable-copy): New procedure.
(evaluate): Add 'source' parameter and pass it to the 'evaluate' program.
(process-specs): Define 'compile?'.  Pass #:writable-copy? to
'fetch-repository'.  Remove call to 'copy-repository-cache'.  Remove
computation of the checkout directory name.  Pass CHECKOUT to 'evaluate'.
* bin/evaluate.in (main): Replace 'cachedir' with 'source'.  Remove
computation of the checkout directory name.

Fixes a regression introduced in
1bab5c4e whereby the
'status+submission-time' order would no longer be honored.

As a result, /api/queue would return the queue ordered by build IDs,
making it largely useless.

* src/cuirass/database.scm (db-get-builds): Remove 'order' and rename
'order-column-name' to 'order'.  Add case for 'status+submission-time'.
* tests/database.scm ("database")["db-get-builds"]: Move below
"db-update-build-status!" test.  Add case for the
'status+submission-time' order.

This reinstates c47dfdf8 and fixes the
issues that led to the revert in
b71f0cdc.

Before that, 'run-server' would force sequential processing of client
requests one after another.

* src/cuirass/http.scm (run-cuirass-server): Rewrite to use its own loop
instead of 'run-server'.  Spawn a database fiber.
(with-database-access): New macro.
(handle-build-request): Expect 'db-channel' and use 'with-database-access'.
(handle-builds-request): Likewise.
(url-handler): Likewise.

* src/cuirass/base.scm (cancel-old-builds): New procedure.

* bin/evaluate.in (main): 'build-things' replacement no longer calls 'exit'.

Fixes a bug whereby some fibers would get a SQLITE_BUSY exception while
accessing the database: see
<https://debbugs.gnu.org/cgi/bugreport.cgi?bug=30644#26>.

Suggested by Danny Milosavljevic <dannym@scratchpost.org>.

* src/cuirass/database.scm (wal-mode): Rename to...
(set-db-options): ... this.  Add call to 'sqlite-exec' for
'busy_timeout'.

* bin/cuirass.in (main): Add 'setvbuf' calls.

* src/cuirass/base.scm (with-store): Rewrite using 'unwind-protect'.
* src/cuirass/database.scm (with-database): Likewise.

* src/cuirass/utils.scm (unwind-protect): New macro.

Fixes a regression introduced in
f083282f, whereby we'd return 0 as the
timestamp for everything in /api/queue.

* src/cuirass/http.scm (build->hydra-build): Make 'finished?' a
Boolean.  Move 'bool->int' call in #:finished definition.

Previously, when an exception was raised from
'handle-build-event' (e.g., a "database is locked" error), we'd throw,
thereby leaving PORT open and we'd never read from it again.  Thus, the
corresponding 'guix-daemon' process would eventually get stuck in a
'write' call to that socket, and its build processes would stall.

* src/cuirass/base.scm (exception-reporter): New procedure.
(spawn-builds): Use it.

Fixes <https://bugs.gnu.org/30618>.
Reported by Andreas Enge <andreas@enge.fr>.

* bin/evaluate.in (fill-job): Remove.
(main): Remove 'database' command-line argument.  Remove DB and its
uses.  Write an (evaluation EVAL JOBS) sexp.
* src/cuirass/base.scm (evaluate)[augment-job]: New procedure.
Use it.  Adjust to read (evaluation EVAL JOBS) sexp.  Call
'db-add-evaluation' and 'db-add-derivation'.

Fixes <https://bugs.gnu.org/30645>.
Reported by Andreas Enge <andreas@enge.fr>.

* src/cuirass/base.scm (spawn-builds): Fix TOTAL vs. COUNT mismatches.

* src/cuirass/http.scm (build->hydra-build): Set #:timestamp to
  #:stoptime when BUILD is finished.

* src/cuirass/database.scm (db-get-builds): Reindent.

* src/cuirass/database.scm (db-get-builds)[assqx-ref]: Rewrite with
'match'.

* src/cuirass/http.scm (request-parameters): Return the empty list when
QUERY is #f.

Fixes a regression introduced in
593cb7be.

* src/cuirass/http.scm (request-parameters): Fix fallback case in
'match' form.  Previously it would return a procedure in this case, as
returned by (const param), leading to a failure down the road in
'sqlite-bind-arguments' as could be seen by running tests/http.scm.

I've seen 'scandir' report #f once, even though that's theoretically
impossible.

* src/cuirass/logging.scm (log-monitoring-stats): Return '() if
'scandir' returns #f.

* src/cuirass/database.scm (db-get-builds): Fix grouping.

* src/cuirass/http.scm (request-parameters): Interpret id and nr parameters as
numbers.

* src/cuirass/database.scm (db-get-builds): Remove debugging output.
(db-get-build): Remove debugging output.

* src/cuirass/http.scm (url-handler): Convert build-id URL part to number.

* src/cuirass/database.scm (db-get-builds): Inline output selection.

* src/cuirass/database.scm (db-get-builds): Modify.
(db-get-build): Modify.

* Makefile.am (CLEANFILES): Add $(nodist_guilesite_DATA).

Previously it would be installed under cuirass/.

* Makefile.am (webmoduledir, webobjectdir): New variables.
(dist_pkgmodule_DATA): Remove src/web/server/fiberized.scm.
(dist_webmodule_DATA, nodist_webobject_DATA): New variables.
(CLEANFILES): Add $(nodist_webobject_DATA).

* src/cuirass/base.scm (clear-build-queue): New procedure.
* bin/cuirass.in (main): Call it.