* bin/evaluate.in (main): Prepend LOAD-PATH to '%load-path' for the
dynamic extend of the 'primitive-load' call.

Fixes a thinko introduced in
1872dd95.

* bin/evaluate.in (main): Make sure 'real-build-things' is bound.

* bin/evaluate.in (main): Use SPEC's #:name as a key in ARGS.
* examples/random-jobs.scm (make-random-jobs): Adjust accordingly.

Previously we would make a SQL query that would return many build jobs,
and then call 'delete-duplicates' on that.  This was extremely wasteful
because the list of returned by the query was huge leading to a heap of
several tens of GiB on a big database, and 'delete-duplicates' would
lead to more GC and it would take ages.

Furthermore, since 'delete-duplicates' is written in C as of Guile
2.2.3, it is uninterruptible from Fiber's viewpoint.  Consequently, the
kernel thread running the 'restart-builds' fiber would never schedule
other fibers, which could lead to deadlocks--e.g., since fibers are
scheduled on a circular shuffled list of kernel threads, once every N
times, a web server fiber would be sent to that kernel thread and not be
serviced.

* src/cuirass/base.scm (shuffle-jobs): Remove.
(shuffle-derivations): New procedure.
(spawn-builds): Take a list of derivations instead of a list of jobs.
(restart-builds): Remove 'builds' parameter.  Remove 'delete-duplicates'
call.  Remove done/remaining partitioning.
(build-packages): Adjust to pass 'spawn-builds' a list of derivations.
* bin/cuirass.in (main): Remove computation of PENDING.  Remove second
parameter in call to 'restart-builds'.

* bin/evaluate.in (main): Always pass an alist as the arguments to PROC,
containing at least 'file-name' and 'revision'.
* examples/random-jobs.scm (make-random-jobs): Display 'file-name' and
'revision' from ARGUMENTS.

This avoids copying things back and forth.

* src/cuirass/base.scm (fetch-repository): Add #:writable-copy?
parameter.  Call 'make-writable-copy' when it's true.
(copy-repository-cache): Remove.
(make-writable-copy): New procedure.
(evaluate): Add 'source' parameter and pass it to the 'evaluate' program.
(process-specs): Define 'compile?'.  Pass #:writable-copy? to
'fetch-repository'.  Remove call to 'copy-repository-cache'.  Remove
computation of the checkout directory name.  Pass CHECKOUT to 'evaluate'.
* bin/evaluate.in (main): Replace 'cachedir' with 'source'.  Remove
computation of the checkout directory name.

* bin/evaluate.in (main): 'build-things' replacement no longer calls 'exit'.

* bin/cuirass.in (main): Add 'setvbuf' calls.

Fixes <https://bugs.gnu.org/30618>.
Reported by Andreas Enge <andreas@enge.fr>.

* bin/evaluate.in (fill-job): Remove.
(main): Remove 'database' command-line argument.  Remove DB and its
uses.  Write an (evaluation EVAL JOBS) sexp.
* src/cuirass/base.scm (evaluate)[augment-job]: New procedure.
Use it.  Adjust to read (evaluation EVAL JOBS) sexp.  Call
'db-add-evaluation' and 'db-add-derivation'.

* src/cuirass/base.scm (clear-build-queue): New procedure.
* bin/cuirass.in (main): Call it.

* bin/cuirass.in (show-help, %options): Add "--threads".
(main): Honor it.  Pass #:parallelism to 'run-fibers'.

* src/cuirass/logging.scm (log-monitoring-stats): New procedure.
* bin/cuirass.in (main): Add a fiber that calls it regularly.

* bin/cuirass.in (main): Pass #:parallel? to 'spawn-fiber' for the web
server.

* src/cuirass/utils.scm (essential-task): New procedure.
* bin/cuirass.in (main): Wrap each fiber in 'essential-task'.

* bin/cuirass.in (main): Pass #:hz to 'run-fibers'.

* bin/cuirass.in (main): Add 'exit-channel' and read from it.
Catch exceptions in the main fiber and write to that channel upon
error.

* bin/cuirass.in (main): Print message while fetching pending builds.

Previously STORE would be closed on the first context switch, before
'restart-builds' has completed, leading to an error (writing to a closed
port).

* src/cuirass/base.scm (restart-builds): Remove 'store' parameter.  Wrap
body in 'with-store'.
* bin/cuirass.in (main): Adjust accordingly.

* bin/cuirass.in: Import (guix store).

* src/cuirass/database.scm (db-get-builds)[format-where-clause]:
Honor (status pending) filter.
* src/cuirass/base.scm (restart-builds): New procedure.
* bin/cuirass.in (main): Fetch pending builds.  Start fiber that invokes
'restart-builds' on them.

This seems to fix a bug whereby the server would never process the FD
returned by 'accept'.

* bin/cuirass.in (main): Move 'run-cuirass-server' outside of
'spawn-fiber'.

* README: Mark Fibers as required.
* configure.ac: Check for Guile 2.2 only.  Check for (fibers).
* bin/cuirass.in (main): Use (fibers).  Run 'process-specs' and web
server in separate fibers.
* src/cuirass/base.scm (with-store): New macro.
(non-blocking-port): New procedure.
(evaluate): Use 'non-blocking-port'.  Use 'read-string' followed by 'read'.
(process-specs): Move 'db-add-stamp' right after 'string=?' comparison.
Run evaluation and subsequent builds in a separate fiber.
* src/cuirass/http.scm (run-cuirass-server): Pass 'fibers as the second
argument to 'run-server'.  Use 'log-message' instead of 'format'.
* src/cuirass/database.scm (with-database): Remove 'dynamic-wind'.

* src/cuirass/base.scm (prepare-git): New procedure.  Body moved from...
(process-specs): ... here.  Remove it.
* bin/cuirass.in (main): Call 'prepare-git'.

* bin/cuirass.in (show-help): Add help for --listen option.
(%options): Add listen option.
(main): Pass host to run-cuirass-server.
* doc/cuirass.texi (Invocation): Add --listen option.
* src/cuirass/http.scm (run-cuirass-server): Add named #:host parameter.

integration.

* bin/evaluate.in (fill-job): New procedure.
(main): Use it to fill informations (nix-name, system) that will later be
added to database.
* doc/cuirass.texi (Database)[Derivation]: Add system and nix_name fields.
(Database)[Builds]: Add id, status, timestamp, starttime and stoptime
fields. Remove output field.
(Database)[Outputs]: New table describing the build outputs.
* src/cuirass/base.scm (build-packages): Add new fields to build object before
adding it to database.
* src/cuirass/database.scm (db-get-build, db-get-builds): New procedures to get
a build by id from database and a list of builds using filter parameters
respectively.
* src/schema.sql (Outputs) : New table.
(Derivations): Add system and nix_name columns.
(Builds): Remove output column and add id, status, timestamp, starttime and
stoptime columns.

* src/cuirass/base.scm (%fallback?): New exported parameter.
(process-specs)[set-build-options]: Pass fallback parameter.
* bin/cuirass.in (%options): Add fallback.
(main): Set %fallback? parameter.
(show-help): Update.

* bin/cuirass.in: Replace λ by lambda.
* bin/evaluate.in: Ditto.
* build-aux/guix.scm: Ditto.
* examples/gnu-system.scm: Ditto.
* examples/guix-track-git.scm: Ditto.
* src/cuirass.scm: Ditto.
* src/cuirass/base.scm: Ditto.
* src/cuirass/database.scm: Ditto.
* src/cuirass/ui.scm: Ditto.
* src/cuirass/utils.scm: Remove λ* macro.

* src/cuirass/utils.scm (mkdir-p, make-user-module,
  call-with-temporary-directory, with-directory-excursion): Remove because
  already defined in guix.
* tests/utils (with-directory-excursion): Remove associated test.
* src/cuirass/base.scm: Use (guix build utils) to provide procedure removed
  from (cuirass utils).
* bin/evaluate.in: Ditto.
* bin/cuirass.in: Use "make-user-module" provided by (guix ui).

This fixes an issue in the database, where rows from the 'Evaluations' table
were having an "#f" value instead of a valid reference to their corresponding
specification.

* bin/evaluate.in (main): Use specification #:name instead of obsolete #:id.

* bin/cuirass.in (%options): Add "--load-path" and "-L" command line options.
(show-help): Adapt.
* src/cuirass/base.scm (%guix-package-path): New parameter.
(set-guix-package-path!): New procedure.
(evaluate): Call "evaluate" script with '%guix-package-path'.
* bin/evaluate.in (main): Match 'guix-package-path' command line argument and
handle it.

Signed-off-by: Mathieu Lirzin <mthl@gnu.org>

* bin/cuirass.in (%options): Add "--port" and "-p" command line options.
(show-help): Adapt.
(main): Set default to 8080.  Call 'run-cuirass-server' with this.
* src/cuirass/http.scm (run-cuirass-server): Display the port number.
* doc/cuirass.texi (Invocation): Document new option.

This reverts most of 4f0d6657.  The load paths
are still hard coded after installation, however 'pre-inst-env' wrapper is
used for defining them in the local build environment.  This is more
convenient for running the tests manually or launching a Guile REPL with all
the Cuirass modules accessible.

* build-aux/pre-inst-env.in (GUILE_LOAD_PATH, GUILE_LOAD_COMPILED_PATH): New
variables.
* bin/cuirass.in: Comment Guile load paths.
* Makefile.am (local_load_path, local_load_compiled_path): Delete.
(AM_TESTS_ENVIRONMENT): Don't set Guile load paths.
(do_subst): Use install directories for Guile load paths.
(install-exec-hook): Uncomment 'cuirass' hard coded load paths.

* configure.ac (bin/cuirass, bin/evaluate): Let 'make' generate them.
* bin/cuirass.in: Hard code load paths to find Cuirass modules.
* Makefile.am (do_subst): Update substitution.
(bin/cuirass, bin/evaluate): New rules.
(EXTRA_DIST): Distribute 'bin/cuirass.in' and 'bin/evaluate.in'.
(MOSTLYCLEANFILES): Add '$(bin_SCRIPTS)'.
(install-exec-hook): New target.
(local_load_path, local_load_compiled_path): New variables.
(AM_TESTS_ENVIRONMENT): Set Guile load paths.
* build-aux/pre-inst-env.in: Don't modify Guile load paths.

* src/cuirass.scm: Re-export (cuirass http) module.
* bin/cuirass.in (main): Run HTTP server and move the 'process-specs'
loop in another thread.

* bin/cuirass (main): Remove 'let' before main loop.

* guix.scm: New file; specify Guix package.
* guix/ci.scm: New file; expose to Guix.
* build-aux/pre-inst-env.in: Add it to GUIX_PACKAGE_PATH.
* bin/evaluate.in (main): Lookup proc using name specified by #:proc.
* tests/guix-track-git.scm: New file.
* tests/hello-git.scm: Test it.

Signed-off-by: Mathieu Lirzin <mthl@gnu.org>

* bin/cuirass.in (options): Add --use-substitutes.
(show-help): Idem.
(main): Set %use-substitutes?.

Signed-off-by: Mathieu Lirzin <mthl@gnu.org>

* bin/cuirass.in (main): Use 'let' instead of 'let*'.

* src/cuirass.scm: New file.
* Makefile.am (guilesitedir, dist_guilesite_DATA)
(nodist_guilesite_DATA): New variables.
(pkgmoduledir): Refer to 'guilesitedir'.
(CLEANFILES): Add 'nodist_guilesite_DATA'.
* bin/cuirass.in: Use (cuirass) module.
* bin/evaluate.in: Likewise.

* src/schema.sql (Evaluations): Add 'revision' column.
* src/cuirass/database.scm (db-add-evaluation): Adapt.
All callers changed.