* src/cuirass/database.scm (build-weather): New macro.
(build-status->weather): New procedure.
(db-get-builds): Return the build weather using the new procedure.
* src/cuirass/http.scm (build->hydra-build): Also return the weather.
* src/cuirass/templates.scm (weather-class, weather-title): New procedures.
(build-eval-table): Display the weather.
* tests/database.scm ("db-get-build weather"): New tests.
* tests/http.scm (build-query-result): Adapt it.

* src/cuirass/database.scm (db-update-build-status): Only consider evaluations
that happened before the current one. This is useful when updating the build
status of old evaluations.

* src/cuirass/database.scm (db-get-worker): Handle the case where no worker
were found.

* src/sql/upgrade-3.sql: Fix syntax.

* src/sql/upgrade-3.sql: New file.
* Makefile.am (dist_sql_DATA): Add it.
* src/schema.sql (Builds)[last_status]: New field.
* src/cuirass/database.scm (db-update-build-status!): Honor it.
(db-get-builds): Return it.
* tests/database.scm ("db-update-build-status!"): New test.

Evaluations are added sequentially to database but builds are not always
registered nor performed in order. This means that a build corresponding to an
older evaluation can be returned first if it was completed last. Order by
descending evaluation id to prevent it.

* src/cuirass/database.scm (db-get-builds): Add "evaluation" order.
* src/cuirass/http.scm (url-handler): Order latestbuilds by descending
evaluation number.

* src/cuirass/database.scm (db-add-worker): Rename ...
(db-add-or-update-worker): ... into this new procedure.
(db-get-worker, db-remove-unresponsive-workers): New procedures.
* src/cuirass/remote-server.scm (%workers): Remove it.
(pop-build): Adapt it.
(remove-unresponsive-workers!): Remove it.
(read-worker-exp): Adapt it.
(zmq-start-proxy): Ditto.
* tests/database.scm ("db-add-worker"): Rename ...
("db-add-or-update-worker"): ... into this new test.
("db-get-worker", "db-remove-unresponsive-workers"): New tests.

* src/cuirass/templates (workers-status): Truncate long lines.

* src/cuirass/templates.scm (workers-status): Sort workers consistently.

* src/cuirass/templates.scm (workers-status): Remove unused "build-row"
procedure.

* src/cuirass/templates.scm (workers-status): Handle the case where multiple
builds are started on a worker.

* src/cuirass/database.scm (db-get-build-percentage): Prevent division by zero.

* src/cuirass/templates.scm (build-details): Fix build duration display.

* src/cuirass/database.scm (db-update-build-status!): Set starttime when the
build status is "submitted".

* src/cuirass/templates.scm (workers-status): Sort machines.

* src/cuirass/templates.scm (workers-status): Improve display.
* src/cuirass/http.scm (url-handler): Adapt it.

* src/cuirass/database.scm (db-get-build-percentage): New procedure.
* tests/database.scm ("db-get-build-percentage"): New test.

* src/sql/upgrade-2.sql: New file.
* Makefile.am (dist_sql_DATA): Add it.
* src/schema.sql (Workers): Add "machine field".
* src/cuirass/database.scm (db-get-builds): Return "worker" field.
(db-add-worker): Honor new "machine" field.
(db-get-workers): Ditto.
* src/cuirass/remote-worker.scm (remote-worker): Adapt it.
* src/cuirass/remote.scm (<worker>)[machine]: New field.
(worker-machine): New procedure.
(worker->sexp, sexp->worker): Adapt accordingly.
(generate-worker-name): Ditto.
* tests/database.scm (%dummy-worker): Add "machine" field.

* src/cuirass/remote-server.scm (remote-server): Have stdout/stderr
line-buffered.
* src/cuirass/remote-worker.scm (remote-worker): Ditto.
* src/cuirass/remote.scm (publish-server): Discard publish logs.

Signed-off-by: Mathieu Othacehe <othacehe@gnu.org>

* src/cuirass/database.scm (db-get-builds-by-search): Fix pagination.

* tests/database.scm (db-init): Force database creation.
* tests/http.scm (db-init): Ditto.
* tests/metrics (db-init): Ditto.

Fix a regression introduced by cbc46267.

* src/cuirass/database.scm (db-get-builds): Collect BuildProducts id field.

* src/cuirass/remote.scm: New file.
* src/cuirass/remote-server.scm: New file.
* src/cuirass/remote-worker.scm: New file.
* bin/remote-server.in: New file.
* bin/remote-worker.in: New file.
* Makefile.am (bin_SCRIPTS): Add new binaries,
(dist_pkgmodule_DATA): add new files,
(EXTRA_DIST): add new binaries,
(bin/remote-server, bin/remote-worker): new targets.
* .gitignore: Add new binaries.
* bin/cuirass.in (%options): Add "--build-remote" option,
(show-help): document it,
(main): honor it.
* src/cuirass/base.scm (with-build-offload-thread): New macro,
(%build-remote?, %build-offload-channel): new parameters,
(make-build-offload-thread): new procedure,
(build-derivations/offload): new procedure,
(restart-builds): use it to offload builds when %build-remote? is set,
(build-packages): ditto.

src/cuirass/base.scm (build-packages): Remove debug message that can be
inaccurate if queue-size is superior to 1.

* src/sql/upgrade-17.sql: New file.
* Makefile.am (dist_sql_DATA): Add it.
* src/schema.sql (Builds): Add "machine" field.
* src/cuirass/database.scm (db-update-build-machine!): New procedure.
* src/cuirass/base.scm (handle-build-event): Use it to record the machine
performing the build.

* bin/cuirass.in (%options): Add "queue-size" argument.
(show-help): Document it.
(main): Honor it.
* src/cuirass/database.scm (%db-writer-queue-size): New parameter,
(with-queue-writer-worker): use it.

* src/cuirass/database.scm (db-get-evaluation): New procedure.
* src/cuirass/http.scm (url-handler): Add "/api/evaluation" route.

* src/cuirass/http.scm (build->hydra-build): Add "evaluation" field.

Rather than calling sqlite-reset before reading the rows, call sqlite-reset
after. I think this is important to stop SQLite getting stuck because the
statement is still live even though it's not being read from after these
procedures return.

Assuming I've got the ordering right, I think using sqlite-fold-right directly
is simpler code as well.

* src/cuirass/database.scm (db-get-builds-by-search, format-build-products,
db-get-events): Rewrite fetching result rows.

sqlite-prepare will reset cached statements before returning them, so these
sqlite-reset calls are redundant.

* src/cuirass/database.scm (db-get-builds-query-min, db-get-builds-query-max):
Remove sqlite-reset calls.

All these statements are cached, but that's pointless if they're finalized
after use, because they'll need recreating on next use.

* src/cuirass/database.scm (%sqlite-exec, db-get-builds-query-min,
db-get-builds-query-max): Replace sqlite-finalize with sqlite-reset.

* src/cuirass/database.scm (db-get-builds): Fix pagination.

* src/cuirass/database.scm (db-get-builds-min, db-get-builds-max): Rewrite SQL
queries without nesting.

When the periodic argument is passed to a job, it will be registered only if
the time difference between the current time and the registration time of the
last build is greater than the specified period. Make sure that the last job
that is searched is part of the same specification.

* src/sql/upgrade-16.sql: New file.
* Makefile.am (dist_sql_DATA): Add it.
* src/schema.sql (Builds_job_name_timestamp): New index.
* src/cuirass/database.scm (db-get-time-since-previous-build): Add
"specification" argument,
(db-register-builds): pass it.

This is a work-around: https://issues.guix.gnu.org/44742.

* src/cuirass/base.scm (fetch-inputs): Set "current-error-port" parameter to a
void port to prevent the fetch progression bar to appear.

* src/cuirass/database.scm (db-get-time-since-previous-build): New procedure,
(db-register-builds): if the period argument is set, only register builds
which last registration is older than the specified period.

This reverts commit 38790945 that causes:

2020-11-02T11:05:08 fatal: uncaught exception 'wrong-type-arg' in 'build' fiber!
2020-11-02T11:05:08 exception arguments: ("struct-vtable" "Wrong type argument in position 1 (expecting struct): ~S" (#f) (#f))
In ice-9/boot-9.scm:
  1731:15 12 (with-exception-handler #<procedure 7fb1a93f9930 at ic…> …)
  1736:10 11 (with-exception-handler _ _ #:unwind? _ # _)
    718:2 10 (call-with-prompt ("break") #<procedure 7fb1ab76f440 a…> …)
    718:2  9 (call-with-prompt ("continue") #<procedure 7fb1ab77084…> …)
In ice-9/eval.scm:
    619:8  8 (_ #(#(#<directory (guile-user) 7fb1ac680f00> #<var…> …)))
In srfi/srfi-1.scm:
    634:9  7 (for-each #<procedure 7fb1a9525900 at cuirass/base.scm…> …)
In ice-9/boot-9.scm:
  1731:15  6 (with-exception-handler #<procedure 7fb1a95a94e0 at ic…> …)
  1669:16  5 (raise-exception _ #:continuable? _)
  1764:13  4 (_ #<&compound-exception components: (#<&assertion-fail…>)
In cuirass/utils.scm:
    319:8  3 (_ _ . _)
In ice-9/boot-9.scm:
  1731:15  2 (with-exception-handler #<procedure 7fb1ab2e3720 at ic…> …)
In cuirass/utils.scm:
   320:22  1 (_)
In unknown file:
           0 (make-stack #t)
ERROR: In procedure make-stack:
In procedure struct-vtable: Wrong type argument in position 1 (expecting struct): #f

Set the store socket as non-blocking so that fibers communicating with the
store don't get blocked as described here: https://issues.guix.gnu.org/43565.

* src/cuirass/base.scm (with-store): Set the store socket as non-blocking.
(build-derivations&): Unset current-read-waiter and current-write-waiter.