More or less mimics Hydra's `GithubStatus.pm` adaptive throttling:
- Log rate limit status when remaining <= 2000
- Sleep when remaining <= 1000
Adaptive delay spreads requests over reset window. Rate limits are
extracted from *every response* for better accuracy. Accuracy is
*critical* because the primary failure is to get rate limited more
often due to miscalculation.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I3b1fa444c937715c604002d71510bcf76a6a6964
Extract `X-RateLimit-*` headers from responses and calculate an adaptive
delay. Minimum delay is 1 seconds to prevent division by 0.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ib35b0d0e720098e2c68ced88a8821c7b6a6a6964
Hydra maxUnsupportedTime compatibility. DUration is optional, defaults
to `None` and is configurable through `fc.toml` under
`queue_runner.unsupported_timeout` key. E.g., "1h".
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I299722e2a3865c873e212a7615b97b806a6a6964
Now creates a path-to-name lookup `HashMap` from the outputs JSON and
matches each output path to its correct name. Both the `build_outputs`
table insert and GC root registration loops now use the same lookup
to ensure we're consistent.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Id9f4ce0be6131a7c1a8ce6775ab249db6a6a6964
Replaces JSON blob storage with BuildOutput records and parse derivation
outputs after successful build, then INSERT into `build_outputs` per
output.
Warnings are logged on storage failures, but it's not fatal.
- Parse derivation outputs after successful build
- INSERT into build_outputs table per output
- Log warnings on storage failures (non-fatal)
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I30120a5ee4aea1bdb170987b22ddc2df6a6a6964
Btree supports NULL values and provides better flexibility
than hash index for nullable columns.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ida28ef6f88683c360e6f405efca435af6a6a6964
Send "running" commit status when builds are claimed. Also fixes
missing completion notifications for aggregate builds.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I3b4dab8e30d6e278b7d38e2222a800a56a6a6964
Helpers for sending forge status updates when builds are created and
claimed.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Iddca8b33fc31a9e23968752ef112e0d26a6a6964
Adds a `notification_tasks` table and a background worker to (hopefully
reliably) deliver webhooks, git status updates, and e-mail notifications
with automatic retry on transient failures.
This was one of the critical gaps, finally done.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I794967c66958658c4d8aed40793d67f96a6a6964
Adds an `ActiveBuild` registry (DashMap of `<Uuid, CancellationToken>`)
to `WorkerPool` and get `dispatch()` to create a per-build token to race
`run_build` against it via Tokio's `select!`.
The `cancel_checker_loop` then polls the DB every N seconds (currently 2)
for builds cancelled while running, and triggers their tokens.
Existing `kill_on_drop(true) on `nix build` processes handles
subprocess cleanup when the future is dropped. Thank you past me for
your insight.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ic8af58e92972c7d5d104d9c717e9217d6a6a6964
