21446c6dcb
fc-queue-runner: implement persistent notification retry queue with exponential backoff
...
Adds a `notification_tasks` table and a background worker to (hopefully
reliably) deliver webhooks, git status updates, and e-mail notifications
with automatic retry on transient failures.
This was one of the critical gaps, finally done.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I794967c66958658c4d8aed40793d67f96a6a6964
2026-02-28 12:18:18 +03:00
d0ffa5d9e5
fc-server: implent proper rate limiting with token bucket algorithm; fix rate_limit_rps
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I68237ff6216337eba1afa8e8606d545b6a6a6964
2026-02-28 12:18:16 +03:00
e7425e0abf
fc-common: consolidate database migrations; simplify
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ia808d76241cec6e8760d87443bb0dc976a6a6964
2026-02-28 12:18:13 +03:00
23a4a8e348
fc-common: format
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I946272ee6563f5bca0844c5a25ba08f66a6a6964
2026-02-28 12:18:07 +03:00
5b472a2f57
fc-common: add GC pinning and machine health infrastructure
...
Migration 017 adds `builds.keep`, `jobsets.keep_nr`, and health tracking
columns to `remote_builders`. Repo layer implements `set_keep`,
`list_pinned_ids`, `record_failure` with exponential backoff,
`record_success`, and `find_for_system` filtering of disabled builders.
GC root cleanup now skips pinned builds.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ibba121de3dc42f71204e3a8f5776aa8b6a6a6964
2026-02-28 12:18:04 +03:00
4100ac54c2
fc-common: implement deficit-based fair-share scheduling in list_pending
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ic1345cfdf712aa6ee6f0eeae45b3e62b6a6a6964
2026-02-28 12:17:51 +03:00
d401177902
fc-common: add get_cancelled_among query for cancel-checker polling
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Iabc5fc932c8d7f1d19a80a27965524136a6a6964
2026-02-16 23:42:44 +03:00
65a6fd853d
fc-common: add failed paths cache infrastructure
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I35f9bfb044160151cf73c43ed9ada3476a6a6964
2026-02-16 23:42:41 +03:00
4c56b192f0
fc-common: fix BuildStatus sqlx rename to snake_case
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I00434e0156b3dc1dfd26699e4b103bd46a6a6964
2026-02-16 23:42:40 +03:00
235c9834b7
fc-evaulator: allow fail-fast behaviour
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I1da41766f1c499347279c41f2316f4376a6a6964
2026-02-16 23:42:39 +03:00
e274389d12
fc-common: add PostgreSQL LISTEN/NOTIFY infrastructure
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Iffdb2fa758825e8c5d5791bf4fb15c8e6a6a6964
2026-02-16 23:42:36 +03:00
541cd7832f
treewide: replace BuildStatus::Completed with BuildStatus::Succeeded
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I965dfaca211f9fde527a84a54ae972576a6a6964
2026-02-16 23:42:21 +03:00
f6fcf49aa7
tests: replace BuildStatus::Completed with BuildStatus::Succeeded
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ia12f816c203d6ce51485788d0414894f6a6a6964
2026-02-16 13:02:27 +03:00
85970e249c
fc-common: extend BuildStatusFilter with all status variants
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I58c8ebbe937035c2398af4eea0eaa3cf6a6a6964
2026-02-16 13:02:25 +03:00
5b7648220c
fc-common: update notifications to handle all BuildStatus variants
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I99db557ae204f3a4ffec6be70386ecc16a6a6964
2026-02-16 13:02:24 +03:00
f4772036ce
fc-common: add database migration for extended build status codes
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I0c13eda985d634e63189ba6907e488ae6a6a6964
2026-02-16 13:02:22 +03:00
2b763833d4
fc-common: add extended BuildStatus enum with new status code
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: If00cae8b2f7d4a7ad2d64bfe70a5c4186a6a6964
2026-02-16 13:02:21 +03:00
a5768d46eb
fc-common: allow configuring url schemes to allow for testing
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I99912d7c45f1a4664d4823ddd793b5af6a6a6964
2026-02-16 00:08:23 +03:00
b745550011
various: tiny cleanup
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I312766a6178be3898b51c2863f502bb06a6a6964
2026-02-15 23:37:53 +03:00
f7081317ee
various: reuse HTTP client; eliminate intermediate string allocations; add tests
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I18b89e1aae78a400a89c9d89423ce1da6a6a6964
2026-02-15 23:37:52 +03:00
38ed7faee2
various: replace silent error discards with logged warnings
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I465d760b5330980270b64b4a89abc09f6a6a6964
2026-02-15 23:37:51 +03:00
aa4ebf2f5b
various: harden input validation; add SSRF protection; fix default API key role
...
Default API key role was "admin", which was something that I forgot to fix during testing. We
change it to "read-only".
Additionally repository URLs now reject `file://` scheme (another testing artifact) localhost,
private IP ranges, and cloud metadata endpoints. Nix expressions reject path traversal (`..`)
and absolute paths. Validation is called at the evaluator endtrypoint before command construction.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I35729c6aa9ec4ff8d1ea19bd57ea93646a6a6964
2026-02-15 23:37:49 +03:00
a2b638d4db
nix: attempt to fix VM tests; general cleanup
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I65f6909ef02ab4599f5b0bbc0930367e6a6a6964
2026-02-15 23:37:49 +03:00
537fa823a7
fc-server: add timeseries API endpoints for visualisation
...
Adds:
- `build_stats_timeseries()` for build counts over time
- `duration_percentiles_timeseries()` for P50/P95/P99
- `system_distribution()` for per-system counts
and of course, REST endpoints for `/api/v1/metrics/timeseries/*`. This
is a good start for data visualisation. My professors would be proud.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I3c0b9d14592945a661af77b7edf338a86a6a6964
2026-02-14 18:08:20 +03:00
75ff45fc91
various: initial support for S3 cache upload
...
Not too stable yet, but might work.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: If134e7e45aa99ce8d18df7b78b1f881b6a6a6964
2026-02-14 18:08:19 +03:00
ec5fbb453d
fc-common: add AlertConfig and AlertManager for error tracking
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Iaf2f52f6e0cf33e3275528ac13cd92046a6a6964
2026-02-14 18:08:17 +03:00
f8f9703faa
fc-common: add build_metrics table and repository
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: If6842311f49bfcba9e1b11fa8bc9748c6a6a6964
2026-02-14 18:08:16 +03:00
0dc09dbd19
various: fix auto-fisable Clippy lints
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I79a708981b5aee43eadbd48d69fb78be6a6a6964
2026-02-08 22:23:29 +03:00
3a03cf7b3e
treewide: format with nightly rustfmt; auto-fix Clippy lints
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: If4fd0511087dbaa65afc56a34d7c2f166a6a6964
2026-02-08 22:23:28 +03:00
c6c64d568f
fc-common: add advanced search functionality with filters and sorting
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ib48f9bd42eea289b9eb8b13e3bd60ed86a6a6964
2026-02-08 22:23:25 +03:00
d4d9297d96
fc-common: add declarative sync for remote builders and channels
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I3dae89f04777f6d941824606aebe34446a6a6964
2026-02-08 22:23:24 +03:00
abd16319f2
fc-common: add sync_for_project for declarative members
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ic283bf87771e82a27c4c579314a21c1d6a6a6964
2026-02-08 22:23:23 +03:00
7e6fc22ba2
fc-common: add declarative sync for webhooks and notifications
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I0b7c46feba776837158507bfe883cbfa6a6a6964
2026-02-08 22:23:22 +03:00
dead111dfb
fc-common: add upsert and sync_for_jobset for declarative jobset inputs
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I40f2724aeb9615e8c37a397187064fa66a6a6964
2026-02-08 22:23:21 +03:00
73919f2f9e
treewide: format with nightly rustfmt; auto-fix Clippy lints
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I15d9215ab506b37954468d99746098326a6a6964
2026-02-08 22:23:20 +03:00
10a118bd50
various: update tests for jobset state field changes
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ifdd7f216fcc27fface0cb16d56c5e37b6a6a6964
2026-02-08 22:23:19 +03:00
865b2f5f66
fc-common: better support declarative users with password file
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I1eac6decd68a4e59a52fecaecdd476b26a6a6964
2026-02-08 22:23:17 +03:00
1df28f6049
fc-common: add jobset state enum; add db migration"
...
I'm going to delete all migrations one of those days...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I8e2e32118c2d85438a8b343614038eda6a6a6964
2026-02-08 22:23:14 +03:00
b9780dea1c
chore: bump MSRV to 1.95 for nightly Rust
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I3e30e54bbdf9f382c31b29bbcaeb44be6a6a6964
2026-02-08 22:23:12 +03:00
794e4a8e61
common,evaluator: add PR fields to evaluation; add jobset ordering test
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I305c988aceb8c72c65b740cb323c0ef06a6a6964
2026-02-07 22:09:28 +03:00
6aa3a7357f
common: improve search and repo tests
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ic34fb20b5fc0e400f3ed2da441796f6b6a6a6964
2026-02-07 22:09:25 +03:00
b6012b932f
chore: update dependencies and configuration
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I5bab99cb7e7bb2125e7410b54911021a6a6a6964
2026-02-07 22:09:24 +03:00
8c1968c863
common: improve notifications system
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I084bb95a4eb79d5a15f7c062c112124c6a6a6964
2026-02-07 22:09:23 +03:00
caadb52f64
common: enhance user repository with validation
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ic96bef36e3b4d1ea6b2db9752b26dd3a6a6a6964
2026-02-07 22:09:22 +03:00
2eae49f313
fc: initial pull request evaluation support
...
The migration adds PR support, models expose PR fields, webhooks handle
PR events, and tests validate it. To be honest the migrations are a bit
redundant at the moment, but I'd like to handle my old deployments so
it's nice(r) to have them. I *am* testing those on baremetal.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I02fb4540b62d3e8159ac18b9fa63be916a6a6964
2026-02-07 22:09:20 +03:00
1c18306822
various: eliminate redundant disk check; improve error handling
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I1f37cc60380790bc1bf11f143194ad116a6a6964
2026-02-05 23:05:02 +03:00
4b375bc546
various: remove unused imports and parameters; fix clippy lints
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ia7a4438e1aa73de2ea1bc6cdf26998f06a6a6964
2026-02-05 22:45:23 +03:00
dec4753567
fc-server: add (more) advanced search API
...
Basically, implements a multi-entity search functionality with filters
for projects, jobsets, evaluations and builds. Also fixes COUNT query to
apply same filters as main query, and fixes an offset mismatch in
response. Some integration tests have also been added, but chances are
we'll want to write VM tests for this.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Icdda77966a7218f54fd34b78bdc9b55c6a6a6964
2026-02-05 22:45:22 +03:00
550603c4bb
fc-common: add disk space monitoring and error handling
...
Adds a `DiskSpace` error variant and an `is_disk_full()` helper
alongside a `check_disk_space()` util that uses libc's `statsfs` on
unix. FC now detects disk space errors and logs recovery instructions
if applicable.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ia6d2c472219dc9d6eed6901dc733d7bb6a6a6964
2026-02-05 22:45:20 +03:00
f974e3dd16
fc-common: test user management fully
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Id144b7ba20283599437b0af86f8d716d6a6a6964
2026-02-05 22:45:15 +03:00
