9c8f91ef27
watchdog: make metrics rate limit configurable; document env vars
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I01033406c32bd4e31a76e676be97af046a6a6964
2026-03-10 13:19:38 +03:00
fd3a832f7b
internal/normalize: simplify IP parser; cleanup
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I7a860779a4fe49b8034d66f2abd910fc6a6a6964
2026-03-10 13:19:36 +03:00
98611ca452
api/handler: fix X-Real-IP header validation
...
When trusted proxy headers are enabled, the code accepted `X-Real-IP`
without validating it. The attacker could simply set `X-Real-IP` to an
arbitrary and that IP would be recorded as is. We validate the IP format
and ensure it's not from a trusted proxy, and add test cases.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ic1e761ea623a69371a28ad15d465d6c66a6a6964
2026-03-10 10:40:03 +03:00
ffa2af62be
api/handler: check if each IP in X-Forwarded-For is *not* in trusted networks before accepting
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Id54c1584650fcee64de70d1f99e542c16a6a6964
2026-03-10 08:56:03 +03:00
d1181d38f0
watchdog: add log sanitization and request tracking
...
Stuff to prevent possible log injection attacks via weird characters,
now sanitized with `strconv` stuff.
- X-Request-ID is now traced in ingestion handler
- ValidateWithMap renamed to Validate (xd)
- Some new tests :D
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I286ec399a5c4a407f0cc117472c079446a6a6964
2026-03-10 08:43:53 +03:00
4189d14d65
api/event: remove legacy validate function; use domain map
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I9a68733cf16b09ef6381161452bda1e56a6a6964
2026-03-10 08:43:52 +03:00
02c4f11619
api/handler: O(n) linear scan -> O(1) map lookup
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ic2080f59be1eea905e8ca95e90e34d4d6a6a6964
2026-03-10 08:43:51 +03:00
d2f28ded61
aggreggate/uniques: use string builder for hashing
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I7371a878c05f704feef26c6e86a04b956a6a6964
2026-03-10 08:43:50 +03:00
0f38a062e9
various: reduce file I/O & pre-parse CIDRs
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I288c299d35fdc833c802e22682f14b8e6a6a6964
2026-03-10 08:43:33 +03:00
b7b1d62cfc
various: resolve linter warnings
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ia83658dd31f9e4042d0a3a754fc6e4106a6a6964
2026-03-07 10:04:30 +03:00
d975c7b2d1
internal/aggregate: implement hourly salt rotation for unique visitors
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I5861c5bb55153349d0710cc07c1595a96a6a6964
2026-03-07 08:48:32 +03:00
27b3641717
various: add internal health and runtime metrics
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Iae1dcf8495a00159d588c6e2344312f36a6a6964
2026-03-02 22:38:28 +03:00
d7cdf2cc49
chore: fix typo in dailySalt comment
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I8f0d0bf4bc597f0aecfd98c292f38cdb6a6a6964
2026-03-02 22:38:27 +03:00
6977a501b1
internal: better device classification via UA parsing
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I6c78f1eebe71ef4cf037ebbda2caaeb36a6a6964
2026-03-02 22:38:26 +03:00
7b06c4f2ca
various: extract magic numbers into named constants
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I854b2f9b5f39e4629c32e5681e6322826a6a6964
2026-03-02 22:38:24 +03:00
f46697bd21
internal/ratelimit: prevent time drift in TokenBucket refills
...
The TokenBucket ratelimiter accumulated time drift over multiple refills
because I'm an idiot. We were using 'now' as base for lastFill calc. but
this could case rate limiting to become inaccurate over time. Now we
advance lastFill by *exact* periods from previous value.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ia3990b441ab6072f51dfdfa4a2511b5f6a6a6964
2026-03-02 22:38:23 +03:00
4e0b8f0d0a
interal/api: replace liner array scan with hashmap lookup in domain validation
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Iac969e7dc6e4ca3f93410fccac1995636a6a6964
2026-03-02 22:38:22 +03:00
987ddd92cc
internal/aggregate: make shutdown context-aware proper goroutine sync
...
Adds `WaitGroup` to track background goroutine and make Shutdown respect
context deadlines
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ia7f074725717f037412dacb93e34105b6a6a6964
2026-03-02 22:38:21 +03:00
2ef2dabf93
internal/aggergate: make HLL state path configurable
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I8ff8ef25ad945aae918bea97ee39d7ea6a6a6964
2026-03-02 22:38:19 +03:00
f988174145
watchdog: migrate to Cobra and Viper for config management; search /etc for configs
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I65dbf466cb030dccc7025585d6282bd26a6a6964
2026-03-02 22:38:18 +03:00
bf8390a916
chore: format with golines
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I11a2f3273abf08c8cf02e0c335e26d826a6a6964
2026-03-02 22:38:14 +03:00
18fe1a8234
internal/api: better multi-sites support; validate events against allowed domains
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Iff1ced4966b4d42cfd6dfefb0cfd97696a6a6964
2026-03-02 22:38:11 +03:00
da1fab4257
internal: fix the tests broken by hardening
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: If95a5258a393542564f68b3a1ebc7ff66a6a6964
2026-03-02 22:38:07 +03:00
7e1ef845e8
internal/api: resolve IPv6 handling; prevent XFF spoofing & add rate limiting
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ibe415a133bbc8bd533a21ed1ccd44cf36a6a6964
2026-03-02 22:38:05 +03:00
8187608b38
internal/api: centralize constants; improve validation & santize errors
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I3c3e4acb12a5a965bfaba950bf9aa5776a6a6964
2026-03-02 22:38:04 +03:00
8392992b41
internal/aggregate: optimize HyperLogLog to prevent O(16384) operations
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ibc7e6d7a86e8679e299c46debee9683f6a6a6964
2026-03-02 22:38:03 +03:00
bb56df6423
internal/aggregate: optimize path registry for read-heavy workloads
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ibc477830b471fe09838b7477fe73ffa56a6a6964
2026-03-02 22:38:02 +03:00
ffb4ab2295
internal/normalize: harden against possible attacks; optimize registry
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Iaf89cda3c480d6a8371e5f146ee95fcf6a6a6964
2026-03-02 22:38:01 +03:00
b2256183e1
config: add security and performance sections to sample config; validate
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ieda42bcbd09014c45fb14bee579f829c6a6a6964
2026-03-02 22:38:01 +03:00
c3b77696aa
internal: centralize size/length constants; better DoS protection
...
...also adds a bounded custom event registry for cardinality control but
I ran out of space in the commit message. Praise be to the long
descriptions...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ic205f69804c7fb24c39fa84abdd9546b6a6a6964
2026-03-02 22:38:00 +03:00
993e47e603
internal/aggregate: add HyperLogLog unique visitor tracking
...
Extracts IP from X-Forwarded-For/X-Real-IP/RemoteAddr. Only active
when `config.Site.SaltRotation` is set.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ieef93b81e9894fc2e9e129451bf2dfdf6a6a6964
2026-03-02 22:37:58 +03:00
e0ec475a81
internal/api: ingestion handler; wire normalization pipeline
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I1890a039b874fcc76ac4a545c2901d4e6a6a6964
2026-03-02 22:37:55 +03:00
c5109ace92
internal/api: add event model with validation
...
Supports both pageview and custom event types
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Iaf48291cd952865ea9ec21361ae33c746a6a6964
2026-03-02 22:37:54 +03:00
bc4d3fed53
internal: initial metrics aggregator
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I9cdd6e2b33bb65182568db9db4460bc46a6a6964
2026-03-02 22:37:53 +03:00
ce848ed6f0
internal: add bounded path registry to prevent cardinality explosion
...
"cardinality explosion" would make for an epic rock band name...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I53cceb00ab9b17039b1fb1389977bf6b6a6a6964
2026-03-02 22:37:52 +03:00
be4534bac8
internal: add referrer classification
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ibcaa09bfd4767876ea1cdd5b61c53b476a6a6964
2026-03-02 22:37:51 +03:00
0691e5ee34
internal: implement path normalization w/ configurable rules
...
Strips query strings and URL fragmenets, prevents unbounded Prometheus
metrics by normalizing paths like:
- `/users/12345/profile -> /users/:id/profile`
- `/page?utm_source=twitter -> /page`
- `/a/../b -> /b`
etc.
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: I72f2fa2452f4666567143d052b5716476a6a6964
2026-03-02 22:37:50 +03:00
4c84393286
config: data structures; basic tests
...
Signed-off-by: NotAShelf <raf@notashelf.dev>
Change-Id: Ia7d6f19a46ec8a4987ea429ec6502f676a6a6964
2026-03-02 22:37:47 +03:00
