okami.mom/web-extract-go
git clone https://git.nkp.pet/okami.mom/web-extract-go
5a90548
main
admin dashboard
open http://127.0.0.1:8788/admin/ while the http service is running. the dashboard is compiled into the go binary. no node process, external font, cdn, or second database is used at runtime.
enable and sign in
set a separate, unique password of at least 12 characters in the root .env:
ADMIN_PASSWORD=your-long-private-admin-password ADMIN_COOKIE_SECURE=false
keep .env mode 0600. restart the http process after changes. an empty password disables all /admin routes. this password never grants access to the public api, and the public api token never grants access to the dashboard.
./bin/web-extract
the codex mcp launcher uses --stdio and does not start an http listener. run the http command separately for the dashboard. both can read/write the same sqlite database; the existing mcp process does not need to be restarted to see its new log rows here.
sessions are server-side, http-only, same-site strict cookies, expire after 12 hours, and are revoked on logout or server restart. there are at most 128 sessions and ten failed password attempts per minute per process. state-changing routes reject cross-origin browser requests and require json. logs are private and responses are not cached. provider credentials are never returned to the browser.
the default loopback binding is intentional. if exposing this remotely, use https and keep the dashboard behind your private network or access controls. behind an https-terminating proxy, preserve the public host header and set ADMIN_COOKIE_SECURE=true; forwarded headers are not trusted to decide cookie security. direct tls also sets secure cookies. plain remote http is not a supported secure deployment.
activity
- rolling 24-hour request totals and errors, excluding synthetic
extractfollow-through rows. - jev + summarizer call totals, failed attempts, and mean model-call latency, not end-to-end latency.
- summarizer token totals with reporting coverage; jev token counts are not included.
- configured providers/limits and database/wal disk usage. “configured” is not an upstream health check.
- newest-first request pages, endpoint/error filters, and manual or opt-in 10-second refresh.
- original model inputs, exact raw model replies, jev decisions, selected passages, offsets, budgets, and judgment scores.
the dashboard reads existing logs without rewriting rows or changing training exports. additive indexes support request correlation. model failures can be retries inside a successful request; ok does not imply every url succeeded. historical rows do not store complete http responses or end-to-end timing. per-url statuses and full api responses are shown for the current playground run, not reconstructed for historical requests. each trace is bounded to 100 request/evidence rows and 200 calls per model stage.
playground
choose search, contents, or answer, enter your own query/urls, and select output controls. expand “request json” to inspect the exact payload or edit advanced options. results show generated summaries separately from verbatim highlights, per-url statuses, raw json, and the correlated model/evidence trace.
these are real, potentially paid model/search calls. they use the same public api validation/pipeline and are logged with caller admin-dashboard. canceling interrupts a request but does not erase calls already completed. there is no automatic query rerun.
development and verification
frontend source is in web/src; shadcn components are vendored from the official new-york registry under web/src/components/ui. dependencies are pinned with a pnpm lockfile. the small web/dist build is retained so a normal go build needs no node install.
./scripts/build # rebuild frontend and go binary pnpm --dir web dev# optional vite dev server; api proxies to :8788 pnpm --dir web testpnpm --dir web test:browsergo test ./...go test-race ./...go vet ./...
browser tests launch an isolated go fixture with temporary sqlite data and mock providers, then drive the built dashboard using installed chrome. they cover login, pagination/filtering, search, contents partial failure, answer, trace inspection, validation errors, logout, and mobile overflow. no browser is downloaded. set CHROME_PATH for another chromium executable. rebuild the frontend before browser tests. optionally set ADMIN_SCREENSHOTS to capture desktop/mobile checks.
verified locally · 2026-09-19
go tests, race checks, vet, frontend typecheck/build, request-shape tests, and the browser fixture passed. desktop/mobile review passed after adding an explicit keyboard-accessible horizontal-scroll region for the activity table.
three additional real-provider queries were run through the browser dashboard, not just direct api calls:
| endpoint | query | result | browser round trip |
|---|---|---|---|
| search | sqlite wal concurrent writers official documentation; sqlite.org filter | 2 results, 2 summaries, 53 jev calls | 8.14 s |
| contents | how do %w, errors.Is, and errors.As work in go?; go.dev/blog/go1.13-errors | 1 result, 25 jev calls, 1 failed format attempt followed by a successful summary | 4.43 s |
| answer | can sqlite wal allow multiple writers to write at the same time? | answer, 3 citations, 137 jev calls | 13.08 s |
all returned http 200 and could be opened as correlated request traces. the successful request containing a failed summarizer attempt remained visible as such in the model-call log. these are smoke checks, not a retrieval-quality benchmark. the ui fixture's partial-url failure check used a mocked 404; the live calls above used successful public pages.
1# admin dashboard 2 3open `http://127.0.0.1:8788/admin/` while the http service is running. the dashboard is compiled into the go binary. no node process, external font, cdn, or second database is used at runtime. 4 5## enable and sign in 6 7set a separate, unique password of at least 12 characters in the root `.env`: 8 9``` dotenv 10ADMIN_PASSWORD=your-long-private-admin-password 11ADMIN_COOKIE_SECURE=false 12``` 13 14keep `.env` mode `0600`. restart the http process after changes. an empty password disables all `/admin` routes. this password never grants access to the public api, and the public api token never grants access to the dashboard. 15 16``` sh 17./bin/web-extract 18``` 19 20the codex mcp launcher uses `--stdio` and does **not** start an http listener. run the http command separately for the dashboard. both can read/write the same sqlite database; the existing mcp process does not need to be restarted to see its new log rows here. 21 22sessions are server-side, http-only, same-site strict cookies, expire after 12 hours, and are revoked on logout or server restart. there are at most 128 sessions and ten failed password attempts per minute per process. state-changing routes reject cross-origin browser requests and require json. logs are private and responses are not cached. provider credentials are never returned to the browser. 23 24the default loopback binding is intentional. if exposing this remotely, use https and keep the dashboard behind your private network or access controls. behind an https-terminating proxy, preserve the public host header and set `ADMIN_COOKIE_SECURE=true`; forwarded headers are not trusted to decide cookie security. direct tls also sets secure cookies. plain remote http is not a supported secure deployment. 25 26## activity 27 28- rolling 24-hour request totals and errors, excluding synthetic `extract` follow-through rows. 29- jev + summarizer call totals, failed attempts, and mean **model-call** latency, not end-to-end latency. 30- summarizer token totals with reporting coverage; jev token counts are not included. 31- configured providers/limits and database/wal disk usage. “configured” is not an upstream health check. 32- newest-first request pages, endpoint/error filters, and manual or opt-in 10-second refresh. 33- original model inputs, exact raw model replies, jev decisions, selected passages, offsets, budgets, and judgment scores. 34 35the dashboard reads existing logs without rewriting rows or changing training exports. additive indexes support request correlation. model failures can be retries inside a successful request; `ok` does not imply every url succeeded. historical rows do not store complete http responses or end-to-end timing. per-url statuses and full api responses are shown for the current playground run, not reconstructed for historical requests. each trace is bounded to 100 request/evidence rows and 200 calls per model stage. 36 37## playground 38 39choose `search`, `contents`, or `answer`, enter your own query/urls, and select output controls. expand “request json” to inspect the exact payload or edit advanced options. results show generated summaries separately from verbatim highlights, per-url statuses, raw json, and the correlated model/evidence trace. 40 41these are real, potentially paid model/search calls. they use the same public api validation/pipeline and are logged with caller `admin-dashboard`. canceling interrupts a request but does not erase calls already completed. there is no automatic query rerun. 42 43## development and verification 44 45frontend source is in `web/src`; shadcn components are vendored from the [official new-york registry](https://ui.shadcn.com/docs/installation/vite) under `web/src/components/ui`. dependencies are pinned with a pnpm lockfile. the small `web/dist` build is retained so a normal go build needs no node install. 46 47``` sh 48./scripts/build # rebuild frontend and go binary 49pnpm --dir web dev # optional vite dev server; api proxies to :8788 50pnpm --dir web test 51pnpm --dir web test:browser 52go test ./... 53go test -race ./... 54go vet ./... 55``` 56 57browser tests launch an isolated go fixture with temporary sqlite data and mock providers, then drive the built dashboard using installed chrome. they cover login, pagination/filtering, search, contents partial failure, answer, trace inspection, validation errors, logout, and mobile overflow. no browser is downloaded. set `CHROME_PATH` for another chromium executable. rebuild the frontend before browser tests. optionally set `ADMIN_SCREENSHOTS` to capture desktop/mobile checks. 58 59### verified locally · 2026-09-19 60 61go tests, race checks, vet, frontend typecheck/build, request-shape tests, and the browser fixture passed. desktop/mobile review passed after adding an explicit keyboard-accessible horizontal-scroll region for the activity table. 62 63three additional real-provider queries were run through the browser dashboard, not just direct api calls: 64 65| endpoint | query | result | browser round trip | 66| --- | --- | --- | --- | 67| search | sqlite wal concurrent writers official documentation; sqlite.org filter | 2 results, 2 summaries, 53 jev calls | 8.14 s | 68| contents | how do %w, errors.Is, and errors.As work in go?; go.dev/blog/go1.13-errors | 1 result, 25 jev calls, 1 failed format attempt followed by a successful summary | 4.43 s | 69| answer | can sqlite wal allow multiple writers to write at the same time? | answer, 3 citations, 137 jev calls | 13.08 s | 70 71all returned http 200 and could be opened as correlated request traces. the successful request containing a failed summarizer attempt remained visible as such in the model-call log. these are smoke checks, not a retrieval-quality benchmark. the ui fixture's partial-url failure check used a mocked 404; the live calls above used successful public pages.