fix(ops OPS53): deploy — compose run consome o stdin do bash -s (migrate com </dev/null); build via --network host + proxy socat (BuildKit recusa rede bridge custom)
All checks were successful
CI / static (push) Successful in 2m5s
CI / int (push) Successful in 2m18s
CI / build (push) Successful in 3m17s
CI / e2e (1) (push) Successful in 5m28s
CI / e2e (2) (push) Successful in 4m35s
CI / checks (push) Successful in 1s
CI / deploy homeserver (teqo-1313) (push) Successful in 4m39s

This commit is contained in:
Francisco Solla 2026-08-17 22:50:22 -03:00 committed by fsolla
parent 7a8f4d9830
commit a34cd9b68a
3 changed files with 47 additions and 15 deletions

View file

@ -12,10 +12,12 @@ com a suite verde publica o site em `jorgesolla1313.com.br` — sem passo manual
`ssh homeserver "bash -s -- <sha>" < scripts/deploy-homeserver.sh`. `ssh homeserver "bash -s -- <sha>" < scripts/deploy-homeserver.sh`.
4. O script, no **homeserver**: guarda de HEAD (`main` remoto == SHA do job, 4. O script, no **homeserver**: guarda de HEAD (`main` remoto == SHA do job,
senão skip) → `flock` (serializa) → build da imagem standalone + migrator senão skip) → `flock` (serializa) → build da imagem standalone + migrator
(BuildKit, secrets do `~/stack/teqo-1313.env`, rede `stack_default`) → (BuildKit, secrets do `~/stack/teqo-1313.env`, `--network host` com proxy
socat `teqo-1313-build-proxy` na `stack_default` para o build alcançar o
`postgres` — o proxy é criado idempotentemente pelo script) →
push em `localhost:5000` → swap dos tags de imagem no push em `localhost:5000` → swap dos tags de imagem no
`~/stack/docker-compose.yml` (backup antes) → **migrations** `~/stack/docker-compose.yml` (backup antes) → **migrations**
(`docker compose --profile maintenance run --rm teqo-1313-migrate`) → (`docker compose --profile maintenance run --rm teqo-1313-migrate </dev/null`) →
`docker compose up -d teqo-1313` → healthcheck → smoke (`/`, `docker compose up -d teqo-1313` → healthcheck → smoke (`/`,
`/campanha/login`, `/admin`, barreira 307, WebAuthn login-options, `/campanha/login`, `/admin`, barreira 307, WebAuthn login-options,
`api/revalidate` com o secret real). `api/revalidate` com o secret real).
@ -59,14 +61,16 @@ roda após build+push ok; rollout só após migrate ok).
## Falhas conhecidas ## Falhas conhecidas
| Sintoma | Causa | Tratamento | | Sintoma | Causa | Tratamento |
| ----------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Job verde sem deploy ("stale run") | Push mais novo venceu; o deploy antigo saiu da fila | Nada a fazer — o run do SHA novo deploya | | Job verde sem deploy ("stale run") | Push mais novo venceu; o deploy antigo saiu da fila | Nada a fazer — o run do SHA novo deploya |
| Todo job docker quebra com `exec: "node": executable file not found in $PATH` | `runner.envs.PATH` do `~/.forgejo-runner/config.yaml` (workstation) com paths só de host (nvm/bun) — o container perde o node do toolcache (`/opt/acttoolcache`) | Corrigir o PATH no config (toolcache primeiro — ver comentário no próprio arquivo) + `systemctl --user restart forgejo-runner` (incidente 2026-08-17) | | Todo job docker quebra com `exec: "node": executable file not found in $PATH` | `runner.envs.PATH` do `~/.forgejo-runner/config.yaml` (workstation) com paths só de host (nvm/bun) — o container perde o node do toolcache (`/opt/acttoolcache`); edição do config exige o toolcache primeiro (ver comentário no próprio config) | Corrigir o PATH no config (toolcache primeiro — ver comentário no próprio arquivo) + `systemctl --user restart forgejo-runner` (incidente 2026-08-17) |
| Build OOM no homeserver | Laptop 8c/16GB com o stack ativo (~12GB livres medidos) | Re-dispatch; se recorrente, item futuro: build na workstation com túnel | | Deploy para logo após o migrate ("Done." e nada mais, EXIT=0) | `docker compose run` anexa stdin por padrão — o container consome o resto do script que vai no pipe do `bash -s`; o bash chega a EOF e termina sem rodar o rollout | O script já usa `< /dev/null` no `run --rm` do migrate (não remover); sintoma visto 2026-08-17 no primeiro deploy |
| `checkout@v5` falha no job `host` | Label host do act_runner | Fallback: baixar o script direto do Forgejo (`curl -s https://git.solla.dev/fsolla/teqo/raw/branch/main/scripts/deploy-homeserver.sh`) | | Build falha: `network mode "stack_default" not supported by buildkit` | BuildKit (drivers docker e docker-container) recusa rede bridge custom no `--network` | O script builda com `--network host` + proxy socat `teqo-1313-build-proxy` (na `stack_default`, publicado em 127.0.0.1:5433, criado idempotentemente com `restart: unless-stopped`) + `DATABASE_URL` reescrita para o loopback |
| Migrate falha | Drift/erro de schema | Job vermelho; site segue no container antigo; corrigir e re-mergear | | Build OOM no homeserver | Laptop 8c/16GB com o stack ativo (~12GB livres medidos) | Re-dispatch; se recorrente, item futuro: build na workstation com túnel |
| Smoke falha pós-up | Regressão de runtime | Rollback automático (restore + `up -d`) + job vermelho; investigar | | `checkout@v5` falha no job `host` | Label host do act_runner | Fallback: baixar o script direto do Forgejo (`curl -s https://git.solla.dev/fsolla/teqo/raw/branch/main/scripts/deploy-homeserver.sh`) |
| Migrate falha | Drift/erro de schema | Job vermelho; site segue no container antigo; corrigir e re-mergear |
| Smoke falha pós-up | Regressão de runtime | Rollback automático (restore + `up -d`) + job vermelho; investigar |
## Segurança (decisão deliberada) ## Segurança (decisão deliberada)

View file

@ -84,11 +84,30 @@ echo "$REGISTRY_PASSWORD" | docker login localhost:5000 --username "$REGISTRY_US
# --- build -------------------------------------------------------------- # --- build --------------------------------------------------------------
# BuildKit rejects a custom bridge network on `--network`, so the build runs
# with `--network host`; the prod DB (only reachable inside stack_default) is
# proxied to the host loopback by a one-off socat container on the compose
# network, and DATABASE_URL is rewritten to that endpoint. The proxy is
# idempotent and survives reboots (restart: unless-stopped).
ensure_db_proxy() {
if ! docker inspect teqo-1313-build-proxy >/dev/null 2>&1; then
docker run -d --name teqo-1313-build-proxy \
--network stack_default \
--restart unless-stopped \
-p 127.0.0.1:5433:5433 \
alpine/socat TCP-LISTEN:5433,fork TCP:postgres:5432 >/dev/null
fi
}
build_image() { build_image() {
local target="$1" image="$2" local target="$1" image="$2"
say "building $target ($image)" say "building $target ($image)"
DOCKER_BUILDKIT=1 docker build \ ensure_db_proxy
--network stack_default \ # The build's DATABASE_URL points at the loopback proxy; the outer env
# keeps the original value and nothing is echoed.
local build_db_url="${DATABASE_URL/@postgres:5432/@127.0.0.1:5433}"
DATABASE_URL="$build_db_url" DOCKER_BUILDKIT=1 docker build \
--network host \
--build-arg "NEXT_PUBLIC_SITE_URL=$NEXT_PUBLIC_SITE_URL" \ --build-arg "NEXT_PUBLIC_SITE_URL=$NEXT_PUBLIC_SITE_URL" \
--secret "id=database_url,env=DATABASE_URL" \ --secret "id=database_url,env=DATABASE_URL" \
--secret "id=payload_secret,env=PAYLOAD_SECRET" \ --secret "id=payload_secret,env=PAYLOAD_SECRET" \
@ -134,7 +153,10 @@ trap 'rollback "unexpected failure"' ERR
cd "$STACK_DIR" cd "$STACK_DIR"
say "applying pending migrations (maintenance service teqo-1313-migrate)" say "applying pending migrations (maintenance service teqo-1313-migrate)"
docker compose --profile maintenance run --rm teqo-1313-migrate || rollback "migrations failed" # `< /dev/null`: `compose run` attaches stdin by default; the container would
# consume the rest of the script piped into `bash -s` (bash then hits EOF and
# exits right after the migrate step, skipping the rollout).
docker compose --profile maintenance run --rm teqo-1313-migrate </dev/null || rollback "migrations failed"
# --- rollout ------------------------------------------------------------ # --- rollout ------------------------------------------------------------

View file

@ -28,8 +28,14 @@ describe('scripts/deploy-homeserver.sh (OPS53 deploy pipeline)', () => {
expect(script).toContain('id=payload_secret,env=PAYLOAD_SECRET') expect(script).toContain('id=payload_secret,env=PAYLOAD_SECRET')
}) })
it('builds inside the compose network so the static generation reaches the prod DB', () => { it('reaches the prod DB from the build (loopback proxy on the compose network)', () => {
expect(script).toContain('--network stack_default') // BuildKit rejects `--network <bridge>`; the build uses --network host and
// a socat proxy (on stack_default, published on the host loopback) with a
// rewritten DATABASE_URL.
expect(script).toContain('--network host')
expect(script).toContain('teqo-1313-build-proxy')
expect(script).toContain('TCP:postgres:5432')
expect(script).toContain('127.0.0.1:5433')
}) })
it('applies migrations through the maintenance service before the rollout', () => { it('applies migrations through the maintenance service before the rollout', () => {