Barbelic devdocs

Appearance

유저 실사용 시뮬레이션 (Browser full-stack E2E) ​

v0.17.7 표시 이름 — #1470: 번호가 있는 브라우저 행동 검사는 유저 실사용 시뮬레이션(browser-journeys, 로컬 키 browser), 화면 크기·탭 구조 검사는 화면 정합성 검사(viewport-matrix, 로컬 키 viewport)로 표시한다. 명령·잡 ID·JSON 키·아티팩트 이름은 유지한다. 아래 과거 버전의 검사 수와 운영 기록은 당시 기준이며, 문서 게시만으로 앱 실행 설정이 바뀌지는 않는다.

Repository split (#1463): Human journey files (README.md and USER-JOURNEY.md) live under the same case path in dekerd/Barbelic-docs; executable case.json, regression tests, and support code remain in dekerd/Barbelic. The package descriptions below refer to the logical case across those repositories. Add or update each artifact in its owning repository and cross-link the changes. The app CI must not read the human documents. See repository boundaries.

Barbelic keeps two deliberately different integration suites:

  • e2e/crudRoundtrip.e2e.mjs and e2e/emptyAccountJourney.e2e.mjs exercise the real repository and database without rendering a browser.
  • error-cases/CASE-###-slug/regression.spec.mjs uses real Chromium, the shipped React UI, Supabase Auth/RLS/RPCs, raw PostgreSQL read-back, reload/re-entry, unexpected-error observation, and verified cleanup. These case-owned files cover both incident regressions and proactively selected expected user behaviors.

Each numbered directory is a self-contained package: USER-JOURNEY.md is the at-a-glance user-state and step-by-step UI route, README.md is the detailed human record, case.json is its typed registry manifest, and regression.spec.mjs is its executable journey. kind: "incident-regression" requires incident and Red/Green evidence; kind: "expected-behavior" may be added manually without a prior error or Red run. Both require linked critical checkpoints and the same fail-closed completion contract.

Numbered cases are split by execution boundary rather than by whether an incident occurred:

  • standard-full-stack cases are discovered by playwright.config.mjs and run against local Supabase plus deployed production.
  • production-external-oauth cases traverse the deployed app plus the real external identity provider.
  • production-provider-assumed-success cases use playwright.external-auth.config.mjs but replace only the provider-owned successful result boundary, then exercise every deployed app/Supabase/database layer.

The local and external-provider discovery sets are mutually exclusive. The local-only Auth Admin and fault-injection profiles are described below. Running the normal browser suite cannot silently mark an external-provider case Green.

Required PR gate (v0.17.2) ​

Every PR that changes executable code, configuration, dependencies, tests, migrations, or an unrecognized path runs the full automatic inventory. Only the explicitly listed documentation/image/template paths are exempt. A manual full-ci label can require the full run. Impact-based subsets are not enabled.

The independent e2e/required-coverage.json inventory contains 56 numbered browser journeys (CASE001–057, except CASE003) and 14 viewport scenarios. Browser jobs use four isolated Supabase stacks/shards; a separate evidence job checks the exact tested commit, complete shard set, discovered/tested IDs, first-attempt success, all seven completion signals, and cleanup. Missing cases, skipped/fixme/expected-failure tests, retries, missing artifacts, and a failed predecessor cannot satisfy required verify.

Run the same local stack preparation, real Edge Functions, app plus standalone admin build, tests, and evidence aggregation with:

bash
npm run ci:local -- --full

For focused development, npm run ci:local -- --only browser runs the numbered journeys; it is not the complete PR gate. The helper derives local Auth Admin credentials and the local JWT test secret from its isolated stack. Required runs set E2E_REQUIRED_PROFILE=local: missing local capability fails before discovery instead of silently switching to a credential subset.

  • local-auth-admin-full-stack provisions and verifies disposable users and supports destructive account/ownership checks only in local Supabase.
  • fault-injection-full-stack declares exact HTTP attempts and, where needed, recovery UI/error receipts or a bounded offline interval. Each declaration must actually occur and converge; it is not a general error ignore list. Local-only cases remain excluded from deployed credential runs.
  • CASE049/051 control only provider outcomes and then verify the real application's callback, Auth and database behavior. CASE003 remains separate provider-assumed QA evidence; none of these proves live provider consent or native SDK completion.

CASE048 uses real HTTPS, the shipped service worker, cached offline navigation, retained IndexedDB, a real worker update and one canonical write. It does not claim a changed application bundle; old/current bundle coexistence remains CASE039. Native devices, supervised provider completion, release performance budgets, and populated upgrade/restore checks retain their separate release evidence requirements.

Read the implementation record for exact completed work, actual runs and remaining release evidence.

Local full-stack run ​

Use the isolated local CI helper for the full mandatory run:

bash
npm run ci:local -- --full

화면 정합성 검사 (Viewport matrix) ​

e2e/viewport-matrix.spec.mjs (playwright.viewport.config.mjs, npm run test:e2e-viewport) renders the built bundle on the representative viewport of every tier in docs/platform/viewport-tiers.md (320 / 360 / 375 / 390 / 440 touch, 820 / 984 / 932×440 touch, 800×900 mouse, 1440×900 / 1280×720 desktop) and walks home · 일지 · 리포트 · 피드 · 내 정보. It asserts structure only — no horizontal overflow, column width equal to the tier rule, host background not transparent, desktop zoom floor — and never pixel snapshots. Screenshots land in test-results/viewport-matrix/ and CI uploads them as viewport-matrix-screenshots for eyeballing. It runs as 화면 정합성 검사 (viewport-matrix) with the same local Supabase environment:

bash
npm run ci:local -- --only viewport

Service-managed mode creates an authenticated disposable user, completes onboarding through the real RPC, injects that returned Supabase session before the app boots, and deletes the user after the run. Disposable-user setup may retry only recognized local Auth upstream/5xx failures, before a numbered journey starts; permanent setup failures and every retry-only Playwright case result still fail CI. The same production bundle is pointed at local Supabase through a test-only pre-boot runtime override; no login bypass or privileged browser API is added.

Production run ​

.github/workflows/prod-smoke.yml runs the lightweight CRUD canary after a successful Vercel Production deployment. The numbered browser cases are an explicit integration-release gate: after the automatic CRUD job is Green, run Production smoke manually on main with full_browser=true once for that release. The workflow rejects a manual non-main ref before checkout or production-secret access. A failed CRUD job prevents the more expensive browser suite from starting. Both lanes use only the public Supabase key plus the dedicated production E2E user's email/password:

  • PROD_SUPABASE_URL
  • PROD_SUPABASE_ANON_KEY
  • PROD_SMOKE_EMAIL
  • PROD_SMOKE_PASSWORD

The browser uses normal authenticated RLS. It creates uniquely marked records, verifies exact raw rows, deletes completed sessions, and archives custom exercises through an owner-bound RPC. Cleanup is read back through the same authenticated client before the run can pass; a production service-role key is never exposed to this workflow.

Every requested full-browser run publishes JSON results plus failure traces, screenshots, video, an HTML report, and a per-case journey-completion artifact. CI keeps one retry for diagnostic evidence but treats a retry-only pass as flaky and fails the gate. Missing production browser credentials fail the numbered-case job instead of producing a misleading skipped Green. A standard-full-stack case becomes production_verified only after the exact local and manually requested deployed browser jobs are recorded in its case.json.

Provider-assumed Kakao success run ​

CASE-003 exercises the desktop login screen, the real Kakao button, exact Supabase PKCE authorization request, a credential-free live check that production Supabase redirects that request to Kakao, the shipped callback/exchange-success path, a freshly issued browser session, existing-account resolution, owner-scoped profile read-back, and reload persistence.

Only Kakao-owned account authentication, mobile approval, consent, and the resulting external code exchange are assumed successful. The test runner fulfills that boundary with a session freshly issued for the existing production smoke account. It does not preload storage or add a production auth bypass; the app must persist the session through its normal exchangeCodeForSession success handling.

Run it with the existing production smoke account:

bash
E2E_PROVIDER_BOUNDARY_REQUIRED=1 \
E2E_APP_URL="https://www.barbelic.com" \
E2E_SUPABASE_URL="https://PROJECT.supabase.co" \
E2E_SUPABASE_ANON_KEY="..." \
E2E_TEST_EMAIL="..." \
E2E_TEST_PASSWORD="..." \
npm run test:e2e-provider-assumed-auth

.github/workflows/external-auth-smoke.yml exposes the same run manually and nightly when ENABLE_KAKAO_E2E is true. It reuses PROD_SUPABASE_URL, PROD_SUPABASE_ANON_KEY, PROD_SMOKE_EMAIL, and PROD_SMOKE_PASSWORD. Missing configuration fails the required run. Provider-page traces, screenshots, video, and artifact uploads remain disabled; evidence contains only sanitized origins, paths, request-contract booleans, and assertion results.

Green proves the conditional statement that, when Kakao returns a valid success result, Barbelic processes it all the way to the persistent authenticated home with valid API and PostgreSQL access. It does not prove that Kakao accepted a credential or mobile approval during the run. A separately supervised live-provider run may still be performed for release QA, but it must not be conflated with this deterministic profile.

<a id="date-independent-e2e-authoring"></a>

Execution dates and calendar assumptions ​

Apply the shared date/time test principles when adding or changing a journey. Tests may use the actual execution date and year. Supplying or freezing the execution clock is not a requirement, and rare failures caused by crossing midnight or a year boundary during a run are accepted. Fix invalid calendar assumptions that repeatedly fail on otherwise normal execution dates. The historical date-independent-e2e-authoring link is retained for existing references; it does not mean removing every dependency on the real clock.

  1. Keep fixtures valid for the intended business relation. Derive "today", "this year", or "previous year" from the actual business calendar when that is what the test means. Do not assume a hard-coded year is always in the past, or that a date a few days ago is still in the displayed month. Use the application's business timezone consistently instead of mixing UTC ISO dates with local calendar dates.
  2. Preserve the real user's date-selection path. Select the intended year/month/day through rendered UI and observe the application's original outgoing request, stored date, and read-back result. Repeat the necessary period navigation after reload or re-entry. Do not rewrite request dates, freeze global browser Date, or add application clock injection merely to make a real E2E pass. The selected UI date alone is not proof that the request or saved date is correct.
  3. Choose a period that can contain the required records. If a journey requires three distinct past dates, an annual report can use the previous calendar year derived from the actual current year and select that year through the UI. "Three past days this year" cannot work on January 1–2. Keep the required count, distinctness, zero-volume coverage, and date assertions; do not truncate the journey or lower expectations to fit an invalid fixture. A test specifically about year-to-date behavior must expect what that period can actually contain.
  4. Respect real data windows and authentication. A fixture's date must fall within the real query/snapshot period and have the intended validity. Wait for required bootstrap/statistics work and verify the real generation and returned data. A stale fixed fixture must not turn a meaningful assertion into an empty/expired result. Keep actual Auth, server, and network clocks running. Where an existing isolated fixture scope is necessary, preserve its cleanup and never suspend shared/production jobs.
  5. Accept the rollover race and fix invalid assumptions. A shared helper reading today's date is not a defect by itself and does not justify converting all its callers to fixed clocks. Do not add execution-time parameters, suite-wide clock overrides, blanket retry rules, or date-matrix requirements to remove the accepted rollover race. Existing narrow tests of expiry/timer mechanics may retain their clock tools without making those tools the default for ordinary journeys.
  6. Validate the affected failure and reuse unchanged evidence. Run the affected helper/test or exact browser case and record how the invalid calendar assumption is resolved. Preserve the original date/count/storage correctness checks. After a previously passing full CI, do not repeat it solely for this date investigation; reuse passing evidence for unchanged coverage. Moving a fixed literal to a newer date, extending a product TTL, or deleting assertions is not a fix.

Adding the next case ​

  1. Allocate the next shared CASE-###; never reuse or renumber an ID. Create error-cases/CASE-###-slug/ with USER-JOURNEY.md, README.md, case.json, and regression.spec.mjs.
  2. Choose incident-regression for an observed failure or expected-behavior for a manually selected product journey. Then declare standard-full-stack, local-auth-admin-full-stack, fault-injection-full-stack, production-external-oauth, or production-provider-assumed-success according to the real route boundary. A provider assumption must be explicit and cannot claim live-provider evidence.
  3. In USER-JOURNEY.md, record the user's tier/auth/data/device/start state and sequential Step 01 through Step n rows. Every row must state the user action, expected rendered result, and expected system/database result; the exact final outcome must match case.json.intendedOutcome.
  4. Record the complete real user route and define as many critical checkpoints as the intended experience needs. Put every exact checkpoint string in case.json, the README, and a named test.step with direct assertions.
  5. Map the checkpoints to every completion signal and execute their assertions through caseContext.prove(signal, callback). A signal is recorded only after its callback succeeds.
  6. Reproduce the user's real input sequence through rendered UI and finish it. Apply the execution-date and calendar-assumption checklist. Assert frontend state, authenticated API/RPC behavior, raw database rows, reload/re-entry, and final rendered output. Patch-presence and nearby checks never substitute for these outcomes.
  7. End with caseContext.completeJourney(page, monitor). It requires all signals, rejects fatal/rendered errors, exceptions, console errors, app error reports, relevant 4xx/5xx and network failures, verifies cleanup by read-back, and fails if any unexpected problem appears despite individual checkpoint success.
  8. For an incident only, run the exact journey against the pre-patch application and preserve the job that reaches the original trigger and fails at a declared product checkpoint. Expected-behavior cases intentionally omit Red proof.
  9. Add a mandatory local case to the independent e2e/required-coverage.json inventory. Run npm run check:error-cases, the focused browser case, npm run check, and npm run build, then preserve entirely Green evidence for every environment required by the selected execution profile in case.json.

Mock-only, component-only, repository-only, save-without-read-back, and shortened journeys remain useful supporting tests, but none can close a numbered full-stack behavior case.