refactor(miniapp): migrate to v2 data layer#14049
refactor(miniapp): migrate to v2 data layer#14049chengcheng84 wants to merge 112 commits intoCherryHQ:v2from
Conversation
Signed-off-by: suyao <[email protected]> # Conflicts: # migrations/sqlite-drizzle/meta/0006_snapshot.json # migrations/sqlite-drizzle/meta/_journal.json # packages/shared/data/api/schemas/index.ts # src/renderer/src/config/minapps.ts
Critical fixes: - C4: Derive BUILTIN_APP_LOGO_MAP from shared preset data instead of manual map - C5: Return OffsetPaginationResponse from GET /miniapps (consistent with other endpoints) - C6: Add Zod runtime validation schemas and parsing in all miniapp handlers Important fixes: - I1: update() returns merged preset for builtin apps (consistent with getByAppId/list) - I2: Validate .returning() result in update/updateStatus, throw notFound if missing - I3: Move ensureDefaultAppPref into transaction + batch queries in reorder() - I4: Replace SELECT-then-INSERT with INSERT ON CONFLICT DO NOTHING (TOCTOU fix) - I5: Whitelist preference-only fields for default app updates - I6: loadCustomMiniApp only overwrites on ENOENT, not on transient errors - I9: Add CHECK constraints on status/type enum columns - I10: Add MiniAppSelect/MiniAppInsert type aliases (project convention) - I11: Fix naming — path param :appId→:id, remove handler alias, StatusUpdateResponse - I12: Remove redundant COUNT query in list() Also fixes: - Add missing reset() method to MiniAppMigrator (BaseMigrator contract) - Add missing Minimax/Dangbei/Ima icon imports and switch cases in getMiniAppsLogo - Update moonshot test expectation to match preset-derived logo key Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: suyao <[email protected]>
Only background and configuration support nullable (explicit clear to null), based on business logic: - logo: must always have a value (visual identity) - supportedRegions: must be explicitly set (region visibility control) - background: nullable — clearing resets to default/transparent - configuration: nullable — clearing removes custom config Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: suyao <[email protected]>
Replace CompoundIcon reference with string key 'openclaw' to match the v2 miniapp logo resolution pattern. Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: suyao <[email protected]>
Every custom miniapp must have: - logo: visual identity is mandatory - bordered: UI rendering needs explicit value - supportedRegions: region filtering depends on this field background and configuration remain optional. Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: suyao <[email protected]>
a4e124a to
ef13948
Compare
c510e33 to
7705457
Compare
Signed-off-by: suyao <[email protected]> # Conflicts: # packages/shared/data/api/schemas/index.ts # src/main/data/api/handlers/index.ts # src/main/data/services/MiniAppService.ts # src/main/data/services/__tests__/MiniAppService.test.ts
Signed-off-by: suyao <[email protected]> # Conflicts: # packages/shared/data/api/schemas/index.ts # packages/shared/data/bootConfig/bootConfigSchemas.ts # packages/shared/data/preference/preferenceSchemas.ts # src/main/data/migration/v2/migrators/mappings/ComplexPreferenceMappings.ts # src/main/data/migration/v2/migrators/mappings/__tests__/ComplexPreferenceMappings.test.ts # src/main/data/services/MiniAppService.ts # src/main/data/services/__tests__/MiniAppService.test.ts # src/renderer/src/components/TopView/index.tsx # src/renderer/src/components/app/Sidebar.tsx # src/renderer/src/hooks/useAppInit.ts
Signed-off-by: suyao <[email protected]>
Signed-off-by: suyao <[email protected]> # Conflicts: # migrations/sqlite-drizzle/meta/_journal.json # src/main/data/services/MiniAppService.ts
oxlint --deny-warnings was failing CI on unused variables from useTabs() destructuring in Sidebar.tsx. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> Signed-off-by: suyao <[email protected]>
|
@DeJeune any updates |
Signed-off-by: suyao <[email protected]> # Conflicts: # packages/shared/data/api/schemas/index.ts # src/main/data/api/handlers/index.ts # src/main/data/migration/v2/migrators/mappings/__tests__/ComplexPreferenceMappings.test.ts # v2-refactor-temp/tools/data-classify/data/target-key-definitions.json
- Fix stale doc comment in MiniAppService (miniappTable → miniAppTable) - Rename private miniappsStatusKeyMap → miniAppsStatusKeyMap in i18n/label.ts - Fix JSX comment casing in MiniAppSettings Generated routeTree.gen.ts intentionally keeps lowercase 'miniapp' since it derives from the URL path /app/miniapp/. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> Signed-off-by: suyao <[email protected]>
There was a problem hiding this comment.
Thanks for the thorough v2 data-layer migration. The overall architecture (DataApi + Cache + Preference split, builtin-preset + DB-row merge, migrator pipeline) is solid and CI is green. However, several blocking issues — around correctness, silent failures, and a naming-convention decision the team needs to commit to — should be resolved before merge.
Critical (must fix)
C1. Dynamic path is not URL-encoded / input is not whitelisted
File: src/renderer/src/hooks/useMiniApps.ts:174-190
const result = await dataApiService.patch(`/miniapps/${appId}`, { body })CreateMiniAppSchema.appId only enforces z.string().min(1), so /, ?, #, whitespace, or " can escape path matching. The same applies to MiniAppPage.tsx:108 where document.querySelector('webview[data-miniapp-id="${appId}"]') can be broken by a " in the id.
Fix: Tighten the Zod schema (z.string().regex(/^[A-Za-z0-9_-]+$/)), use encodeURIComponent(appId) in API paths and CSS.escape(appId) in DOM selectors.
C2. MiniAppService.update() — side effect before validation, and non-atomic ensure→update
File: src/main/data/services/MiniAppService.ts:244-277
ensureDefaultAppPref(appId)runs before the "no applicable fields" validation. Sending{ name: 'x' }for a default app returns 400, but a default-preference row has already been inserted as a side effect.ensureDefaultAppPref→updateis two separate statements. A concurrentresetDefaults()between them makesupdatereturn 0 rows, surfacing as a misleadingnotFoundfor a builtin id.
Fix: Validate updates is non-empty before ensureDefaultAppPref(); wrap both steps in a single db.transaction, or use INSERT ... ON CONFLICT DO UPDATE for builtin apps.
C3. useMiniAppPopup module-level mutable state breaks multi-instance usage
File: src/renderer/src/hooks/useMiniAppPopup.ts:32-58, 99-148
miniAppsCache, cacheCallbacksRef, and cacheVersion are module-scoped globals mutated on every render. In multi-window or concurrent-component scenarios, only the most recently mounted consumer's setOpenedKeepAliveMiniApps is invoked from disposeAfter/onInsert, silently desyncing others. The resize effect reassigns the global cache, but parent components holding the old reference don't re-render.
Fix: Move the cache into useRef or a lifecycle-managed singleton service, and pass setters via proper React refs.
C4. Promise.allSettled swallows partial failures in updateMiniApps family
File: src/renderer/src/hooks/useMiniApps.ts:204-288
return results.filter(r => r.status === 'fulfilled').map(r => r.value)
window.toast?.error('Failed to update miniapps')- Callers can't distinguish "all succeeded" from "partially failed"; local UI drifts from DB with no refetch/rollback.
window.toast?.errorsilently no-ops ifwindow.toastisn't registered (e.g. early boot or mini-window context).- Hardcoded English string — violates the i18n rule.
Fix: Aggregate rejections and throw (or return { fulfilled, rejected }). Route the message through i18next. Trigger invalidate('/miniapps') to resync UI.
C5. loadCustomMiniApp silently drops user data on any file error
File: src/renderer/src/config/miniapps.ts:91-117
Top-level await loadCustomMiniApp() returns [] on any JSON parse / read / permission error at module evaluation. A single corrupt character in custom-miniapps.json permanently hides all custom mini-apps with no user-facing notice.
Fix: On failure, quarantine the bad file as custom-miniapps.json.broken-{ts} before returning [], and surface a toast on next render.
C6. MiniAppMigrator.test.ts violates the DB-testing rule
File: src/main/data/migration/v2/migrators/__tests__/MiniAppMigrator.test.ts
CLAUDE.md: "For any service/handler/seeder that reads or writes SQLite, use setupTestDatabase() from @test-helpers/db". The test hand-rolls vi.fn().mockReturnThis() chains, so it can't catch batch-size regressions, UNIQUE/FK constraint issues, or validate-step ordering bugs.
Fix: Convert to setupTestDatabase() and assert against real DB reads.
C7. Naming convention: adopt Plan A (mini-app / mini_app)
See the Naming Decision section below. This PR carries the name in its first persistent form (SQLite table, API endpoints, preference keys). Fixing it now is a rename; fixing it later is a migration.
Important (strongly recommend fixing)
I1. API path design
- 【这个先HOLD住,暂不用改。reorder的需求比较多,我会做一个统一的规范】
PATCH /miniappsfor batch reorder conflicts with "PATCH = partial update of one resource". Per the api-design-guidelines decision tree, change toPUT /mini-apps/orderwith body{ items: [...] }. DELETE /miniapps/defaultsis semantically a reset action, not a resource deletion. ConsiderPOST /mini-apps/defaults/reset.
I2. Fake pagination in list()
File: src/main/data/services/MiniAppService.ts:187-191
return { items: allItems, total: allItems.length, page: 1 }query has no page/limit, so declaring OffsetPaginationResponse<MiniApp> misleads clients. Align with TagSchemas and return bare MiniApp[].
I3. Double merge of builtin + DB rows across processes
MiniAppService.builtinToMiniApp (main) and useMiniApps.mergeWithPreset (renderer) both fold logo, bordered, background, nameKey, supportedRegions from presets. Drift between them will surface as silent behavior differences.
Fix: Centralize merge in main (Registry Service pattern, per data-api-in-main.md). Renderer consumes the already-merged MiniApp.
I4. Missing withSqliteErrors wrappers
File: src/main/data/services/MiniAppService.ts
create() wraps inserts correctly. update, delete, reorder, resetDefaults, and ensureDefaultAppPref call Drizzle raw, leaking DrizzleQueryError instead of DataApiError on constraint violations.
I5. reorder silently skips missing appIds
File: src/main/data/services/MiniAppService.ts:357-370
result.length === 0 → skipped.push(...) only logs a warning and the handler returns 204 — renderer cannot detect skips.
Fix: Either throw notFound on the first missing id, or return { skipped: [...] } in the response.
I6. antd / styled-components in v2 files
Files: MiniAppPopupContainer.tsx:25 (Alert from antd), MiniAppTabsPool.tsx, MiniAppPage.tsx, WebviewContainer.tsx (all heavy styled.div usage).
CLAUDE.md v2 rule forbids antd / styled-components in new code. Either migrate to @cherrystudio/ui + Tailwind, or mark files as *.v2.tsx WIP per the naming convention.
I7. Hand-maintained MiniApp type drifts from DB schema
packages/shared/data/types/miniapp.ts is hand-written and already diverges from MiniAppSelect on nullability (logo, bordered).
Fix: Derive from $inferSelect, the same way tag DTOs are derived from their Zod schema.
I8. sortOrder collisions between migrator and builtin defaults
File: MiniAppMigrator.ts:96-111
Per-status reindexing to 0..N-1 collides with builtinMiniAppDefaultSortOrder and with create()'s Math.max(builtinCount, ...) logic. Custom apps end up with sortOrder colliding with builtins until the user reorders.
Fix: Offset custom-app sortOrder by builtinMiniAppDefaultSortOrder.size.
I9. Stale closures and ref churn
MiniAppPopupContainer.tsx:300-320, 345-360—useEffectwithreact-hooks/exhaustive-depsdisabled readscurrentAppInfo/currentMiniAppIdthat aren't in deps.WebviewContainer.tsx:32-43—setRef(appid)returns a new function every render, causing React to call the old ref withnulland the new one with the element on every render.
Fix: Memoize handlers with useCallback and include them in deps.
I10. any types in WebviewContainer
WebviewContainer.tsx:66, 220 — event: any, 'true' as any. Use Electron.DidNavigateInPageEvent and allowpopups={'true'}.
I11. MiniAppKind half-rename
DB layer renamed the discriminator type MiniAppType → MiniAppKind, but the column (type), indexes (miniapp_type_idx, miniapp_status_type_idx), check constraint (miniapp_type_check), Zod schema (MiniAppTypeSchema), query field, and entity field on MiniApp all still say type. A partial rename is worse than either extreme.
Fix: Complete the rename — column → kind, indexes → mini_app_kind_idx / mini_app_status_kind_idx, check → mini_app_kind_check, Zod → MiniAppKindSchema, entity field → kind. Combine with Plan A below.
Naming Decision: adopt Plan A
The team has confirmed MiniApp is a compound term (the capital A in PascalCase is the evidence). By the mechanical mapping rule the repo already applies to every other compound noun (MCPServer → mcp-server, KnowledgeBase → knowledge-base, ActiveNode → active-node, RotatedKey → rotated-key, AuthConfig → auth-config), the forms must be:
| Form | Value |
|---|---|
| PascalCase | MiniApp |
| camelCase | miniApp |
| kebab-case | mini-app |
| snake_case | mini_app |
| SCREAMING_SNAKE | MINI_APP |
Rule: a case boundary in any one form must appear as a separator in all other forms.
Required renames in this PR
A. API paths (stateless, free to change)
/miniapps→/mini-apps/miniapps/:id→/mini-apps/:id/miniapps/defaults→/mini-apps/defaults
B. File names (per-directory sub-convention)
api/schemas/miniapps.ts→api/schemas/miniApps.ts(schemas uses camelCase, cf.mcpServers.ts,fileProcessing.ts,temporaryChats.ts)types/miniapp.ts→types/miniApp.tspresets/miniapps.ts→presets/mini-apps.ts(presets uses kebab-case, cf.file-processing.ts,translate-languages.ts)
C. Routes / URL segments
routes/app/miniapp/$appId.tsx→routes/app/mini-app/$appId.tsxroutes/app/miniapp/index.tsx→routes/app/mini-app/index.tsxpages/miniapps/→pages/mini-apps/- All literal path strings (
TabsService.ts:67,useMiniAppPopup.tsnavigation, etc.)
D. DB schema (persisted — but this PR introduces the table for the first time, so no migration chain yet; regenerate)
- Table:
'miniapp'→'mini_app' - Indexes:
miniapp_status_sort_idx→mini_app_status_sort_idx,miniapp_type_idx→mini_app_kind_idx,miniapp_status_type_idx→mini_app_status_kind_idx - Checks:
miniapp_status_check→mini_app_status_check,miniapp_type_check→mini_app_kind_check - Re-run
pnpm agents:generateand replace the in-PR migration file. - Variable
miniAppTableand column names (app_id,sort_order,supported_regions) are already correct.
E. Preference / Cache keys (persisted, but brand new in v2 and not yet shipped)
feature.miniapp.region→feature.mini_app.regionfeature.miniapp.max_keep_alive→feature.mini_app.max_keep_aliveminiapp.opened_keep_alive→mini_app.opened_keep_aliveminiapp.current_id→mini_app.current_idminiapp.show→mini_app.showminiapp.opened_oneoff→mini_app.opened_oneoffminiapp.detected_region→mini_app.detected_region- Touchpoints:
preferenceSchemas.ts,cacheSchemas.ts,cacheValueTypes.ts,PreferencesMappings.ts,v2-refactor-temp/tools/data-classify/data/classification.json; then re-runcd v2-refactor-temp/tools/data-classify && npm run generate. - Do not rename Redux source keys in migrators (
'minapps','minAppRegion','minappsOpenLinkExternal') — they must match historical state.
F. Lingering legacy spellings (cleanup)
ORIGIN_DEFAULT_MIN_APPS→ORIGIN_DEFAULT_MINI_APPS(21 references across 13 files)loadCustomMiniApp→loadCustomMiniApps(returns an array)classification.json:"DEFAULT_MIN_APPS"→"DEFAULT_MINI_APPS"
G. Components (already correct)
components/MiniApp/*— PascalCase component directory stays as-is (React convention).
Protected (do not rename — v2 freeze)
Redux-persisted identifiers that must remain to preserve user state:
src/renderer/src/store/miniapps.ts:MinAppsState, slice name'minApps',setMinApps,setDisabledMinApps,setPinnedMinAppssrc/renderer/src/store/settings.ts:minappsOpenLinkExternal,setMinAppRegionsrc/renderer/src/store/migrate.ts: all legacy key referencesv2-refactor-temp/.../classification.json/inventory.json: source-key mirrors
Test coverage gaps
Must add
- Concurrent
create()→ 409 —Promise.all([create(dto), create(dto)])yields exactly one CONFLICT. Guards against reintroduction of a select-then-insert race. MiniAppMappings.transformMiniAppunit tests — logo URL vs icon-key vs fallback,supportedRegionsnormalization,bordered/boderedtypo compatibility.reordertransaction rollback — simulate an in-loop throw and assert earlier updates rolled back.- Region auto-detection — reset
regionDetectionPromisebetween tests; mockwindow.api.getIpCountryresolving'CN'/'US'and rejecting; verifysetDetectedRegioncalls. updateMiniAppspartial-failure branch — mixed-resultPromise.allSettled, assert toast invoked and fulfilled values returned.
Should add
reorder([])no-op.update()with{}and{ name: undefined }for both default and custom apps.list({ type: 'custom', status: 'pinned' })combined filter.- LRU cache resize path when
maxKeepAliveMiniAppschanges between renders (the in-component copy branch).
Shallow tests to tighten
reorderMiniAppshook test only assertstypeof === 'function'— tautology. Assert the mutation trigger was called with the items.- Migrator "dedup-then-reindex" input contains no duplicates; make dedup visible.
- Service "sort by status priority then sortOrder" only checks two positions; assert full ordering including the within-status tiebreak.
useMiniApps"no-op when visible list unchanged" should assertpatchAppwas never called, not justallApps.length === 1.
Minor / cleanup
- Delete dead commented references in
runtime.ts:59-65, 150-155,i18n/label.ts:180,types/index.ts:665,sidebar.ts:15. - Update
// MinAppssection headers insettings.ts:196, 383to// MiniApps. - Wrap LRU
disposeAfter/onInsertcallbacks (useMiniAppPopup.ts:91-103) intry/catch+logger.error. - Replace
setTimeout(..., 100)retry inTabsService.ts:67with a proper readiness Signal. - Region-detection fallback to
'CN'should emit a one-time structured error for observability. MiniAppMigrator.validateshould track original-source count instats.sourceCount(currently equal topreparedRows.lengthby construction, hiding silent drops).
Strengths
- DataApi scope is fully compliant — every endpoint backs a SQLite row.
create()correctly relies on DB UNIQUE +withSqliteErrorsinstead of select-then-insert (race-free pattern from the docs).- Migrator pipeline is robust: priority-based dedup (pinned > enabled > disabled),
boderedtypo tolerance, batched insert in a transaction. - Cache/Preference keys follow the
domain.snake_caseconvention consistently. - Region filtering is correctly modeled as a view concern; mutations operate on raw DataApi data.
- Centralized logging via
loggerService.withContext(...); noconsole.logintroduced. - Clean deletion of legacy files: old
MinApp/,useMinapps.ts,useMinappPopup.ts,MinAppIcon.tsx,routes/app/minapp/.
Suggested merge path
- Resolve C1–C7 (7 blockers).
- Complete I1–I11 (API shape, merge centralization, error translation, v2 UI rule, type derivation, half-rename).
- Apply Plan A rename — highest leverage on D (DB) and E (preferences) because they're first-write persistent state; leaving them now forces a migration later.
- Add Must-add tests (especially migrator → real DB).
- Clean up dead comments.
Thanks again for the thorough work — this is a large, well-structured migration. With these adjustments, the naming gets locked in once and for all before any of it hits production.
- API paths: /miniapps → /mini-apps - DB table: miniapp → mini_app - Indexes/checks: miniapp_* → mini_app_* - Preference keys: feature.miniapp.* → feature.mini_app.* - Cache keys: miniapp.* → mini_app.* - Constants: ORIGIN_DEFAULT_MIN_APPS → ORIGIN_DEFAULT_MINI_APPS - Sidebar icon: 'miniapp' → 'mini_app' - Update all tests and mock paths accordingly - Add migration 0012_striped_hex
Critical: - C1: appId regex validation + encodeURIComponent + CSS.escape - C2: validate before side effects, wrap update in transaction - C3: module-level state → sharedCache + useRef - C4: Promise.allSettled → settleAndInvalidate + i18n error - C5: quarantine broken custom-miniapps.json - C6: MiniAppMigrator tests → setupTestDatabase() - C7: Plan A naming (mini-app, mini_app, mini_app.*) Important: - I4: ensureDefaultAppPref atomic within transaction - I11: mini_app_type_* → mini_app_kind_* Remaining I1-I3, I5-I10 follow-up PRs.
- Rename `type` to `kind` in MiniApp schema, DB table, and all references - Update `MiniAppService.list()` to return `MiniApp[]` directly instead of a paginated wrapper - Add TODO annotations for API endpoints that deviate from design guidelines (batch reorder and reset-defaults) - Fix `WebviewContainer` ref handling and type safety - Remove unused styled component in `MiniAppsPage` - Add missing `withSqliteErrors` wrappers in service methods
What this PR does
After this PR:
@shared/data/types/miniapp.tsuseMinappshook usinguseQuery/useMutationfrom DataApiappIdinstead ofid/app/miniapp/$appIdBreaking changes
None
Special notes for your reviewer
Checklist
MiniAppMigratorgh pr diffRelease note