feat(ai-image): AI 封面图生成链路（配置 / 运行时 / 任务队列 / 后台抽屉）#2775
Merged
Conversation
Adds the AI image-generation config surface (provider/apiKey/model/endpoint plus default aspect ratio/quality/format) and three error codes (IMAGE_GENERATION_DISABLED, IMAGE_PROVIDER_NOT_CONFIGURED, IMAGE_GENERATION_FAILED) that later image-generation work depends on.
.default() chained after field.xxx() attaches to a new ZodDefault wrapper, losing the metadata Symbol field.xxx already set on the inner schema. Move .default() inside the schema argument for provider/defaultAspectRatio/ defaultQuality/defaultFormat so DSL title/halfGrid/select-options survive. Also replace the tautological config default test (comparing generateDefaultConfig() against itself) with hardcoded literal assertions.
Wraps @earendil-works/pi-ai's generateImagesOpenRouter behind a vendor-neutral IImageRuntime, with pure param-mapping functions that route neutral aspectRatio/quality/format through onPayload for OpenAI-shaped and Gemini-shaped targets.
pi never throws on generation failure; it returns
{ stopReason: 'error'|'aborted', errorMessage, output: [] }. Pins the
adapter's explicit stopReason check with a mocked
generateImagesOpenRouter boundary so a regression that drops the
check fails the test instead of shipping a zero-byte image.
Moves the S3-or-local buffer upload logic (filename/prefix templating, S3Uploader construction, pending-reference bookkeeping) out of FileController.upload into a reusable FileService.uploadBuffer method, so other callers (e.g. AI image generation) can persist buffers through the same storage path as manual uploads. Video keeps its untouched streaming path in the controller.
Adds CoverStylePreset registry to ai.prompts.ts with the signal-geometry preset (distilled from CaliCastle/skills SKILL.md, MIT) and an AI_PROMPTS.cover.compile TypeBox schema producing the nine-axis recipe plus four-paragraph English prompt. Extends the ai-prompts regression fixture suite accordingly.
…literal Export the named preset so it's reachable without indexing the id-keyed registry, and hoist the '16:9' aspect ratio into one const referenced by both defaultAspectRatio and the compiled system prompt's template var.
Adds AITaskType.ImageGeneration + payload, a dedup case keyed on
requestId (so re-generating from the same prompt always yields a new
task), AiTaskService.createImageGenerationTask, and AiImageService's
task handler: validates config, calls IImageRuntime.generateImage,
uploads the first image via FileService.uploadBuffer, and records
{ url, mimeType, prompt } as the task result.
Match every sibling AITaskType member's ai: prefix instead of the bare 'image-generation' literal, since the value is a Redis-persisted discriminator with no PG column behind it — cheap to rename now, free after a deploy.
Adds the HTTP surface for AI image generation: draft-prompt (compiles a cover-preset prompt via the writer text model), generate (enqueues via the existing task queue, no idempotency by design), and presets/models listing. Registers AiImageController in ai.module.ts alongside its sibling AI controllers, mirroring the existing AiSummaryController pattern rather than introducing new cross-module DI wiring.
…y fallback resolveArticleForDraftPrompt crashed with an uncaught TypeError (500) when a note/page (or empty post) had text: null — the exact shape of an in-progress, not-yet-written article, which is the primary use case for drafting a cover mid-authoring. Falls back to an empty summary instead, matching the ?? '' idiom already used for this field in ai-summary.service.ts. Also removes the unused presetList/modelList view exports and adds a regression test for the null-text path.
Add a drawer to draft a prompt, generate cover candidates via the existing ai/image endpoints, and select one into meta.cover, gated on imageGenerationOptions.enable. Reuses the existing task subscription mechanism for live completion updates instead of polling.
Registers a generate_image tool in the write-page AI agent (gated on imageGenerationOptions.enable) that generates an image via POST /ai/image/generate (purpose: inline, fresh requestId per call) and returns the resulting URL as plain tool content once the backend task completes, letting the agent place it via its own insert_node call. Extracts the image-task status interpretation shared by the cover generation drawer (Task 7) and this tool into resolveImageTaskOutcome, and adds a promise-based waitForImageTask poller for non-React callers.
…ype, timeout, preset optionality - image-runtime.adapter: resolve baseUrl from the pi image-model catalog when endpoint is blank, instead of falling through to '' (which sent OpenRouter credentials to api.openai.com by default) - admin: add AITaskType.ImageGeneration enum member, label/filter entries, and task-summary fallback (was degrading to a bare snowflake refId) - ai-image.service: pin the stopReason:'stop'-with-no-images guard with a regression test - admin waitForImageTask: bound the poll loop with a default timeout so a stuck task can't suspend the agent tool forever - CoverGenerationDrawer/use-cover-generation: stop requiring presetId to generate, matching GenerateImageDto's optional presetId - design doc: correct the claim that task responses never echo payload
…level Re-review found the model-level catalog lookup still leaked OpenRouter credentials to api.openai.com whenever the configured model wasn't in pi-ai's static image-model snapshot (a real, growing gap since OpenRouter adds image models continuously). Resolve from the provider's catalog entries instead — every model under a provider shares one baseUrl — and throw IMAGE_PROVIDER_NOT_CONFIGURED rather than silently falling through to '' when the provider itself has no catalog entry. Also: fixed a pre-existing missing-mock-reset bug in image-runtime.adapter.spec.ts that let several assertions silently check the wrong test's call args, added the two test-gap cases re-review named, corrected a wrong justification in the fix report, and fixed a route typo (/task/:id -> /tasks/:id) in the design doc.
… filter params by model capability The prior transport routed through pi's built-in 'openrouter-images' API, which builds a chat-completions-shaped payload (model/messages/stream/ modalities). mapOpenAIImageParams/mapGeminiImageParams patched aspectRatio/ quality/format onto that payload via onPayload as inert top-level keys (size/output_format/generationConfig) nothing on the wire ever consumed — and the two vendors' actual parameter support (OpenAI has no aspect_ratio, Gemini has no quality, confirmed live against OpenRouter's /api/v1/images/models catalog) was never respected. Replace it with a self-written transport (openrouter-images-api.ts) that POSTs /images directly, plus a live capability catalog (image-catalog.ts) fetched from /api/v1/images/models. Before building a request, image-param-mapping.ts's buildOpenRouterImageParams filters aspectRatio/ quality/format down to whatever the resolved model's supported_parameters actually lists, dropping the rest instead of sending them on a guess. providerParams still passes through verbatim (maps to allowed_passthrough_parameters). referenceImages now wires into input_references (previously dead code with no populating caller, and even if populated would have landed as a chat-message image_url instead). GET /ai/image/models now fetches this same live catalog instead of mapping pi's static ~40-entry snapshot, and exposes supported_parameters per model for a future capability-aware admin UI. resolveBaseUrl simplifies accordingly: the provider owns its own endpoint (openrouter.ai default, explicit config wins) with no pi catalog lookup in the path, so there's no longer a code path that can fall through to api.openai.com. Note: pi-ai's registerImagesApiProvider (the generic provider-registration entry point the design assumed) is not part of the package's public surface in 0.82.x — index.ts/providers/*/api/* don't re-export it and the exports map blocks any deep import. ImageRuntimeAdapter therefore calls the new transport directly rather than through pi's generateImages() dispatcher; it still uses pi's public types (ImagesModel/ImagesContext/ ImagesOptions/AssistantImages) for shape compatibility.
…rrect design docs Follow-up after team-lead independently re-verified the registerImagesApiProvider finding and confirmed createImagesModels/createImagesProvider doesn't help either (fixed ImagesOptions, not the generic TOptions the picture parameters need) — proceed with the direct-call approach as designed. - openrouter-images-api.ts: type the response against the documented shape directly (string/number) instead of unknown+typeof everywhere; keep only the guard that matters (Array.isArray(payload.data)) since a well-formed 200 with a missing/empty data array is a real failure mode this feature has been bitten by twice, not a legitimate zero-image success. - Add two transport-level tests pinning that contract directly (missing data array, and items with no b64_json) rather than relying solely on ai-image.service.ts's existing downstream guard. - Correct the registerImagesApiProvider claim in the two tracked design docs that stated it, in place, so the reasoning is discoverable rather than a silent departure (docs/superpowers/specs and .../plans for this feature).
…ms, share capability cache
Code review found a critical regression from the previous round's response-
parsing simplification: parseImageItem's weakened truthy check (!x instead of
typeof x !== 'string') let non-string b64_json values (number/object/boolean/
array) reach Buffer.from() in image-runtime.adapter.ts unguarded. Verified:
123/{}/true throw an uncaught TypeError outside the transport's try/catch;
['a'] decodes to a 1-byte garbage buffer that would be uploaded to S3 and
recorded as a successful result. Fixed at the decode boundary in the adapter
(the IImageRuntime contract every future caller inherits, not the transport),
with a guard for both non-string data and zero-byte decodes.
Also, per review:
- openrouter-images-api.ts: log (Logger.warn, naming the model + dropped
keys) whenever a requested param gets dropped, whether because the
capability catalog fetch failed (fail-closed) or because a resolved model
simply doesn't support that key — same code path covers both, since both
just mean "key absent from supportedParameters".
- image-param-mapping.ts: validate enum values, not just key presence.
Parsed-but-unused capability data (supported_parameters.values) was exactly
the inert-table failure mode this task exists to eliminate — an admin's
free-text defaultAspectRatio could reach a model whose enum doesn't include
it and 400. Range/boolean descriptors are unaffected.
- image-catalog.ts: share the capability-lookup cache (getImageCatalog/
getImageCatalogModel, stale-while-revalidate keyed by resolved base URL)
between GET /ai/image/models and the generate-path lookup, replacing the
controller's ~50 lines of duplicated Map/TTL bookkeeping with one call.
- configs.schema.ts: fix the `provider` field description — it invited the
inverse of the 76c30db bug (configuring provider: 'openai' with a blank
endpoint still sends the key to openrouter.ai, since routing is fully
endpoint-driven now).
- Removed a fix-history-referencing comment and a comment restating control
flow, per the zero-comments policy.
…im comments Final review round, 4 small items: - image-catalog.ts: delete fetchImageCatalogModel — zero production callers remained after the cache work switched resolveSupportedParameters to getImageCatalogModel; it existed only to be exercised by its own tests. fetchImageCatalog itself stays (getImageCatalog calls it internally). - openrouter-images-api.ts: parseImageItem now validates media_type is a non-empty string before using it (same failure class as the b64_json fix: the field is typed string but not runtime-validated, so a truthy non-string value would otherwise reach uploadBuffer's contentType and the stored task result untouched). - image-catalog.spec.ts: add a test driving the refresh-failure path — proves `refreshing` gets reset on a rejected background fetch so a later request can retry, rather than being stale-locked forever. - Comment trims: decode-guard comment down to the invariant it protects; deleted a 3-line docstring-shaped comment on an internal function; a catch comment already restated by the logger.warn two lines below; a test comment restating what the test name already says.
…cycle, surface image config in admin AI group
- provider becomes a real openrouter|custom select instead of a decorative label - model field in the admin config DSL becomes a catalog-backed select (attachImageModelOptionsToFormDSL), falling back to free text with a warn when the catalog can't be fetched - image catalog models now carry a name for use as the select label - generate endpoint accepts a per-request model override (GenerateImageSchema/ImageGenerationTaskPayload), falling back to the configured default when omitted - admin cover drawer gets a model dropdown sourced from GET /ai/image/models
Selecting the custom endpoint preset with a blank endpoint silently degraded to the OpenRouter default instead of failing at save time. Mirrors the existing validateMailProvider pattern: merge the patch into the current config, then reject provider: custom when endpoint is still empty.
Two UX fixes from live testing: - endpoint now only shows when provider is "custom" (showWhen, mirroring the existing mailOptions smtp/resend precedent), since selecting the openrouter preset shouldn't require knowing its URL. - the config-page model catalog fetch no longer requires enable: true first. GET /api/v1/images/models needs no credentials, so the dropdown can populate during first-run setup instead of after the toggle is flipped. The options.controller.e2e-spec.ts test that motivated the gate now mocks fetch directly instead of relying on the controller avoiding the call.
The catalog fetch now sits on GET /config/form-schema (previously gated behind enable: true), so a hanging upstream — not just a failing one — would block the admin settings page indefinitely, including the toggle that would disable the feature. Passes AbortSignal.timeout(5000) into the fetch call; the existing fail-open-to-free-text path already handles the resulting rejection. Generation's POST (openrouter-images-api.ts) is left as-is — it runs on the task queue with a legitimately long duration.
Saving imageGenerationOptions failed with "expected string, received null" once endpoint became conditionally hidden (provider !== 'custom'): the admin form's controlled inputs normally coerce a fetched null to '' just by rendering the field, but a hidden field is never rendered, so its raw null from the default config reached patchAndValid's real Zod validation unmodified — z.string().optional() accepts undefined but rejects null. Matches the existing mailOptions default pattern (from: '', smtp.user: '', resend.apiKey: ''), which has never hit this because it never hides a field.
A stored config that ever held the seeded null default still round-trips null back on save, so rejecting null (previous round) locks out any existing installation, not just protects against garbage input. Accept null and transform it to '' instead. The naive `v ?? ''` approach breaks partial-patch merging: it turns an *omitted* field into '' too, not just an explicit null, which clobbers a previously-configured endpoint/apiKey/model on any patch that doesn't re-send them, and lets encryptObject encrypt a blank apiKey into ciphertext. Caught by the existing merge-preservation and round-trip tests before commit. Fixed by only coercing an explicit null, leaving an absent key as undefined, and widening removeEmptyEncryptedFields to treat null the same as ''. Also fixes isRequired() in configs.dsl.util.ts, which didn't see through the ZodPipe a .transform() wraps a field in, so apiKey/endpoint/model would have shown an incorrect required-asterisk in the admin form.
Prompt-drafting uses the text AI provider (getWriterModel()), configured separately from image generation. A user with only image generation set up got a generic toast and an empty prompt box with no indication that a different config section was the problem. Threads the backend's AI_NOT_ENABLED code through instead of matching on the error message: admin's requestJson previously discarded the error envelope's code when throwing, so add ApiRequestError to carry it. The drawer now shows a persistent inline explanation with a link to Settings -> AI, leaving the textarea and preset chips untouched so writing a prompt by hand still works with no text provider configured.
The drawer showed a style preset and a prompt textarea as sibling controls, forcing the same decision twice. Preset is now the default mode (no textarea); "View/Edit Prompt" compiles once and hands off to manual editing. Compilation moved into the image-generation task handler so the enqueue endpoint still returns immediately, and a failed compile degrades to a title + hardConstraints prompt instead of failing the whole generation.
hardConstraints is a rejection list, not a description — concatenating it with the raw article title produced a prompt with nothing to draw and a Chinese title an image model can't use, so Gemini returned text instead of an image. Add a self-contained fallbackPrompt per preset carrying the positive style description, and use that instead when compilation fails.
The task-status query never refetched while the tab was backgrounded (refetchIntervalInBackground defaults to false), so the drawer could sit on a stale snapshot forever once the tab lost visibility. Separately, the server broadcasts a task's terminal status and its result as two socket phases, so a completed snapshot can arrive with no result attached yet — the client was treating that gap as a hard "missing_url" failure instead of waiting for the follow-up update. Together these meant a successful generation could complete server-side with nothing ever appearing in the candidates panel.
SafeDep Report Summary
No dependency changes detected. Nothing to scan. This report is generated by SafeDep Github App |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 222cae1f1c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+321
to
+325
| const chunks: Buffer[] = [] | ||
| for await (const chunk of file.file) { | ||
| chunks.push(chunk) | ||
| } | ||
| const buffer = Buffer.concat(chunks) |
There was a problem hiding this comment.
When local storage is used with type=file, icon, or avatar, maxFileSize is undefined and this loop now accumulates the entire multipart body in memory before writing it. These uploads previously streamed directly to disk, so a legitimately large attachment can now exhaust the Node heap and terminate the server; retain the streaming path for local uploads or impose a bounded size before buffering.
Useful? React with 👍 / 👎.
Comment on lines
+222
to
+225
| mimeType: | ||
| typeof item.media_type === 'string' && item.media_type.length > 0 | ||
| ? item.media_type | ||
| : DEFAULT_IMAGE_MIME_TYPE, |
There was a problem hiding this comment.
When a valid response omits the optional media_type field and the requested output format is JPEG or WebP, this fallback labels the returned bytes as PNG. AiImageService subsequently derives both the filename extension and upload Content-Type from this value, so non-PNG generations are persisted and served with incorrect metadata; derive the fallback MIME type from the effective outputFormat instead.
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI 封面图生成：从配置、运行时适配、任务队列到后台抽屉的完整链路。
Summary
imageGenerationOptions配置块（provider / apiKey / endpoint / model），customprovider 强制要求 endpoint；修复 configs DSL.default()元信息丢失与加密字段的null处理。IImageRuntime适配器 + OpenRouter 专用 Images API 传输层，按模型 capability 过滤参数，5s 超时约束模型目录拉取，解析失败即显式报错而非静默降级。signal-geometry封面风格预设，服务端编译 prompt；预设模式下无需前端填写 prompt，降级路径也有真实的兜底 prompt。ai:image-generation任务类型接入现有 AI task queue，含控制器、views 与 faux e2e。generate_imageagent tool；缺少文本 AI provider 时给出明确提示。FileService.uploadBuffer，打破AiImageModule -> FileModule -> CommentModule循环依赖。Test plan
pnpm -C apps/core test— ai-image service / catalog / param-mapping / runtime adapter / openrouter transport / dto、configs dsl 与 service、file controller 与 service、ai-task service、prompts schema 回归。pnpm -C apps/core test src/modules/ai/ai-image.faux.e2e.spec.ts— 生成链路 faux e2e。pnpm -C apps/admin test—use-cover-generation、api/ai-image、api/http、image-tools。无 schema 变更，不涉及数据库迁移。