Skip to content

fix(gemini): strip data URI prefix from PDF base64 data#14134

Open
404-Page-Found wants to merge 3 commits intoCherryHQ:mainfrom
404-Page-Found:hotfix/fix-gemini-pdf-base64
Open

fix(gemini): strip data URI prefix from PDF base64 data#14134
404-Page-Found wants to merge 3 commits intoCherryHQ:mainfrom
404-Page-Found:hotfix/fix-gemini-pdf-base64

Conversation

@404-Page-Found
Copy link
Copy Markdown
Contributor

What this PR does

Before this PR:
Uploading a PDF file and sending it to Gemini models failed with a "Base64 decoding failed" error because the payload incorrectly included the data:application/pdf;base64, prefix.

After this PR:
The application correctly strips the Data URI prefix from the base64 data for PDF files, ensuring only the raw base64 string is sent to the API.

Fixes #14097

Why we need it and why it was done in this way

Certain API proxies or specific model versions are stricter about the inline_data.data format and do not handle the Data URI prefix. Stripping it ensures compatibility across different providers while remaining compliant with the expected standard of raw base64 for these fields.

The following tradeoffs were made:

  • Added a simple regex check to strip the prefix if present. This adds a negligible overhead but significantly improves compatibility.

The following alternatives were considered:

  • Modifying the backend to handle the prefix, but since this is a frontend-to-SDK interaction, fixing it at the source of parameter preparation is more direct and robust.

Breaking changes

None.

Special notes for your reviewer

I've verified that �ase64File returns raw base64 in FileStorage.ts, but added this defensive check in ileProcessor.ts to ensure that any data passed to the AI SDK is clean.

Checklist

Release note

Fixed an issue where PDF file uploads to Gemini models failed due to incorrect Base64 encoding format.

Copy link
Copy Markdown
Collaborator

@kangfenmao kangfenmao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

分析

这个修复的意图是好的,但从代码分析来看,这段防御性代码 永远不会被触发

查看 FileStorage.ts:785-791

public base64File = async (_: Electron.IpcMainInvokeEvent, id: string) => {
  const filePath = path.join(this.storageDir, id)
  const buffer = await fs.promises.readFile(filePath)
  const base64 = buffer.toString('base64')  // ← 纯 base64,没有 data URI 前缀
  const mime = `application/${path.extname(filePath).slice(1)}`
  return { data: base64, mime }
}

buffer.toString('base64') 永远不会返回 data:...;base64, 前缀。PR 描述中也确认了这一点:

I've verified that base64File returns raw base64 in FileStorage.ts

因此这段 strip 逻辑是死代码,不会解决 #14097 的根本问题。

需要补充测试

如果确认此修改确实解决了问题,请提供以下服务商的实际测试结果:

  • Vertex AI — PDF 上传 + 发送消息
  • Gemini(官方 API) — PDF 上传 + 发送消息
  • CherryIN — PDF 上传 + 发送消息

建议

  1. 请进一步调查 #14097 的真正根因——如果 base64File 返回的确实是纯 base64,那问题可能出在其他环节
  2. 如果仍要保留防御性检查,建议加一个 logger.warn 记录异常情况,方便追踪问题来源

@404-Page-Found
Copy link
Copy Markdown
Contributor Author

Summary of changes:\n\n- Added defensive stripping and a warning in main: src/main/services/FileStorage.ts\n- Added defensive warning in renderer: src/renderer/src/aiCore/prepareParams/fileProcessor.ts\n- Added unit test: src/renderer/src/aiCore/prepareParams/tests/fileProcessor.test.ts\n\nVerification done locally:\n- Ran linter & formatter (passed)\n- Ran renderer unit tests (passed)\n- Ran main unit tests: some unrelated failures in src/main/services/agents/services/cherryclaw/tests/prompt.test.ts (appears unrelated to these changes)\n\nNotes for reviewer:\n- The strip logic is now executed in the main process where base64 originates, so it will no longer be dead code. Any unexpected occurrences will be visible in main logs via the new warning message.\n- If you still observe the original failure (#14097), please capture the main process logs and share the logged warning (if present) so we can trace where the data URI was introduced.\n\nIf you want, I can also open a follow-up PR to add an integration test that covers the end-to-end flow (write a file containing a data URI and verify the IPC returns stripped base64).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: fail to chat with pdf

2 participants