Prevent vale style review from removing code blocks#701
Conversation
The LLM was making overly aggressive suggestions that removed entire code blocks from documentation. This adds multiple safeguards: - Add getLinesInCodeBlocks() to detect lines inside fenced code blocks - Skip any suggestion targeting lines inside code blocks - Reject suggestions where suggested text is <70% of original length - Strengthen LLM prompt with explicit "NEVER MODIFY CODE" instructions - Extract validation helpers to reduce cognitive complexity Fixes the issue where PR #700 removed 173 lines of code examples. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…tyle-review - Make isDestructiveChange use trimmed lengths to match wouldRemoveContent - Update prompt to reflect actual validation logic (allows 30% shorter text) This fixes false positives where whitespace-only changes were incorrectly rejected, and aligns the LLM prompt with the actual 70% length threshold in the code.
|
Bugbot Autofix resolved both bugs found in the latest run.
|
The function now tracks the opening backtick count and only closes a code block when encountering a line with at least as many backticks. This prevents nested code blocks (e.g., 4+ backticks wrapping 3-backtick blocks) from being incorrectly detected.
|
Bugbot Autofix resolved the bug found in the latest run.
|
- Enforce 3-space indentation limit for code fences per CommonMark spec - Validate closing fences only accept whitespace after backticks - Prevent misidentification of indented backticks as fence markers - Ensure non-whitespace content after backticks keeps fence open
|
Bugbot Autofix resolved both of the 2 bugs found in the latest run.
|
- Extract magic numbers to constants (MAX_FENCE_INDENT, TAB_STOP_WIDTH) - Reduce cognitive complexity in getLinesInCodeBlocks by extracting helper functions - Fix line formatting in isDestructiveChange Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The getLinesInCodeBlocks function now detects both backtick ( and ~~~ - Enhanced isValidClosingFence to match opening fence character - Updated getLinesInCodeBlocks to track fence type
|
Bugbot Autofix resolved the bug found in the latest run.
|
There was a problem hiding this comment.
Line 18 in 35be995
I think maybe it's missing parentheses around the rule:
(?s)([\s\S]*?)
or
(?s)*([\s\S]*?)
if there may be leading whitespace (I don't think that would be true for a code block)
There was a problem hiding this comment.
Claude Code:
The issue: Vale's BlockIgnores regex doesn't work for fenced code blocks because:
- For .md files: Vale parses Markdown natively and creates code scopes. IgnoredScopes = code
works for most content, BUT lines starting with # inside code blocks get misidentified as
headings. - For .mdx files: Vale doesn't have native MDX support - it treats MDX as plain text, so:
- No code scopes are created → IgnoredScopes = code doesn't help
- BlockIgnores regex is supposed to work but has known issues
(BlockIgnores not working for .mdx files errata-ai/vale#115, Vale is picking up errors for stuff in code blocks in GitHub markdown errata-ai/vale#387)
The bottom line: The commenter's suggestion about parentheses won't fix it - it's a Vale
limitation for MDX files. That's exactly why this PR's vale-style-review.ts script adds its own
getLinesInCodeBlocks() function to detect code blocks independently.
teallarson
left a comment
There was a problem hiding this comment.
I think this is just generating PRs, not direct commits to main, so seems safe to try out
|
That is correct, @teallarson :) |
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Summary
Changes
getLinesInCodeBlocks()function to detect lines inside fenced code blocksTest plan
pnpm vale:review --pr <number>on a PR with code blocks and verify no code is modified🤖 Generated with Claude Code
Note
Hardens the vale-style-review script to avoid destructive edits and never touch code blocks.
countLeadingWhitespace, `countLeadingFenceChars`, `isFenceCandidate`, `isValidClosingFence`, `getLinesInCodeBlocks`; skips any suggestions on those lineshasValidFields,wouldRemoveContent(≥70% length),isDestructiveChange(>50% length change) and applies them informatReviewCommentsMIN_SUGGESTED_LENGTH_RATIO,MAX_FENCE_INDENT,TAB_STOP_WIDTH; adjusts logging and filtering but preserves overall workflowWritten by Cursor Bugbot for commit 606e9b4. This will update automatically on new commits. Configure here.