Telescopetest-io: add AI content filtering#144
Telescopetest-io: add AI content filtering#144Judyzc wants to merge 10 commits intocloudflare:mainfrom
Conversation
| .replace(/<(script|style|noscript|head|template)[\s\S]*?<\/\1>/gi, '') | ||
| .replace(/<[^>]+>/g, ' ') | ||
| .replace(/&/g, '&') | ||
| .replace(/</g, '<') | ||
| .replace(/>/g, '>') | ||
| .replace(/"/g, '"') | ||
| .replace(/'/g, "'") | ||
| .replace(/ /g, ' ') | ||
| .replace(/&[a-z]+;/gi, ' ') | ||
| .replace(/\s+/g, ' ') |
There was a problem hiding this comment.
Hmm, I think this sort of replacement won't work across newlines, and is omitting valid escaped text.
We should most likely be parsing the HTML and extracting the text nodes (most likely via https://developer.mozilla.org/en-US/docs/Web/API/DOMParser/parseFromString) from the parsed document.
Also, do we need to extract text at all? Like, assuming the content scanner is an LLM capable of sifting through structured documents, it probably could be passed the HTML document as-is and make a determination on the content?
There was a problem hiding this comment.
I think the [\s\S] part of the regex allows it to work over newlines, shown here and through testing.
I don't think DOMParser works with Cloudflare workers, explained here, though I might be wrong. Cloudflare has its own HTMLRewriter tool I could use but that adds in streaming. There's also this third-party library linkedom I could try using, but what are your thoughts?
For needing to extract text, the LLM seems to be for conversation like strings: https://developers.cloudflare.com/workers-ai/models/llama-guard-3-8b/, so I haven't actually tested with just the HTML document. I can probably try this too though.
| .join(' ') | ||
| .replace(/\s+/g, ' ') | ||
| .trim() | ||
| .slice(0, 4000); |
There was a problem hiding this comment.
We're intentionally only scanning the first ~4k characters?
There was a problem hiding this comment.
We can probably increase this to ~100,000 chars b/c the model can take 131,072 tokens, but yeah we can definitely increase this.
Judyzc
changed the title
| signal: AbortSignal.timeout(10_000), | ||
| }); | ||
| const html = await response.text(); | ||
| return html |
There was a problem hiding this comment.
@Judyzc did you have any success with sending HTML to the agent here?
sergeychernyshev
added
the
ticket
…to find bad? sites
…t max only 2 AI calls, tinkered with ai-content-filtering
…G to IN_PROGRESS to be less confusing
3 participants
Related to #143. This PR sets up AI content filtering for telescopetest.io, as described in the issue.
content_ratingcolumn in D1 thetestsmetadata table. Auto-generated a migration file (0002) for this with Prisma by following the READMElib/ai/ai-content-rater.ts, which adds the functionrateUrlContent().rateUrlContent()is called on inupload.tsPOST endpoint withwaitUntil()and always returns eitherSAFEorUNSAFE. If this AI content check gets interrupted by user refresh, it can be called again by the telescopetest.io/results/[testId] page, which now polls (with the GET endpoint tests/[testId]/rating) and blocks displaying results until a rating is given.ENABLE_AI_RATING=falsein a.dev.varsfile as described in the README. Then, unsafe content will be displayed on the /results page with a flag.QUESTION/REQUEST:
misc: