feat: remote functions cache API

[dummdidumm]

Adds a new remote functions cache API. Simple example:

/// file: src/routes/data.remote.js
import { query, command } from '$app/server';

export const getFastData = query(async () => {
	const { cache } = getRequestEvent();
	cache('100s');
	return { data: '...' };
});

export const updateData = command(async () => {
	// invalidates getFastData;
	// the next time someone requests it, it will be called again
	getFastData().invalidate();
});

For more info see the updated docs.

This is very WIP, and the adapter part isn't implemented yet (there are a few ways to approach it and we need to agree on the other APIs first). But it workds in dev and preview (public cache is implemented as runtime cache which very likely needs some more hardening).

TODOs/open questions:

• right now event.cache() is "last one wins" except for tags which are merged. After sitting with it for a while this is I think the most straightforward and understandeable solution, but we can also approach it differently.
  • there could be one entry for public and one for private, i.e. you can have both. Drawback is that you might accidentally cache something publicly that you want to keep private
  • other way around: as soon as something is declared private it cannot become public anymore. Drawback is that you might call a private remote function from a public remote function, being fully aware of that and you only use secure parts of the privately-cached remote function so you are limited by the framework's decision now
  • error if you have both private and public. My least favorite option because we should recover gracefully; if that's your favorite you'd rather go "use private and ignore public; maybe have a warning at dev time"
  • same options for ttl and stale
• is ttl and stale descriptive enough? Should it be maxAge and swr instead (closer to the web cache nomenclature)?
• how to best integrate this with adapters? either they provide a file with some exports which are like hooks which we call at specific points (setHeaders, invalidate etc) or we don't do anything and do this purely via headers, and adapters can check these headers and either do runtime cache based on it and/or add cdn cache headers (though maybe they have to clone the response then; not sure how much of an overhead that is and if that matters)
• this only works for remote functions right now, and it only works when you are calling them from the client. We could additionally have a SvelteKit-native runtime cache for public caching, and/or the adapter can hook into this to cache somewhere else than in memory (Vercel can use runtime cache, CF can use their cache, etc; i.e. this is related to the question above). This way we get more cache hits between client/server calls (or rather, we can get full page request cache this way, which we don't have at all right now).
• can this be enhanced in a way that this is usable for full page requests, too (e.g. inside handle hook?). Private cache doesn't make sense there at least. I'd say it's possible to implement and would be intuitive with this API (we can say "do this in handle or load", or "assuming you use remote functions only we take the lowest cache across all of them as the page cache", etc etc, many possibilities) but we should do that later and not bother with it now.

[changeset-bot]

🦋 Changeset detected

Latest commit: 7557320

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages

Name	Type
@sveltejs/kit	Minor
@sveltejs/adapter-node	Patch
@sveltejs/adapter-vercel	Patch
@sveltejs/adapter-netlify	Patch
@sveltejs/adapter-cloudflare	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[svelte-docs-bot]

Preview: https://svelte-dev-git-preview-kit-15678-svelte.vercel.app/

[elliott-with-the-longest-name-on-github]

I don't think the URL is the right cache key here... it should probably be the remote function key instead. Also, tags should be additive, not replace the default key.

[elliott-with-the-longest-name-on-github]

One thing I don't think is quite right here: Caching remote functions requires two coordinated layers of caching. One of them is a runtime cache at the server level, which needs to be used to cache direct-from-server remote function calls. This cache should be granular -- i.e. if I call getUser and getTeam and they have separate cache durations, that's fine -- they can be cached separately. The other is the "request-level" cache, which basically needs to take the lowest common denominator TTL and apply it to the request.

Overall, I really think SvelteKit core should not do anything with the information from the cache API -- instead, the adapters should do everything:

• Adapters should provide a get(key: string): Promise<string | undefined> function, which SvelteKit calls for every query
• Adapters should provide invalidate(key: string): Promise<void> and invalidateTag(tag: string): Promise<void> functions, which SvelteKit delegates to for
• Adapters should receive a Map<string, CacheInvocation> object that they can use to do... whatever they want. For example, the Vercel adapter would likely map over this and cache everything in the runtime cache, and, if the request is a remote request for a query endpoint, it would consolidate the runtime cache TTLs to find the soonest-to-be-invalidated entry and set the overall request cache time to that

[ottomated]

Should be a type like

`${number}d`  | `${number}h` | `${number}m` | `${number}s`  | `${number}ms`

[dummdidumm]

Ms/day don't really make sense imo but otherwise yes; on my list of todos once we agree on the API 👍

[ottomated]

Would cache('immutable') be an accepted option? i.e. no ttl, only gets invalidated manually

[dummdidumm]

Caching remote functions requires two coordinated layers of caching

Overall, I really think SvelteKit core should not do anything with the information from the cache API

Both these points are correct, and in fact right now this PR doesn't do anything in core with the public cache, this is all handled outside of the SvelteKit runtime (dev/preview for now, adapters are todo). We can discuss how exactly to do this once we agree on the user-facing API👍

[Rich-Harris]

Disorganised thoughts incoming:

• Should it be on getRequestEvent() or should it just be an import? What are the advantages/disadvantages? (One advantage of the import: there's less cost to growing the surface area, e.g. if we later add a refreshAfter('60s') API or whatever. Though a corresponding disadvantage is that imports are less discoverable than event.*)
• What does this API actually do? (I know I could read the code but you told us not to 😜) That influences API design choices like nomenclature. I assume that for private it's not setting actual HTTP cache control headers because then I don't think it would be possible to programmatically invalidate them?
• Assuming then that this populates an origin-scoped Cache object in the browser, presumably we can populate it with hydrated data?
• I would go with maxAge and staleWhileRevalidate — clearer than ttl and stale or swr. I think clarity > brevity in this case
• I don't think cache should accept string | CacheOptions — maxAge isn't an option, so it would be better to do cache(maxAge, options?)
• having both refresh() and invalidate() feels potentially confusing. I wonder if something like refresh({ force: true }) would be better
• 'last one wins' — I really don't think this is the right choice. I think that just like you can't do headers.set('cache-control', ...) multiple times today, you shouldn't be able to call cache(...) multiple times for a single remote function. The framework can't possibly divine the developer's intent; it should force them to be explicit about it
• Do we really need tags? It's something people ask for because it's what they're used to, but I think it's a bad solution to the problem. We've resisted the urge to support tag-based refreshes for good reasons (relating to type safety, bundling and so on) and I think the same thinking applies here. While we can't have an equivalent of requested, we could support cache.invalidate(myQuery) alongside myQuery(123).invalidate() and I bet that would cover all realistic use cases without sacrificing the aforementioned benefits. If I'm wrong, we can always add tags in future but my vote would be to leave them out until then
• Public and private caches are such different beasts that I'm not totally sure they should even share an API. For example what does it mean for me to do somePubliclyCachedQuery().invalidate()? I can't invalidate other people's browser storage. The public cache implies adapter-specific behaviour that could involve KV stores and semi-proprietary headers and credit cards (you often have to pay for this stuff) while the private cache is just a Cache object with the same behaviour everywhere. I feel like mixing them up in a single function is going to lead to a lot of confusion. (Aside: I think 'server' vs 'client' caching might be a better way of thinking about this, rather than 'public' vs 'private', which a) is very HTTP-specific and b) doesn't really communicate that you're dealing with entirely distinct mechanisms)
• 'can this be enhanced in a way that this is usable for full page requests' — I don't think it makes sense for the client cache, because what would it mean? That would only make sense in the context of a service worker. So the question is whether exposing the adapter-provided server caching mechanism provides enough value over current approaches (i.e. cache-control, ISR, etc) to justify the cost of making the API less focused

[Rich-Harris]

A third option, between an import and a property of RequestEvent — methods on query:

import { query, command } from '$app/server';

export const getFastData = query(async () => {
-	const { cache } = getRequestEvent();
-	cache('100s');
+	query.cache('100s');
	return { data: '...' };
});

This solves all the problems at once — we don't need to worry about cluttering RequestEvent, query.* is discoverable as we add new stuff, and it neatly explains why you can only call the method inside a query (whereas anything on RequestEvent is a weird forbidden appendage in any other context)

[Rich-Harris]

While we're thinking about a cache API, I'd love for us to think about bfcache — this is one of the very few genuine weaknesses of SPA navigation relative to The Olde Web.

Even then, traditional bfcache is a bit ropey. If I click into one of my pending tasks, mark it as complete, and then navigate back to my task list, the completed task is still visible in the pending list. You see this bug a lot with GitHub. But the back navigation sure is fast!

We can offer the best of both worlds: we can keep the list of pending tasks cached, but also invalidate it if one gets completed. That way, when I navigate back without completing a task it can happen instantly, but if I did update the list then SvelteKit knows that it has to fetch fresh data.

This might be controversial but I think this behaviour should be opt-out rather than in. Something like this, perhaps:

export const getPendingTasks = query(async () => {
  // disable bfcache altogether for this query
  query.bfcache(false);

  return await db.select().from(task).where(...);
});

export const getPendingTasks = query(async () => {
  // enable bfcache, but configure it
  query.bfcache({
    limit: 5, // evict if we're more than 5 navigations away from having used this query
    maxAge: '10m' // evict after 10 minutes whatever happens
  });

  return await db.select().from(task).where(...);
});

Maybe bfcache is the wrong name since it's not what's actually happening.

Implementation-wise, we would continue to remove queries from memory when they were offscreen, but instead of discarding their data altogether we would put them in a session-scoped Cache. If a query was refreshed in a command/form handler, and wasn't visible onscreen but was in the session-scoped cache, it would be evicted. requested would include queries that lived in the cache, in addition to ones currently onscreen.

As with the prerender Cache, we would delete any orphaned caches on startup.

This would only apply during popstate navigation — everything else would result in fresh data being fetched (modulo whatever caching API we land on here). I think this would all feel pretty nice.