Functionality ideas (large list of suggestions) :)

[dodleh]

ChatterUI is a powerful application that has lots of usability and features. Certainly, it is the most polished and better featured Android LLM interfaces out of the ones I tested. It has major advantages in speed, aesthetics and many other aspects.

I have a couple of suggestions I noticed are very good in apps for LLMs, some inspired by other apps, some of my own:

• ability to directly jump in the last chat, if the user prefers, on app cold start;
• ability to branch chat into another chat, after a specific point; (*)
• context length used (this is very useful to know if the discussion stays on track or how much space is left until it cannot continue);
• option to limit the discussion when the context is filled;
• ability to export as plain-text, to the clipboard, the content of a whole chat;
• ability to import/export the content of a whole chat,, as a json file or in other format;
• ability to reflect, if the user chooses to do so, what LLM was used in a chat;
• ability to keep track and load the same LLM used in a particular chat to continue a specific chat or choose another LLM;
• show, if the user chooses to do so, memory usage of the model (useful for debugging);
• implement a graceful exception flag if model cannot be loaded in the memory (now the app crashes);
• implement a setting to automatically replace the KV cache when there is a mismatch, instead of prompting the user;
• present option to change text size (eventually the font), other than through the global Android scaling and customization;
• create a minimal interface to edit a current colour scheme with color choices expressed in hex or decimals; (not essential);

A future ability would be to handle context length as high as 131072 (if enough memory is available), yet that is not essential, for the moment.

(*) The ability to branch a chat could be handled in the typical way or in a more interesting one, by concatenating all the user prompts up to that point into a single text text block, saving tokens and being much more effective (with the user being required only to state how to handle that block of text in his prompt). The typical way is much more easy to implement I admit, and would be a major progress, nonetheless. The typical and the complex, interesting approach (with concatenation), when both are available, could be decided by the user, in a prompt.

Thank you very much and I wish you all the best!

I have to say that your work and others really brings a whole new level of interaction and a great experience that would have been unheard of, in the past, I admire your work!