How to enable Chinese language support for searching?

[ly55521]

请问怎么让其支持中文搜索？
How to enable Chinese language support for searching?

[ryanlelek]

Hello, to help us out with this feature request, please provide:

• Example content / page with Chinese language
• Search query string(s) that should match the above content

[ly55521]

Hello, to help us out with this feature request, please provide:

• Example content / page with Chinese language
• Search query string(s) that should match the above content

搜索的关键字（Search keywords）：接口、注意、青龙、直接操作
页面内容如下（UTF-8 Page content）：
在青龙面板中，启用或禁用环境变量并非通过Python脚本直接操作，而是通过调用青龙面板提供的API接口来实现。以下是一个基本的示例，说明如何使用Python脚本来调用青龙面板的API接口以启用或禁用环境变量。请注意，这需要青龙面板已经配置好了API密钥，并且API接口支持这些操作。

[ly55521]

Hello, to help us out with this feature request, please provide:

您好，为了帮助我们解决此功能请求，请提供：

• Example content / page with Chinese language
• 中文示例内容/页面
• Search query string(s) that should match the above content
• 搜索应与上述内容匹配的查询字符串

Have you made any progress?

[zhouslin]

修改 app/core 中lunr.js
import lunr_ru from 'lunr-languages/lunr.zh.js';
导入中文语言包

function getLunr(config) {
if (instance === null) {
instance = lunr;
lunr_stemmer(instance);
lunr_multi(instance);
lunr_tinyseg(instance);
config.searchExtraLanguages.forEach((lang) => {
if (lang === 'zh') {
lunr_ru(instance);
}
});
}
return instance;
}

修改完后还有个bug
修改page.js

const file = await fs.readFile(contentDir+filePath);

这里路径源码是读不到的加上contentDir

就能中文搜索了。

[ly55521]

修改 app/core 中lunr.js import lunr_ru from 'lunr-languages/lunr.zh.js'; 导入中文语言包

function getLunr(config) { if (instance === null) { instance = lunr; lunr_stemmer(instance); lunr_multi(instance); lunr_tinyseg(instance); config.searchExtraLanguages.forEach((lang) => { if (lang === 'zh') { lunr_ru(instance); } }); } return instance; }

修改完后还有个bug 修改page.js

const file = await fs.readFile(contentDir+filePath);

这里路径源码是读不到的加上contentDir

就能中文搜索了。

试了下编译报错，是要安装中文依赖？？？

[ly55521]

修改 app/core 中lunr.js import lunr_ru from 'lunr-languages/lunr.zh.js'; 导入中文语言包

function getLunr(config) { if (instance === null) { instance = lunr; lunr_stemmer(instance); lunr_multi(instance); lunr_tinyseg(instance); config.searchExtraLanguages.forEach((lang) => { if (lang === 'zh') { lunr_ru(instance); } }); } return instance; }

修改完后还有个bug 修改page.js

const file = await fs.readFile(contentDir+filePath);

这里路径源码是读不到的加上contentDir

就能中文搜索了。

node:internal/modules/esm/resolve:844
throw new ERR_MODULE_NOT_FOUND(packageName, fileURLToPath(base), null);
^

Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'lunr-languages' imported from /opt/raneto/app/core/lunr.js
at packageResolve (node:internal/modules/esm/resolve:844:9)
at moduleResolve (node:internal/modules/esm/resolve:901:20)
at defaultResolve (node:internal/modules/esm/resolve:1131:11)
at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:390:12)
at ModuleLoader.resolve (node:internal/modules/esm/loader:359:25)
at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:234:38)
at ModuleWrap. (node:internal/modules/esm/module_job:85:39)
at link (node:internal/modules/esm/module_job:84:36) {
code: 'ERR_MODULE_NOT_FOUND'
}

[zhouslin]

npm install lunr-languages

安装 lunr-languages

[xhj967]

请问现在可以支持中文搜索了吗，我把config配置文件改为zh，好像不生效
have it been support chinese Language searches？I modify config file ‘en’ to ‘zh’，doesn't seem to be working

[xhj967]

修改 app/core 中lunr.js import lunr_ru from 'lunr-languages/lunr.zh.js'; 导入中文语言包

function getLunr(config) { if (instance === null) { instance = lunr; lunr_stemmer(instance); lunr_multi(instance); lunr_tinyseg(instance); config.searchExtraLanguages.forEach((lang) => { if (lang === 'zh') { lunr_ru(instance); } }); } return instance; }

修改完后还有个bug 修改page.js

const file = await fs.readFile(contentDir+filePath);

这里路径源码是读不到的加上contentDir

就能中文搜索了。

I tried this method, but it still doesn't support Chinese search. I want to ask if it is related to the localization configuration of the host
我按这个方法配置了依然没有成功，我想请问一下是否跟主机系统的本地化配置也有关系？
[xu@iZf8z4cfpxgbggro9k9blwZ ~]$ sudo locale LANG=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL= [xu@iZf8z4cfpxgbggro9k9blwZ ~]$

[asano69]

The same search issue occurs with Japanese text as well.
Example sentence to be searched:

連続性の原理は『客観化された可謬主義』の観念である。なぜなら、可謬主義はわれわれの知識が決して絶対的なものではなく、いわばつねに不確実性と不確定性の連続体の中を泳いでいるという教説だからである。そして連続性の教説は、すべての事物がそのように連続体の中を泳いでいるということである。

Example search terms: 連続性 / 原理 / 可謬主義

Searching with these terms fails to return the sentence, but searching for the entire quoted passage at once does surface it in the results. This indicates that the quoted block is being indexed as one single large "word."

Text written in CJK characters — similarly to Classical Greek or Classical Sanskrit — has no spaces between words. Therefore, full-text search in the same manner as modern Indo-European languages requires a morphological analysis system (i.e., a custom tokenizer).

A simpler approach is to mechanically split the text into every 3-character sequence (e.g., 連続性, 続性の, 性の原, …), build a trigram FTS index, and then enable prefix-match searches against the FTS table using 1- or 2-character queries (e.g., 連%, 連続%, 原理%). I wonder how other applications handle this implementation.

[asano69]

Personally, I think that in most cases, I wouldn’t have much trouble even without full-text search, as long as I can search titles using regular expressions. However, there also seem to be internationally compatible full-text search libraries such as FlexSearch and Fuse.js.