[Bug] 本地导出为乱码，导入失败（Title: [Bug] The local export is garbled and the import fails.） #3562
Comments
Title: [Bug] The local export is garbled and the import fails. |
taurusduan
changed the title
|
It seems that only the client side has this issue. |
|
this issue related to #3395 |
|
+1，在 macOS 下的客户端也有这个问题。 |
+1, the client under macOS also has this problem. |
|
+1，希望能尽快修复，否则我只能在一台设备上使用 |
+1, hope it gets fixed soon otherwise I can only use it on one device |
这是同一个bug。问题在于这里转换text到uint8array的方式是错误的， 示例 text = "你好世界hellworld"
console.log(new Uint8Array([...text].map((c) => c.charCodeAt(0))))
// Uint8Array(13) [96, 125, 22, 76, 104, 101, 108, 108, 119, 111, 114, 108, 100]
console.log(new Uint32Array([...text].map((c) => c.charCodeAt(0))))
// Uint32Array(13) [20320, 22909, 19990, 30028, 104, 101, 108, 108, 119, 111, 114, 108, 100]正确的做法是用TextEncoder let str = "你好世界";
let encoder = new TextEncoder();
let utf8Array = encoder.encode(str);
// Uint8Array(21) [228, 189, 160, 229, 165, 189, 228, 184, 150, 231, 149, 140, 104, 101, 108, 108, 119, 111, 114, 108, 100]这个bug应该是跨平台的，只要用户数据（聊天内容，对话标题..）包含中文（或者任何超出0~255编码范围的字符）就必定会触发。另外因为数据本身被截断了，意味着修复前导出的内容无法简单恢复（aka 数据永久丢失了） This is the same bug. The problem lies in the incorrect way of converting text to a Uint8Array. The Example: text = "你好世界hellworld"
console.log(new Uint8Array([...text].map((c) => c.charCodeAt(0))))
// Uint8Array(13) [96, 125, 22, 76, 104, 101, 108, 108, 119, 111, 114, 108, 100]
console.log(new Uint32Array([...text].map((c) => c.charCodeAt(0))))
// Uint32Array(13) [20320, 22909, 19990, 30028, 104, 101, 108, 108, 119, 111, 114, 108, 100]The correct approach is to use let str = "你好世界";
let encoder = new TextEncoder();
let utf8Array = encoder.encode(str);
// Uint8Array(21) [228, 189, 160, 229, 165, 189, 228, 184, 150, 231, 149, 140, 104, 101, 108, 108, 119, 111, 114, 108, 100]This bug is likely cross-platform, as long as user data (chat content, conversation titles, etc.) includes Chinese characters (or any characters beyond the 0 to 255 encoding range), it will trigger the bug. Additionally, since the data itself is truncated, it means that the exported content before the fix cannot be easily recovered (i.e., the data is permanently lost). |
|
omg you are not smart, bug its not in that things |
|
its because of this format are text not a json if you see my fork there is no problem whatever you export |
|
this a fucking correct https://github.com/H0llyW00dzZ/ChatGPT-Next-Web/blob/main/app/utils.ts#L40 |
|
also this issue is duplicated |
|
2.10.3导出中文还是有乱码 |
2.10.3 There are still garbled characters when exporting Chinese. |
|
v2.10.3 still has this issue |
|
When my mask is written in Chinese and I export it, it will not be able to be opened correctly due to encoding errors, and therefore cannot be imported. The same problem also occurs when exporting Chinese chat data. |
@Yidadaa I hope this problem can be fixed. Since v2.9.8, the correct json cannot be exported (will Import failed). If needed, I can provide my work in v2.9.7 and [v2.9.8](https ://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/releases/tag/v2.9.8) exported the same local data. |
|
fix by #3972 v2.11.3 我这里导出已经正常了，大家可以试一试 |
fix by #3972 v2.11.3 My export is now normal, you can give it a try |
I have already gone to use the chatbox |
|
同步了最新代码，全新从windows11 chrome导出，接着导入ios 17 的chrome的时候还是报【导入失败】 导出的文件大小 3112 KB，没有超过localStorage 5MB的限制 失败后，有个报错界面显示： 基本确定是 ios 的浏览器的问题，桌面端的firefox，chrome都没问题 |
The latest code has been synchronized and exported from Windows 11 chrome. When importing into ios 17 chrome, it still reports [Import failed] |
Describe the bug
A clear and concise description of what the bug is.
版本为：2.9.13 本地数据导出后为乱码，然后就无法导入了。（Version is: 2.9.13 After exporting the local data, it becomes garbled and cannot be imported.）
To Reproduce
Steps to reproduce the behavior:
导出导入（Export and import）
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Deployment
Desktop (please complete the following information):
windows11+谷歌浏览器（Windows 11 + Google Chrome）
Smartphone (please complete the following information):
Additional Logs
Add any logs about the problem here.
The text was updated successfully, but these errors were encountered: