Context (somewhat irrelevant to the bug report): I have an InSim application written in C#, using InSimDotNet library. InSimDotNet offers classes that help convert LFS-encoded bytes into Unicode strings and back. I wanted to have a way of confirming that decoding and encoding operations are lossless, thus wrote a class called 'LfsEncodingChecker': a class that does a round-trip, and checks if input matches round-tripped data. For example: GetString() takes in bytes and gives a unicode string, then I use the string and call GetBytes() on it, and check if the original input bytes match the bytes generated from GetBytes(). Same check applies for GetBytes() function. In case of a data loss, it logs a warning message. I saw this in my logs: (explanation below)

Explanation: My checker worked as intended but the server sent me incorrect LFS-encoded bytes, which subsequently made the checker warn for a different reason. A player named Epicproes had a nickname ("｢OK｣ Epic (G29)") that contained Japanese (CP932) characters '｢' and '｣' which should have been sent as '\xA2' and '\xA3' respectively but they arrived as "\xEF\xBF\xBD".



Here are different representations of what I got and what the server should have sent:




Note: In all cases, nickname was truncated by the server to fit in 24 bytes with a null terminator. This is why the nickname ends with "(G2" instead of "(G29)".


I asked ChatGPT about this mysterious EF BF BD sequence, and it seems to be UTF-8 encoded "replacement character" represented as '�'.



I used Epicproes'es nickname and tried reproducing the same issue but InSim was working as intended. I tried locally (client), on a temporary server, and on the same couple (insim server & lfs server, though I had restarted InSim earlier for maintainance, which might have reset things on server side, I don't know) but couldn't reproduce it.

LFS doesn't do any kind of conversions like that, and this forum is purely about LFS bugs.

Please can you discuss on an InSimDotNet thread?

There was something similar recently. https://www.lfs.net/forum/post/2120244#post2120244

This forum section is purely to let me know about LFS bugs, for me to fix. But people are reporting issues with text strings that other programs have already converted.

Can you please check the LFSBRASIL server October 18th log?


The characters '�' perfectly match the EFBFBD sequence in Latin 1. His nickname is truncated to fit 24 bytes at some point, which reflects on the connect and disconnect messages; nickname is abruptly cut at "(G2" instead of being "(G29)".

I am serious busy working 12 to 14 hour days trying to finish the new version - and nothing has changed in LFS code pages in recent times.

So, please. before I look into it at all can you have a look to see DIRECTLY the output from LFS? Special Japanese characters are output as double-byte characters from Windows code page 932

LFS.exe has absolutely no knowledge or implementation of anything to do with UTF-8 etc.

It only knows single byte ASCII and code pages (including double-byte code pages for Chinese Japanese and Korean). It does not do UTF-8 or any kind of UTF.