The online racing simulator
Manually editing cfg.txt resulted in invalid LFS-encoded string
Context (somewhat irrelevant to the bug report): I have an InSim application written in C#, using InSimDotNet library. InSimDotNet offers classes that help convert LFS-encoded bytes into Unicode strings and back. I wanted to have a way of confirming that decoding and encoding operations are lossless, thus wrote a class called 'LfsEncodingChecker': a class that does a round-trip, and checks if input matches round-tripped data. For example: GetString() takes in bytes and gives a unicode string, then I use the string and call GetBytes() on it, and check if the original input bytes match the bytes generated from GetBytes(). Same check applies for GetBytes() function. In case of a data loss, it logs a warning message. I saw this in my logs: (explanation below)

[2025-10-18 02:06:51 INF] [ModularInSim.Program] --- Started ---
[2025-10-18 15:30:26 WRN] [ModularInSim.PortableEncoding.LfsEncodingChecker] GetString(): Round-trip data integrity check failed.
bytes1 '5E4AEFBFBD4F4BEFBFBD2045706963202847325E4C20636F6E6E656374656420284570696370726F657329'
bytes2 '5E4A3FBD4F4B3FBD2045706963202847325E4C20636F6E6E656374656420284570696370726F657329'
result '^J?スOK?ス Epic (G2^L connected (Epicproes)'
at System.Environment.get_StackTrace()
at ModularInSim.PortableEncoding.LfsEncodingChecker.GetString(Byte[], Int32, Int32)
at InSimDotNet.Packets.IS_MSO..ctor(Byte[])
[TRUNCATED]
[2025-10-18 15:30:26 WRN] [ModularInSim.PortableEncoding.LfsEncodingChecker] GetString(): Round-trip data integrity check failed.
bytes1 '5E4A5E33EFBFBD4F5E374BEFBFBD204570696320284732'
bytes2 '5E4A5E333FBD4F5E374B3FBD204570696320284732'
result '^J^3?スO^7K?ス Epic (G2'
at System.Environment.get_StackTrace()
at ModularInSim.PortableEncoding.LfsEncodingChecker.GetString(Byte[], Int32, Int32)
at InSimDotNet.Packets.IS_NCN..ctor(Byte[])
[TRUNCATED]
[2025-10-18 15:30:35 WRN] [ModularInSim.PortableEncoding.LfsEncodingChecker] GetString(): Round-trip data integrity check failed.
bytes1 '5E4AEFBFBD4F4BEFBFBD2045706963202847325E4C20646973636F6E6E656374656420284570696370726F657329'
bytes2 '5E4A3FBD4F4B3FBD2045706963202847325E4C20646973636F6E6E656374656420284570696370726F657329'
result '^J?スOK?ス Epic (G2^L disconnected (Epicproes)'
at System.Environment.get_StackTrace()
at ModularInSim.PortableEncoding.LfsEncodingChecker.GetString(Byte[], Int32, Int32)
at InSimDotNet.Packets.IS_MSO..ctor(Byte[])
[TRUNCATED]

Explanation: My checker worked as intended but the server sent me incorrect LFS-encoded bytes, which subsequently made the checker warn for a different reason. A player named Epicproes had a nickname ("「OK」 Epic (G29)") that contained Japanese (CP932) characters '「' and '」' which should have been sent as '\xA2' and '\xA3' respectively but they arrived as "\xEF\xBF\xBD".



Here are different representations of what I got and what the server should have sent:


Hex:
"5E4A5E33 EFBFBD 4F5E374B EFBFBD 20457069632028473200" (received) (24 bytes)
"5E4A5E33 A2 4F5E374B A3 204570696320284732392900" (expected) (24 bytes)

String:
"^J^3\xEF\xBF\xBDO^7K\xEF\xBF\xBD Epic (G2" (received)
"^J^3\xA2O^7K\xA3 Epic (G29)" (expected)

Unicode:
"^J^3�O^7K� Epic (G2" (received)
"^J^3「O^7K」 Epic (G29)" (expected)



Note: In all cases, nickname was truncated by the server to fit in 24 bytes with a null terminator. This is why the nickname ends with "(G2" instead of "(G29)".


I asked ChatGPT about this mysterious EF BF BD sequence, and it seems to be UTF-8 encoded "replacement character" represented as '�'.



I used Epicproes'es nickname and tried reproducing the same issue but InSim was working as intended. I tried locally (client), on a temporary server, and on the same couple (insim server & lfs server, though I had restarted InSim earlier for maintainance, which might have reset things on server side, I don't know) but couldn't reproduce it.
Attached images
epicproes_nickname.png
representations.png
chatgpt_EFBFBD.png
insim_working_as_intended.png
LFS doesn't do any kind of conversions like that, and this forum is purely about LFS bugs.

Please can you discuss on an InSimDotNet thread?

There was something similar recently. https://www.lfs.net/forum/post/2120244#post2120244

This forum section is purely to let me know about LFS bugs, for me to fix. But people are reporting issues with text strings that other programs have already converted.
Can you please check the LFSBRASIL server October 18th log?


The characters '�' perfectly match the EFBFBD sequence in Latin 1. His nickname is truncated to fit 24 bytes at some point, which reflects on the connect and disconnect messages; nickname is abruptly cut at "(G2" instead of being "(G29)".
Attached images
lfsbrasil_log_oct18.png
I am serious busy working 12 to 14 hour days trying to finish the new version - and nothing has changed in LFS code pages in recent times.

So, please. before I look into it at all can you have a look to see DIRECTLY the output from LFS? Special Japanese characters are output as double-byte characters from Windows code page 932

LFS.exe has absolutely no knowledge or implementation of anything to do with UTF-8 etc.

It only knows single byte ASCII and code pages (including double-byte code pages for Chinese Japanese and Korean). It does not do UTF-8 or any kind of UTF.
I have talked with Epicproes and issue became appearent. Long story short, he edited the cfg.txt file and saved it with UTF-8. At some point, Japanese characters A2 and A3 of the 'Ply Name' both turned into �, which ended up being encoded as EFBFBD.

In Japanese code page, LFS displays EFBF sequence as a square (no match) and BD as ス.


I had an assumption in my code that LFS wouldn't send invalid LFS-encoded bytes, which end up falling back to ? on the decoder, thus causing a round-trip data loss. This is my problem to solve. Apps in general should not rely on any external input being valid.

Editing cfg.txt is just one way of inserting invalid LFS-encoded strings into the game. Any code that runs locally can be altered. Any packets that get sent can be modified on the fly. Bad input can reach server side one way or another.

Ideally, on the server side, there should be checks/sanitizations at each input point, but... I assume that would be too much of a work for such a simple mistake that originated from cfg.txt being edited manually that led to a nickname to be an invalid LFS-encoded string.

I am hesitant to make suggestions but I will say, simply checking/sanitizing the cfg.txt 'Ply Name' should be a good start (and probably enough) and we can accept the fact that any LFS-encoded string can be improperly encoded (possibly by someone messing with memory and/or packets etc.). Maybe the 'Ply Name' can be passed through the clipboard paste logic that falls back to ? in case of unknown characters.

Extra: I searched the logs ('�') and found another person with EFBFBD, so this issue is rare but not one instance only.
Attached images
epicproes_broken_nickname.png
Thanks for the further research and explanation.

I think I'll just leave this thread as not "answered" so I can have a look when I have some time.
Sorry to bump the thread. I initially thought LFS server converted a totally valid LFS-encoded nickname into an invalid one. I updated the title to match.

If you are not going to implement server-side checks (which I personally find it to be an overkill in this case), you probably shouldn't even bother implementing client-side checks unless you want to. Sure, invalid characters get displayed as squares and whatnot, but it doesn't cause any bigger issues than that on LFS side, as far as I can tell.

Without server-side checks, servers and InSim apps can still get invalid strings by other means, which kind of defeats the purpose. InSim developers should be aware of the possibility of receiving invalid strings. Might as well not bother with client-side checks for that reason. Your decision.

FGED GREDG RDFGDR GSFDG