The online racing simulator
Uhh I love that

As for pyinsim itself, I'm currently experimenting with Struct objects to use the compiled format string and classes for the packets instead of UserDicts.

_PACKET_FORMAT = {
ISP_ISI : struct.Struct('4B2HBcH16s16s'),
ISP_VER : struct.Struct('4B8s6sH'),
ISP_TINY : struct.Struct('4B'),
ISP_SMALL : struct.Struct('4BI'),
ISP_STA : struct.Struct('4BfH10B6s2B'),
ISP_SCH : struct.Struct('8B'),
ISP_SFP : struct.Struct('4BH2B'),
ISP_SCC : struct.Struct('8B'),
ISP_CPP : struct.Struct('4B3i3H2Bf2H'),
ISP_ISM : struct.Struct('8B32s'),
ISP_MSO : struct.Struct('8B128s'),
ISP_III : struct.Struct('8B64s'),
ISP_MST : struct.Struct('4B64s'),
ISP_MTC : struct.Struct('8B64s'),
ISP_MOD : struct.Struct('4B4i'),
ISP_VTN : struct.Struct('8B'),
ISP_RST : struct.Struct('8B6s2B6H'),
ISP_NCN : struct.Struct('4B24s24s4B'),
ISP_CNL : struct.Struct('8B'),
ISP_CPR : struct.Struct('4B24s8s'),
ISP_NPL : struct.Struct('6BH24s8s4s16s8Bi4B'),
ISP_PLP : struct.Struct('4B'),
ISP_PLL : struct.Struct('4B'),
ISP_LAP : struct.Struct('4B2I2H4B'),
ISP_SPX : struct.Struct('4B2I4B'),
ISP_PIT : struct.Struct('4B2H8B2I'),
ISP_PSF : struct.Struct('4B2I'),
ISP_PLA : struct.Struct('8B'),
ISP_CCH : struct.Struct('8B'),
ISP_PEN : struct.Struct('8B'),
ISP_TOC : struct.Struct('8B'),
ISP_FLG : struct.Struct('8B'),
ISP_PFL : struct.Struct('4B2H'),
ISP_FIN : struct.Struct('4B2I4B2H'),
ISP_RES : struct.Struct('4B24s24s8s4s2I4B2H2BH'),
ISP_REO : struct.Struct('36B'),
ISP_NLP : struct.Struct('4B'),
ISP_MCI : struct.Struct('4B'),
ISP_MSX : struct.Struct('BBBB96s'),
ISP_MSL : struct.Struct('BBBB128s'),
ISP_CRS : struct.Struct('4B'),
ISP_BFN : struct.Struct('8B'),
ISP_AXI : struct.Struct('6BH32s'),
ISP_AXO : struct.Struct('4B'),
ISP_BTN : struct.Struct('12B240s'),
ISP_BTC : struct.Struct('8B'),
ISP_BTT : struct.Struct('8B96s'),
ISP_RIP : struct.Struct('8B2H64s'),
ISP_SSH : struct.Struct('8B32s'),
NODELAP : struct.Struct('2H2B'),
COMPCAR : struct.Struct('2H4B3i3Hh'),
OUTSIM : struct.Struct('I12f4i'),
OUTGAUGE : struct.Struct('I4sH2B12f16s16si'),
}

Classes look like this
class IS_TINY:
def __init__(self, ReqI = 0, SubT = None):
self._Size = 4
self._Type = ISP_TINY
self.ReqI = ReqI
self.SubT = SubT
def pack(self):
return _PACKET_FORMAT[self._Type].pack(self._Size, self._Type, self.ReqI, self.SubT)

and the __raisePacketEvent would simply do something like
(...)
packet = _PACKET_DEFINITIONS[packetType](*_PACKET_FORMAT[packetType].unpack(data)[2:])
(...)

Still working on the class definitions, don't know how it'll work out but I'll definitely give it a try. UserDicts are a good solution but they do feel kind of "hackish" to me
Quote from morpha :As for pyinsim itself, I'm currently experimenting with Struct objects to use the compiled format string and classes for the packets instead of UserDicts.

Awesome style mate, love it! Kinda looks like a php pack / unpack function to me!
Quote from morpha :Uhh I love that

As for pyinsim itself, I'm currently experimenting with Struct objects to use the compiled format string and classes for the packets instead of UserDicts.
(...)

Still working on the class definitions, don't know how it'll work out but I'll definitely give it a try. UserDicts are a good solution but they do feel kind of "hackish" to me

Well, keep me updated on your progress. While I made the decisions I made for pyinsim for good reasons, I am planning to start working on pyinsim 2.0 at some point, and I'm open to any and all suggestion and improvement ideas. While I'm still not sure having a separate class for each packet is ideal, I do think a lot of improvements (read simplification ) could be made to the current implementation.

At the moment I'm working on a few examples of making OutGauge and OutSim apps using the speedmeter module, which I'll upload once I get them finished. I've basically got them working, but they've exposed a couple of annoying issues with pyinsim, so I'll need to release another version of pyinsim before I can finish them.
By now I'm thinking your solution was in fact the better one, I had originally intended to use a base class providing a pack function like this:
def pack(self):
return _PACKET_FORMAT[self.Type].pack(*self.__dict__.values())

but unfortunately the dict is not in the order the variables have been assigned in (meaning it's effectively packing in random - or at least not the intended - order) and I don't see an efficient way to fix that.

Anyway, here're some new and some optimized functions:
_eatNullChars = lambda str_: str_[:str_.find('\0')]

Because not all strings are really NULL-terminated, such as the numberplate. Not sure about this one though, find might not be optimal.

def speedToMs(speed):
"""Convert speed to meters per second.

Args:
speed - The speed to convert.

Returns:
Metres per second.

"""
return speed / 327.68


def speedToKph(speed):
"""Convert speed to kilometers per hour.

Args:
speed - The speed to convert.

Returns:
Kilometers per hour.

"""
return speed / 91.02

def speedToMph(speed):
"""Convert speed to miles per hour.

Args:
speed - The speed to convert.

Returns:
Miles per hour.
"""
return speed / 146.486067

def directionToDegrees(direction):
"""Convert direction to degrees.

Args:
direction - The direction to convert.

Returns:
Degrees ranging from 0 to 180°.
"""
return direction / 182.04

def lengthToMetres(length):
"""Convert length to meters.

Args:
length - The length in LFS world coordinate units.

Returns:
The length in meters.
"""
return length / 65536.0

Direct speed conversion, added direction conversion and renamed distToMeters (metre is a unit of length so this seems more appropriate).

I'm still working on the classes and I'll post my version as soon as it's done but as I said, I do think your solution is the more efficient one.
Quote from morpha :By now I'm thinking your solution was in fact the better one, I had originally intended to use a base class providing a pack function like this:
def pack(self):
return _PACKET_FORMAT[self.Type].pack(*self.__dict__.values())

but unfortunately the dict is not in the order the variables have been assigned in (meaning it's effectively packing in random - or at least not the intended - order) and I don't see an efficient way to fix that.

Yeah, I had that problem too, it's to do with the way hashtables are stored in memory. I had to turn the dict back into a list by looping through the keys in the packet definition, and then use that to pack the struct.

Quote :
Anyway, here're some new and some optimized functions:
_eatNullChars = lambda str_: str_[:str_.find('\0')]

Because not all strings are really NULL-terminated, such as the numberplate. Not sure about this one though, find might not be optimal.

Well at the moment I just do
_eatNullChars = lambda str_: str_.rstrip('\000')

which seems pretty optimal to me really. Stuff like the numberplate is unpacked so seldom though I'm not sure a change here would really make much difference to be honest.

Quote :Direct speed conversion, added direction conversion and renamed distToMeters (metre is a unit of length so this seems more appropriate).

I'm still working on the classes and I'll post my version as soon as it's done but as I said, I do think your solution is the more efficient one.

Thanks for these.
Quote from DarkTimes :Well at the moment I just do
_eatNullChars = lambda str_: str_.rstrip('\000')

which seems pretty optimal to me really. Stuff like the numberplate is unpacked so seldom though I'm not sure a change here would really make much difference to be honest.

Your rstrip() may not iterate over as many chars as my find() does, but it returns a copy instead of a simple slice - eliminating the performance gain - and it's not as reliable.

My numberplate reads "42D" ingame, it's correctly unpacked using my implementation of _eatNullChars but yours returns unexpected additional characters including the terminating NUL.
Coming from LFS, the raw string looked like this
rawstr = '42D\000!!6&'

now because it's not NUL-padded, the characters following the terminating character are not guaranteed to be NUL, causing your function to effectively do nothing with this particular type of string.

>>> def stwtch():
tst = time.time()
tstr = 'abcdefghijklmnopqrstuvwxyz\000'
for i in xrange(0, 10000000, 1):
tstr.rstrip('\000')
return time.time() - tst

>>> print stwtch()
8.02099990845
>>> def stwtch():
tst = time.time()
tstr = 'abcdefghijklmnopqrstuvwxyz\000'
for i in xrange(0, 10000000, 1):
tstr.find('\000')
return time.time() - tst

>>> print stwtch()
7.92100000381

I win
OK - well I didn't realise it was a bug - of course I'll fix that. In terms of performance, a few milliseconds here or there isn't an issue with an InSim program really, as the actual amount of data being processed is quite small. If you just write all the InSim data to a binary file and then parse the packets out of that instead of from a socket, then pyinsim will read a whole 40 lap race in about a second, and I'd wager the majority of that is just basic file IO. The goal of pyinsim was always to be as easy to use for the developer as possible (hence the use of Python to begin with ), so my advice is to make it's as easy to use and fun to write as possible, and leave performance concerns to those silly C programmers.
My version finally reached a state in which a comparison was possible.
I expected it to consume a little more RAM than your version but less CPU cycles due to the precompiled formatting strings.
However, after a 2 lap race on AS5 (2 AI drivers) with ISF_MCI @ 50ms interval, your version had allocated a total of 6220K, my version with roughly 50 additional classes surprisingly allocated just 6036K. I guess the UserDict module adds a little unecessary - but in this case completely insignificant - overhead.

As for CPU usage, no accurately measureable difference, 0 to 2% for both versions in this particular test (receive all packets and just print "recv pack")

So far the only difference between our versions seems to be a few bytes of RAM and "packet.data" instead of "packet['data']".

By the way the _eatNullChars I posted is still not perfect, it's eating the last char of a string that doesn't contain any NULL chars (like a plate with 8 characters) because find() returns -1 instead of None if the string does not contain the needle. I don't think this is the best solution but it's the best I could come up with:
_eatNullChars = lambda str_: str_[:(str_.find('\0') if str_.find('\0') != -1 else None)]

Uploaded 1.1.2
Uploaded 1.1.2 to the first post.

Changes since 1.1.1
  • Added insimConnect() function, which provides a shorthand for creating and initialising an insim connection.
  • Renamed distToMeters() to lengthToMeters().
  • Added directionToDegrees(), radiansToDegrees() and radiansToRpm() functions.
  • Added checkAdmin() function which checks admin pass length.
  • Several small optimisations.
  • Fixed: bug with removing terminating line characters from C-style strings.
  • Fixed: checkIP now returns True if the IP string is 'localhost'.
  • Fixed: thread error bug when attempting to reconnect after the connection has been closed.
Thanks for morpha for suggestions and improvement ideas.
I decided to release my version despite it's dodgy state, still needs testing and OutSim and OutGauge are currently not working. I also removed the xml packet stuff but it wouldn't be hard to port if someone actually uses it

It's based on 1.1.2 so the newly added conversion functions are included, in optimized form (2.0 * math.pi replaced with 6.28318531 etc., there's no point in doing these calculations over and over again).
Also changed the spelling of some function names to proper, british English
Attached files
pyinsim_112_morpha.rar - 9.1 KB - 282 views
Cool thanks.
Uploaded pyinsim 1.1.3 to the first post.

Changed since 1.1.2
  • Added timeStr(time, hours=False), which is a shorthand for creating a formatted time string.
  • Added mpsToMph(mps) and mpsToKph(mps) functions, which converts meters per second to MPH and KPH.
  • Added distance(a=(0, 0, 0), b=(0, 0, 0)) method, which returns the distance between two points.
  • Fixed bug in WEA_* weather enum
  • Fixed bug with Tyres in IS_PIT packet
Quote from Dygear :Awesome style mate, love it! Kinda looks like a php pack / unpack function to me!

Haha, I'm two months late, but yes, I just re-read this. I wanted to say that pyinsim uses this extensively internally, in fact as far as I'm aware the whole struct.unpack() method is the only suck-free way to unpack binary formatted strings in Python. But anyway, well, I think we can all agree that dynamically typed languages are FTW! Pity those poor souls with their old-fashioned static languages and the hideous amounts of boilerplate they require.

*I love Python. I really, really love it.
I wrote a small example of creating an OutGauge app with wxPython. You need version 2.8.9.2 of wxPython or better for this to work. It's just a basic KPH gauge, but it demonstrates the principle.

import wx
import wx.lib.agw.speedmeter as sm
import math
import os
import pyinsim


HOST = 'localhost'
PORT = 30000
TIMEOUT = 10 # Seconds.


class OutGaugeApp(wx.Frame):
def __init__(self, *args, **kwds):
wx.Frame.__init__(self, *args, **kwds)

# Setup layout
self.CreateStatusBar()
sizer = wx.BoxSizer()
self.SetSizer(sizer)

# Create gauges.
self.createSpeedometer()
sizer.Add(self.speedometer, 1, wx.EXPAND)
self.closing = False

# Start OutGauge.
self.og = pyinsim.OutGauge(TIMEOUT)
self.og.bindTimeout(self.onTimeout)
self.og.bindError(self.onError)
self.og.bind(self.onOutGauge)

try:
self.og.connect(HOST, PORT)
except pyinsim.socket.error, err:
self.SetStatusText('OutGauge Error: %s' % err)

# Bind window events.
self.Bind(wx.EVT_CLOSE, self.onClose)

#Show window.
self.Center()
self.Show()

def onClose(self, evt):
self.closing = True
self.og.close()
self.Destroy()

def createSpeedometer(self):
extrastyle = sm.SM_DRAW_HAND | sm.SM_DRAW_SECTORS | sm.SM_DRAW_MIDDLE_TEXT | sm.SM_DRAW_SECONDARY_TICKS
self.speedometer = sm.SpeedMeter(self, extrastyle=extrastyle)
self.speedometer.SetAngleRange(-math.pi / 6, 7 * math.pi / 6)
intervals = range(0, 201, 20)
self.speedometer.SetIntervals(intervals)
colours = [wx.BLACK] * 10
self.speedometer.SetIntervalColours(colours)
ticks = [str(interval) for interval in intervals]
self.speedometer.SetTicks(ticks)
self.speedometer.SetTicksColour(wx.WHITE)
self.speedometer.SetNumberOfSecondaryTicks(5)
self.speedometer.SetTicksFont(wx.Font(7, wx.SWISS, wx.NORMAL, wx.NORMAL))
self.speedometer.SetMiddleText("Km/h")
self.speedometer.SetMiddleTextColour(wx.WHITE)
self.speedometer.SetMiddleTextFont(wx.Font(8, wx.SWISS, wx.NORMAL, wx.BOLD))
self.speedometer.SetHandColour(wx.Colour(255, 50, 0))
self.speedometer.DrawExternalArc(False)

def onTimeout(self, og, timeout):
if hasattr(self, 'closing') and not self.closing:
og.close()
self.SetStatusText('OutGauge: LFS timed out')

def onError(self, og, err):
og.close()
self.SetStatusText('OutGauge Error: %s' % err)

def onOutGauge(self, og, packet):
# Update the control.
kph = pyinsim.mpsToKph(packet['Speed'])
self.speedometer.SetSpeedValue(kph)


if __name__ == '__main__':
app = wx.App(redirect=True)
style = wx.DEFAULT_DIALOG_STYLE | wx.MINIMIZE_BOX
OutGaugeApp(None, wx.ID_ANY, 'OutGauge', size=(400, 400), style=style)
app.MainLoop()

If you don't like copying and pasting you can download the source below.
Attached images
outgauge.gif
Attached files
wxpython_outgauge.zip - 1.2 KB - 311 views
Uploaded 1.1.4 to the first post.

Changes since 1.1.3:
  • Removed LFSTime class. This has now been replaced by two new functions time(ms) and timeStr(ms).
  • Added metersToMiles(meters) and metersToKilometers(meters) functions.
  • Added Packet.insim() and _CarTrackPacket.insim(), which will return a reference to the InSim object which created the packet.
  • Renamed InSim.getName() to InSim.name().
  • Added several new examples which can be found in the /examples folder.
The new time(ms) function replaces the LFSTime class. It accepts the time in milliseconds as a parameter and returns a tuple containing the hours, minutes, seconds and thousandths of the time. For example:

h, m, s, t = pyinsim.time(milliseconds)

To convert the time into a formated string, use the timeStr(ms) function.

lapTimeStr = pyinsim.timeStr(milliseconds) # Returns 1:24.567 etc..

Hi.

First of all, very good job! I've abandoned PHPLFS and I adopted Pyinsim 4ever! :P

Well, I downloaded the last version 1.1.4, and when i've tested (for example) the buttons example, I got this traceback:

Traceback (most recent call last):
File "insim_buttons.py", line 40, in <module>
insim.send(pyinsim.ISP_ISI, Admin='Pass', IName='^3pyinsim', Flags=pyinsim.ISF_LOCAL)
File "/home/salvarez/_/_p/salvarez/pylfs/pyinsim_1.1.4/examples/pyinsim.py", line 1478, in send
self.sendB(Packet(packetType, **values).pack())
File "/home/salvarez/_/_p/salvarez/pylfs/pyinsim_1.1.4/examples/pyinsim.py", line 1748, in pack
return struct.pack(self.__packStr, [values.append(self[p[0]]) for p in _PACKET_DEFS[self['Type']]])
UnboundLocalError: local variable 'values' referenced before assignment

I've switched this commented lines:

def pack(self):
"""Pack the current packet values into a binary string, which can then
be sent to InSim.

Returns:
A binary formatted string.

"""
[COLOR=Red] return struct.pack(self.__packStr, [values.append(self[p[0]]) for p in _PACKET_DEFS[self['Type']]]) [/COLOR]

[COLOR=SeaGreen]# values = []
# for packetDef in _PACKET_DEFS[self['Type']]:
# values.append(self[packetDef[0]])
# return struct.pack(self.__packStr, *values)[/COLOR]

And all work now.

It's correct?

ty!
Yes, you're quite right, the comments should have been switched. In fact I did switch them before I built the distribution, but maybe I forgot to save it.

I've uploaded a new 1.1.4.1 with this error corrected.

Thanks for pointing this out!
Hi,
I downloaded last night version 1.1.4.1 of Pyinsim and installed Python 2.6.
Everything works fine and very nice to program .

I found a small bug in the lap times example script.
When i drive a lap the output to the console is ss:mm.xx incase of mm:ss.xx

example:
Lap: [NLR]Tim - FOX - 10:01.030 -- but it was a laptime of 1:10.030
Lap: [NLR]Tim - FOX - 4:01.910 -- but it was a laptime of 1:04.910
It seems that was the day for lots of stupid little errors sorry, I've uploaded a fixed version to the first post.
I'm working on a new version I think I'm going to designate pyinsim 1.5 (pyinsim 2.0 is going to be the one that supports Python 3.0). Basically I've finally given in and decided to do away with the whole packets are dictionaries thing, and now each packet has its own individual class. One very cool thing is that pyinsim already contains all the data for each packet, just in a different format, so I wrote some scripts that converted the old packet format into the new one, which means that after a couple of hours work I already have every packet converted, and can also be 95% confident that the packet definitions are bug free, as I am 95% certain that the existing definitions are bug free.

The new packet format allows me to do new things that weren't feasible before, such as proper support for arrays (so things like Tyres and vectors are now proper tuples), and also things like inline converting of times to string format and strings into unicode, so they no longer require separate function calls. That means that what was previously something like...

lapTime = pyinsim.timeStr(lap['LTime'])
playerName = pyinsim.strToUnicode(npl['PName'])

Could now be simplified to...

lapTime = lap.LTimeStr()
playerName = npl.PNameU() # U for Unicode

That being said the actual API is still in development, so how it actually works may change. One of the design philosophies of pyinsim is that it should hide as little from the developer as possible (nothing magic happens behind the scenes), so I need to be careful with any new additions like this that they are purely to make the API simpler and less annoying to use, and not just to placate lazy programmers (like me). I want programmers using pyinsim to have to think about InSim and program to it, and not just to some shiny API that does everything for you.

There are also several new things I want to add such as custom packet definitions, so instead of using pyinsim's IS_NPL packet, you will be able to create your own custom New PLayer packet which pyinsim will treat as if it were its own. I also plan to have some sort of underlying object persistence system, which is hard to explain, but will work like the ReqI does currently and return custom py objects to you in response to requests.

Well, as I say this is all in development, so it may all change, but so far it's all working out as intended. The most time consuming thing will be the documentation, as I want pyinsim to have complete __doc__ and HTML documentation for every packet and every possible value. Sadly it's hard to write a script that does that for you.

Sorry for the wall of text.
It sounds very nice We'll stay tuned!
I have uploaded a beta of pyinsim 1.5 to the first post. The module has undergone a great many changes and is not compatible with previous versions.

Changes Summery
  • Packets are no longer dictionaries and each packet now has its own unique class.
  • InSim now supports TCP and UDP.
  • New class OutReceiver() which handles UDP packets sent by LFS on a seperate port, for OutSim, OutGauge and InSim (MCI/NLP etc..).
  • New event system and events.
  • Support for custom packets.
  • Lots of small changes and tweaks.
Packets

Packets are no longer dictionaries, so you no longer access packet attributes using a dictionary key, but instead use the normal object syntax.

def versionCheck(insim, ver):
# It's a proper object! :)
if ver.InSimVer == pyinsim.INSIM_VERSION:
print 'InSim connected!'
else:
print 'Invalid InSim version'

InSim UDP

You can now connect to InSim in UDP mode, as well as with TCP. You can set the connection mode by passing either INSIM_UDP or INSIM_TCP into the InSim() constructor. The timeout specifies the number of seconds to wait before timing out in UDP mode. When a timeout occurs a EVT_TIMEOUT event is raised (see events below).

insim = pyinsim.InSim(pyinsim.INSIM_UDP, timeout=30.0)

OutReceiver

The old OutSim() and OutGauge() objects have been removed are are now replaced with the new OutReceiver(), which handles all packets arriving on a separate UDP port, including OutSim, OutGauge and MCI or NLP. Here is an example of OutGauge using the new system.

import pyinsim

def outgaugePacket(out, og):
# Check if the shift-light is on.
if og.DashLights & pyinsim.DL_SHIFT and og.ShowLights == 0:
print 'Shift-light on!'

# Create a new OutReceiver() for OutGauge with a timeout of 30 seconds.
out = pyinsim.OutReceiver(pyinsim.OUT_OUTGAUGE, 30.0)

# Bind the OutGauge packet event.
out.bind(pyinsim.ISP_OUTGAUGE, outgaugePacket)

# Start the receiver.
out.start('localhost', 30000)

# Block the thread until the connection times out.
out.run()

Events

The old bindThreadError() and bindConnectionLost() methods have been removed and replaced with a new event system. You bind pyinsim events now in the same method that you bind packet events.

import pyinsim

# EVT_CLOSED event handler.
def insimClosed(insim, reason):
if reason == pyinsim.CLOSE_REQUEST:
print 'InSim closed by a call to InSim.close()'
elif reason == pyinsim.CLOSE_LFS:
print 'InSim closed as LFS closed'

insim = pyinsim.InSim()

# Bind a pyinsim event.
insim.bind(pyinsim.EVT_CLOSE, insimClosed)

# Etc..

Currently the available events are as follows:
  • EVT_CLOSE - InSim connection has closed (contains close reason)
  • EVT_ERROR - An error has been raised on an internal thread (contains error message)
  • EVT_TIMEOUT - LFS has timed out (contains timeout seconds)
Download

Please see the first post for download information.
I finally got around to reading the documentation for my documentation generator, and realised that I've been doing it all wrong. So I've uploaded an improved version with much better documentation, and also some fixes for silly errors I made in the examples.

Edit: fixed some ****ing stupid bugs as well.
Uploaded 1.5.3, which has some small fixes. The only interesting thing is that I added back in the OutSim and OutGauge classes, but they are now just wrappers for the OutReceiver class. It's mainly a cosmetic thing, the functionality is exactly the same.
I must say DarkTimes, your work with pyinsim is inspiring and solely you are making me warm to the idea of developing in python. The simplicity of your API is fantastic
-
(DarkTimes) DELETED by DarkTimes

FGED GREDG RDFGDR GSFDG