Was ist OSER?

OSER ist ein objektorientierter Serialisierer und Deserialisierer für binäre Daten, der in Python implementiert ist. Mit OSER können binäre Daten auf einfache Art und Weise menschenleserlich aufbereitet und verändert werden.

OSER ist ein Baukasten bestehend aus:

Anwendungen

Abstraktion, Analyse und Modifikation von Netzwerkprotokollen, EEPROM-Inhalten, Flash-Inhalten, RAM-Inhalten oder binären Dateien zu Test-, Debug- und Entwicklungszwecken.

Vorteile
Dokumentation

Die Dokumentation befindet sich hier.

Beispiele

You may want to create a custom network protocol for you device or software. Because the message is transmitted via any serial connection it starts with a magic number 0x23. This is a U8.

This magic number is followed by the destination address and the source address. Both are U16.

Additionally there is a service-id as U32.

The service-id is followed by the total payload length as U16 without crc. This way the transport-layer must not be aware of the protocol itself.

The payload consists of a type, a length and the data itself. The type has symbolic names:

  • 0: string
  • 1: list of 16-bit integers
  • 2: list of floats

A frame ends with a 32-bit crc with polynomial 0x1EDC6F41.

All values are transmitted in big endian.

The following abstraction realizies a frame for this protocol:

>>> from oser import ByteStruct
>>> from oser import Constant
>>> from oser import UBInt8, BFloat, UBInt16, UBInt32
>>> from oser import Enum
>>> from oser import Array
>>> from oser import Switch
>>> from oser import String
>>> from oser import CRCB32
>>> from oser import to_hex

>>> class SerialProtocol(ByteStruct):
...     def __init__(self, *args, **kwargs):
...         ByteStruct.__init__(self, *args, **kwargs)
...
...         self.start_of_frame = Constant(UBInt8, 0x23)
...         self.destination = UBInt16(0)
...         self.source = UBInt16(0)
...         self.service_id = UBInt32(0)
...         self.payload_length = UBInt16(0)
...         self.payload = Payload()
...         self.crc = CRCB32(strict=True, polynomial=0x1EDC6F41)
...
...     def encode(self, fullData="", contextData=""):
...         self.payload_length.set(self.payload.size())
...         return super(SerialProtocol, self).encode(fullData, contextData)
...
>>> class Payload(ByteStruct):
...     def __init__(self, *args, **kwargs):
...         ByteStruct.__init__(self, *args, **kwargs)
...
...         self.type = Enum(UBInt32,
...                          values={
...                              "string" : 0,
...                              "u16list" : 1,
...                              "f32list" : 2,
...                          }, value="string")
...         self.length = UBInt16(0)
...         self.data = Switch(condition=lambda self: self.type.get(),
...                            values={
...                                "string"  : String(length=lambda self: self.length.get()),
...                                "u16list" : Array(length=lambda self: self.length.get(), prototype=UBInt16),
...                                "f32list" : Array(length=lambda self: self.length.get(), prototype=BFloat),
...                            })
...
>>> frame = SerialProtocol()
>>> frame.service_id.set(1337)

>>> # string
... frame.payload.type.set("string")
>>> frame.payload.length.set(10)
>>> frame.payload.data.set("abcdefghij")

>>> binary = frame.encode()
>>> print frame
SerialProtocol():
    start_of_frame: 35 (UBInt8)
    destination: 0 (UBInt16)
    source: 0 (UBInt16)
    service_id: 1337 (UBInt32)
    payload_length: 16 (UBInt16)
    payload: Payload():
        type: 'string' (UBInt32)
        length: 10 (UBInt16)
        data: 'abcdefghij'
    crc: 1472812111 (CRCB32)

>>> print to_hex(binary)
   0|  1|  2|  3|  4|  5|  6|  7|  8|  9| 10| 11| 12| 13| 14| 15| 16| 17| 18| 19| 20| 21| 22| 23| 24| 25| 26| 27| 28| 29| 30
\x23\x00\x00\x00\x00\x00\x00\x05\x39\x00\x10\x00\x00\x00\x00\x00\x0A\x61\x62\x63\x64\x65\x66\x67\x68\x69\x6A\x57\xC9\x54\x4F
>>> print frame.introspect()
   -    -  SerialProtocol():
   0 \x23      start_of_frame: 35 (UBInt8)
   1 \x00      destination: 0 (UBInt16)
   2 \x00
   3 \x00      source: 0 (UBInt16)
   4 \x00
   5 \x00      service_id: 1337 (UBInt32)
   6 \x00
   7 \x05
   8 \x39
   9 \x00      payload_length: 16 (UBInt16)
  10 \x10
   -    -      payload: Payload():
  11 \x00          type: 0 (UBInt32)
  12 \x00
  13 \x00
  14 \x00
  15 \x00          length: 10 (UBInt16)
  16 \x0a
   -    -          data: String():
  17 \x61              'a'
  18 \x62              'b'
  19 \x63              'c'
  20 \x64              'd'
  21 \x65              'e'
  22 \x66              'f'
  23 \x67              'g'
  24 \x68              'h'
  25 \x69              'i'
  26 \x6a              'j'
  27 \x57      crc: 1472812111 (CRCB32)
  28 \xc9
  29 \x54
  30 \x4f

>>> # list of u16
... frame.payload.type.set("u16list")
>>> frame.payload.length.set(3)
>>> for ii in range(3):
...     frame.payload.data[ii].set(ii)
...
>>> binary = frame.encode()
>>> print frame
SerialProtocol():
    start_of_frame: 35 (UBInt8)
    destination: 0 (UBInt16)
    source: 0 (UBInt16)
    service_id: 1337 (UBInt32)
    payload_length: 12 (UBInt16)
    payload: Payload():
        type: 'u16list' (UBInt32)
        length: 3 (UBInt16)
        data: Array():
        [
            @0: 0 (UBInt16)
            @1: 1 (UBInt16)
            @2: 2 (UBInt16)
        ]
    crc: 575351022 (CRCB32)

>>> print to_hex(binary)
   0|  1|  2|  3|  4|  5|  6|  7|  8|  9| 10| 11| 12| 13| 14| 15| 16| 17| 18| 19| 20| 21| 22| 23| 24| 25| 26
\x23\x00\x00\x00\x00\x00\x00\x05\x39\x00\x0C\x00\x00\x00\x01\x00\x03\x00\x00\x00\x01\x00\x02\x22\x4B\x28\xEE
>>> print frame.introspectspan>()
   -    -  SerialProtocol():
   0 \x23      start_of_frame: 35 (UBInt8)
   1 \x00      destination: 0 (UBInt16)
   2 \x00
   3 \x00      source: 0 (UBInt16)
   4 \x00
   5 \x00      service_id: 1337 (UBInt32)
   6 \x00
   7 \x05
   8 \x39
   9 \x00      payload_length: 12 (UBInt16)
  10 \x0c
   -    -      payload: Payload():
  11 \x00          type: 1 (UBInt32)
  12 \x00
  13 \x00
  14 \x01
  15 \x00          length: 3 (UBInt16)
  16 \x03
   -    -          data: Array():
   -    -          [
  17 \x00              @0: 0 (UBInt16)
  18 \x00
  19 \x00              @1: 1 (UBInt16)
  20 \x01
  21 \x00              @2: 2 (UBInt16)
  22 \x02
   -    -          ]
  23 \x22      crc: 575351022 (CRCB32)
  24 \x4b
  25 \x28
  26 \xee

>>> # list of float
... frame.payload.type.set("f32list")
>>> frame.payload.length.set(2)
>>> for ii in range(2):
...     frame.payload.data[ii].set(float(ii*1.5))
...
>>> binary = frame.encode()
>>> print frame
SerialProtocol():
    start_of_frame: 35 (UBInt8)
    destination: 0 (UBInt16)
    source: 0 (UBInt16)
    service_id: 1337 (UBInt32)
    payload_length: 10 (UBInt16)
    payload: Payload():
        type: 'f32list' (UBInt32)
        length: 2 (UBInt16)
        data: Array():
        [
            @0: 0.0 (UBInt16)
            @1: 1.5 (UBInt16)
        ]
    crc: 3834901926 (CRCB32)

>>> print to_hex(binary)
   0|  1|  2|  3|  4|  5|  6|  7|  8|  9| 10| 11| 12| 13| 14| 15| 16| 17| 18| 19| 20| 21| 22| 23| 24
\x23\x00\x00\x00\x00\x00\x00\x05\x39\x00\x0A\x00\x00\x00\x02\x00\x02\x00\x00\x00\x01\xE4\x93\xF5\xA6
>>> print frame.introspect()
   -    -  SerialProtocol():
   0 \x23      start_of_frame: 35 (UBInt8)
   1 \x00      destination: 0 (UBInt16)
   2 \x00
   3 \x00      source: 0 (UBInt16)
   4 \x00
   5 \x00      service_id: 1337 (UBInt32)
   6 \x00
   7 \x05
   8 \x39
   9 \x00      payload_length: 10 (UBInt16)
  10 \x0a
   -    -      payload: Payload():
  11 \x00          type: 2 (UBInt32)
  12 \x00
  13 \x00
  14 \x02
  15 \x00          length: 2 (UBInt16)
  16 \x02
   -    -          data: Array():
   -    -          [
  17 \x00              @0: 0.0 (UBInt16)
  18 \x00
  19 \x00              @1: 1.5 (UBInt16)
  20 \x01
   -    -          ]
  21 \xe4      crc: 3834901926 (CRCB32)
  22 \x93
  23 \xf5
  24 \xa6

>>> #decode data
... data = "\x23\x00\x00\x00\x00\x00\x00\x05\x39\x00\x1A\x00\x00\x00\x02\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xA4\x58\x03\x64"
>>> frame.decode(data)
>>> print frame
SerialProtocol():
start_of_frame: 35 (UBInt8)
destination: 0 (UBInt16)
source: 0 (UBInt16)
service_id: 1337 (UBInt32)
payload_length: 26 (UBInt16)
payload: Payload():
    type: 'f32list' (UBInt32)
    length: 5 (UBInt16)
    data: Array():
    [
        @0: 0.0 (BFloat)
        @1: 0.0 (BFloat)
        @2: 0.0 (BFloat)
        @3: 0.0 (BFloat)
        @4: 0.0 (BFloat)
    ]
crc: 2757231460 (CRCB32)

   -    -  SerialProtocol():
   0 \x23      start_of_frame: 35 (UBInt8)
   1 \x00      destination: 0 (UBInt16)
   2 \x00
   3 \x00      source: 0 (UBInt16)
   4 \x00
   5 \x00      service_id: 1337 (UBInt32)
   6 \x00
   7 \x05
   8 \x39
   9 \x00      payload_length: 26 (UBInt16)
  10 \x1a
   -    -      payload: Payload():
  11 \x00          type: 2 (UBInt32)
  12 \x00
  13 \x00
  14 \x02
  15 \x00          length: 5 (UBInt16)
  16 \x05
   -    -          data: Array():
   -    -          [
  17 \x00              @0: 0.0 (BFloat)
  18 \x00
  19 \x00
  20 \x00
  21 \x00              @1: 0.0 (BFloat)
  22 \x00
  23 \x00
  24 \x00
  25 \x00              @2: 0.0 (BFloat)
  26 \x00
  27 \x00
  28 \x00
  29 \x00              @3: 0.0 (BFloat)
  30 \x00
  31 \x00
  32 \x00
  33 \x00              @4: 0.0 (BFloat)
  34 \x00
  35 \x00
  36 \x00
   -    -          ]
  37 \xa4      crc: 2757231460 (CRCB32)
  38 \x58
  39 \x03
  40 \x64

Let’s say you want to create an eeprom content using OSER.

The content has the following structure:

Entry Type Position Description
type U8 0 the module type
serial number String[9] 1 .. 9 serial number: four upper case letters, a minus and four digits
hardware-type U8 10 the hardware-type
calibration F32[2] 11 .. 18 calibration factors
crc U32 12 .. 22 32-bit CRC, polynomial 0x1EDC6F41

All values are stored in big endian.

The following abstraction let’s you create this mapping easily:

>>> from oser import ByteStruct
>>> from oser import UBInt8, BFloat
>>> from oser import RegularExpressionMatch
>>> from oser import Array
>>> from oser import CRCB32
>>> from oser import to_hex
>>>
>>> class EEPROM(ByteStruct):
...     def __init__(self, *args, **kwargs):
...         ByteStruct.__init__(self, *args, **kwargs)
...
...         self.type = UBInt8(0)
...         self.serial_number = RegularExpressionMatch(pattern="[A-Z]{4}-[0-9]{4}", length=9, value="AAAA-0000")
...         self.hardware_type = UBInt8(0)
...         self.calibration = Array(length=2, prototype=BFloat)
...         self.crc = CRCB32(strict=True, polynomial=0x1EDC6F41)
...
>>> eeprom = EEPROM()
>>> eeprom.type.set(3)
>>> eeprom.serial_number.set("ASDF-1337")
>>> eeprom.hardware_type.set(23)
>>> for ii in range(2):
...     eeprom.calibration[ii].set((.1+ii)**3)
...
>>> binary = eeprom.encode()
>>> print to_hex(binary)
   0|  1|  2|  3|  4|  5|  6|  7|  8|  9| 10| 11| 12| 13| 14| 15| 16| 17| 18| 19| 20| 21| 22
\x03\x41\x53\x44\x46\x2D\x31\x33\x33\x37\x17\x3A\x83\x12\x6F\x3F\xAA\x5E\x35\x3D\x10\xBB\xCB
>>> print eeprom.introspect()
   -    -  EEPROM():
   0 \x03      type: 3 (UBInt8)
   -    -      serial_number: RegularExpressionMatch():
   1 \x41          'A'
   2 \x53          'S'
   3 \x44          'D'
   4 \x46          'F'
   5 \x2d          '-'
   6 \x31          '1'
   7 \x33          '3'
   8 \x33          '3'
   9 \x37          '7'
  10 \x17      hardware_type: 23 (UBInt8)
   -    -      calibration: Array():
   -    -      [
  11 \x3a          @0: 0.001 (BFloat)
  12 \x83
  13 \x12
  14 \x6f
  15 \x3f          @1: 1.331 (BFloat)
  16 \xaa
  17 \x5e
  18 \x35
   -    -      ]
  19 \x3d      crc: 1024506827 (CRCB32)
  20 \x10
  21 \xbb
  22 \xcb

You can now write this data into your eeprom.

>>> from oser import ByteStruct
>>> from oser import Array
>>> from oser import to_hex
>>> from oser import UBInt16
>>>
>>> class Data(ByteStruct):
...     def __init__(self):
...         super(Data, self).__init__()
...
...         self.length = UBInt16(4)
...         self.data = Array(length=lambda self: self.length.get(),
...                           prototype=UBInt16,
...                           args=(1337,), # when range is extended
...                                         # instantiate prototype with this arguments
...                           values=[UBInt16(ii) for ii in range(2)])
...
>>> instance = Data()
>>> print instance
Data():
    length: 4 (UBInt16)
    data: Array():
    [
        @0: 0 (UBInt16)
        @1: 1 (UBInt16)
        @2: 1337 (UBInt16)
        @3: 1337 (UBInt16)
    ]

>>> print instance.introspect()
   -    -  Data():
   0 \x00      length: 4 (UBInt16)
   1 \x04
   -    -      data: Array():
   -    -      [
   2 \x00          @0: 0 (UBInt16)
   3 \x00
   4 \x00          @1: 1 (UBInt16)
   5 \x01
   6 \x05          @2: 1337 (UBInt16)
   7 \x39
   8 \x05          @3: 1337 (UBInt16)
   9 \x39
   -    -      ]

>>> binary = instance.encode()
>>> print to_hex(binary)
   0|  1|  2|  3|  4|  5|  6|  7|  8|  9
\x00\x04\x00\x00\x00\x01\x05\x39\x05\x39
>>> bytesDecoded = instance.decode(binary)
>>> print bytesDecoded
10
>>> print instance
Data():
    length: 4 (UBInt16)
    data: Array():
    [
        @0: 0 (UBInt16)
        @1: 1 (UBInt16)
        @2: 1337 (UBInt16)
        @3: 1337 (UBInt16)
    ]

>>> print instance.introspect()
   -    -  Data():
   0 \x00      length: 4 (UBInt16)
   1 \x04
   -    -      data: Array():
   -    -      [
   2 \x00          @0: 0 (UBInt16)
   3 \x00
   4 \x00          @1: 1 (UBInt16)
   5 \x01
   6 \x05          @2: 1337 (UBInt16)
   7 \x39
   8 \x05          @3: 1337 (UBInt16)
   9 \x39
   -    -      ]

>>> print instance.data[3]
1337 (UBInt16)

>>> instance.decode("\x00\x14\x00\x00\x00\x01\x00\x02\x00\x03\x00\x04\x00\x05\x00\x06\x00\x07\x00\x08\x00\x09\x00\x0A\x00\x0B\x00\x0C\x00\x0D\x00\x0E\x00\x0F\x00\x10\x00\x11\x00\x12\x00\x13")
42
>>> print instance
Data():
    length: 20 (UBInt16)
    data: Array():
    [
        @0: 0 (UBInt16)
        @1: 1 (UBInt16)
        @2: 2 (UBInt16)
        @3: 3 (UBInt16)
        @4: 4 (UBInt16)
        @5: 5 (UBInt16)
        @6: 6 (UBInt16)
        @7: 7 (UBInt16)
        @8: 8 (UBInt16)
        @9: 9 (UBInt16)
        @10: 10 (UBInt16)
        @11: 11 (UBInt16)
        @12: 12 (UBInt16)
        @13: 13 (UBInt16)
        @14: 14 (UBInt16)
        @15: 15 (UBInt16)
        @16: 16 (UBInt16)
        @17: 17 (UBInt16)
        @18: 18 (UBInt16)
        @19: 19 (UBInt16)
    ]

>>> print instance.introspect()
   -    -  Data():
   0 \x00      length: 20 (UBInt16)
   1 \x14
   -    -      data: Array():
   -    -      [
   2 \x00          @0: 0 (UBInt16)
   3 \x00
   4 \x00          @1: 1 (UBInt16)
   5 \x01
   6 \x00          @2: 2 (UBInt16)
   7 \x02
   8 \x00          @3: 3 (UBInt16)
   9 \x03
  10 \x00          @4: 4 (UBInt16)
  11 \x04
  12 \x00          @5: 5 (UBInt16)
  13 \x05
  14 \x00          @6: 6 (UBInt16)
  15 \x06
  16 \x00          @7: 7 (UBInt16)
  17 \x07
  18 \x00          @8: 8 (UBInt16)
  19 \x08
  20 \x00          @9: 9 (UBInt16)
  21 \x09
  22 \x00          @10: 10 (UBInt16)
  23 \x0a
  24 \x00          @11: 11 (UBInt16)
  25 \x0b
  26 \x00          @12: 12 (UBInt16)
  27 \x0c
  28 \x00          @13: 13 (UBInt16)
  29 \x0d
  30 \x00          @14: 14 (UBInt16)
  31 \x0e
  32 \x00          @15: 15 (UBInt16)
  33 \x0f
  34 \x00          @16: 16 (UBInt16)
  35 \x10
  36 \x00          @17: 17 (UBInt16)
  37 \x11
  38 \x00          @18: 18 (UBInt16)
  39 \x12
  40 \x00          @19: 19 (UBInt16)
  41 \x13
   -    -      ]

>>> print instance.data[15]
15 (UBInt16)
>>> from oser import ByteStruct
>>> from oser import IfElse
>>> from oser import UBInt8, UBInt64
>>> from oser import to_hex
>>>
>>> class Data(ByteStruct):
...     def __init__(self):
...         super(Data, self).__init__()
...
...         self.condition = UBInt8(1)
...         self.data = IfElse(condition=lambda self: self.condition.get() > 0,
...                              ifTrue=UBInt8(1),
...                              ifFalse=UBInt64(0x0123456789abcdef)
...                              )
...
>>> instance = Data()
>>> print instance
Data():
    condition: 1 (UBInt8)
    data: 1 (UBInt8)

>>> print instance.introspect()
   -    -  Data():
   0 \x01      condition: 1 (UBInt8)
   1 \x01      data: 1 (UBInt8)

>>> binary = instance.encode()
>>> print to_hex(binary)
   0|  1
\x01\x01
>>> bytesDecoded = instance.decode(binary)
>>> print bytesDecoded
2
>>> print instance
Data():
    condition: 1 (UBInt8)
    data: 1 (UBInt8)

>>> print instance.introspect()
   -    -  Data():
   0 \x01      condition: 1 (UBInt8)
   1 \x01      data: 1 (UBInt8)

>>> instance.condition.set(0)
>>> print instance
Data():
    condition: 0 (UBInt8)
    data: 81985529216486895 (UBInt64)

>>> print instance.introspect()
   -    -  Data():
   0 \x00      condition: 0 (UBInt8)
   1 \x01      data: 81985529216486895 (UBInt64)
   2 \x23
   3 \x45
   4 \x67
   5 \x89
   6 \xab
   7 \xcd
   8 \xef

>>> binary = instance.encode()
>>> print to_hex(binary)
   0|  1|  2|  3|  4|  5|  6|  7|  8
\x00\x01\x23\x45\x67\x89\xAB\xCD\xEF
>>> from oser import ByteStruct
>>> from oser import Switch
>>> from oser import UBInt8, UBInt16, UBInt32
>>> from oser import to_hex
>>> from oser import Null
>>>
>>> class Data(ByteStruct):
...     def __init__(self):
...         super(Data, self).__init__()
...
...         self.type = UBInt8(1)
...         self.data = Switch(lambda self: self.type.get(),
...                            values={
...                                1: UBInt8(1),
...                                2: UBInt16(2),
...                                3: UBInt32(3)
...                            },
...                            default=Null())
...
>>> instance = Data()
>>> print instance
Data():
    type: 1 (UBInt8)
    data: 1 (UBInt8)

>>> print instance.introspect()
   -    -  Data():
   0 \x01      type: 1 (UBInt8)
   1 \x01      data: 1 (UBInt8)

>>> binary = instance.encode()
>>> print to_hex(binary)
   0|  1
\x01\x01
>>> bytesDecoded = instance.decode(binary)
>>> print bytesDecoded
2
>>> print instance
Data():
    type: 1 (UBInt8)
    data: 1 (UBInt8)

>>> print instance.introspect()
   -    -  Data():
   0 \x01      type: 1 (UBInt8)
   1 \x01      data: 1 (UBInt8)

>>> # 3
... instance.type.set(3)
>>> print instance
Data():
    type: 3 (UBInt8)
    data: 3 (UBInt32)

>>> print instance.introspect()
   -    -  Data():
   0 \x03      type: 3 (UBInt8)
   1 \x00      data: 3 (UBInt32)
   2 \x00
   3 \x00
   4 \x03

>>> binary = instance.encode()
>>> print to_hex(binary)
   0|  1|  2|  3|  4
\x03\x00\x00\x00\x03
>>> # non existent value -> Null()
... instance.type.set(100)
>>> print instance
Data():
    type: 100 (UBInt8)
    data: Null (Null)

>>> print instance.introspect()
   -    -  Data():
   0 \x64      type: 100 (UBInt8)
   -    -      data: Null

>>> binary = instance.encode()
>>> print to_hex(binary)
   0
\x64
Kontakt
Sollten Sie Fragen haben, nutzen Sie bitte das Kontakformular, schreiben Sie eine E-Mail oder rufen Sie uns an.

Ihr Name*
Firma
Adresse
Ort
Land
EMail-Adresse*
Telefonnummer
Nachricht*
Die mit * markierten Felder müssen angegeben werden.