Binary formats

A binary file format puts each number at a known place, in a known width and byte order. std/codec reads and writes them in a containers.Array( of ubyte ), which fs.WriteAllBytes and fs.ReadAllBytes move to and from a file.

#import "std/codec"
#import "std/containers"
#import "std/fs"

'' A record in a fixed binary layout: id, temperature, then a name.
dim rec as containers.Array( of ubyte )
codec.AppendU32LE( rec, 1042 )               '' 4 bytes, little-endian
codec.AppendF32LE( rec, 21.5 )               '' 4 bytes, IEEE 754
dim tag as ustring = "lab"
codec.AppendUVarint( rec, len( tag ) )       '' the length, then the bytes
for i as int32 = 1 to len( tag )
    rec.Push( asc( tag, i ) )
next
print "record: "; codec.HexEncode( rec, true )

dim scratch as ustring = fs.CreateTempDir( ).Value( )
dim path as ustring = fs.PathJoin( scratch, "record.dat" )
fs.WriteAllBytes( path, rec )

'' Read the file back and pick the fields out again.
dim got as containers.Array( of ubyte )
fs.ReadAllBytes( path, got )
dim at as int32 = 8
dim n as uint64 = codec.ReadUVarint( got, at ).Value( )
dim label as ustring
for i as int32 = 0 to n - 1
    label &= chr( got[ at + i ] )
next
print "id: "; codec.GetU32LE( got, 0 ); ", temp:"; codec.GetF32LE( got, 4 ); _
    ", label: "; label
fs.RemoveDirAll( scratch )

Output

record: 120400000000AC41036C6162
id: 1042, temp: 21.5, label: lab

Every name says what it does: Get, Put or Append, then the type — U16, U32, U64, I16, I32, I64, F32, F64 — then LE for little-endian (least significant byte first) or BE for big-endian. Offsets count bytes from 0. Append writes at the end; Put writes at an offset and grows the array if it has to; Get past the end stops the program, as indexing an array does.

AppendUVarint writes a whole number in as few bytes as it needs, seven bits per byte — here the label's length, 3, takes one byte. ReadUVarint( buf, at ) reads one and moves at past it, so the next field starts where at now points. AppendVarint and ReadVarint do the same for numbers that can be negative.

Activities

1. Write the id big-endian instead, and see which bytes of the record change.
2. Make the label 200 characters long. How many bytes does its length take now?