The tiko Programming Language

Code you can read. Speed you can measure. Binaries you can ship.

tiko is a compiled, statically typed programming language for building fast, self-contained programs. This book teaches it to programmers who already know another language.

Every program printed here was compiled and run — on Windows and on Linux — by the exact compiler named in the preface, before the text around it was written.

The text of this book is licensed CC BY-NC-SA 4.0. The example code is MIT, so you can use it in your own programs, including at work.

Preface

Who this book is for

For programmers who already know a language and want tiko. It does not assume you know BASIC, and it does not assume you have met FreeBASIC. Where tiko differs from FreeBASIC in a way that would surprise someone who knows it, a sidebar says so.

How this book is organized

Conventions

The version this book describes

Every program in this book was compiled and run, on Windows and on Linux, with one pinned version of the compiler. The pin, and the exact toolchains, are recorded in the book's repository.

The reference: the tiko wiki

This book teaches the language; it does not list every keyword, option and library function. That is the job of the tiko wiki, github.com/PaulSquires/tiko-lang/wiki, the language's reference documentation. Keep it open beside the book. The pages you will use most:

• Feature Index — every language feature and library, one line each, with a link to its reference page.
• Function Finder — every function, grouped by what you want to do, with one line on what each one does.
• Keyword Index — every keyword, statement and operator, A to Z, each with its full definition.
• Compatibility — for code brought from FreeBASIC: what differs, and what is the same.

The wiki also carries tutorials (concurrency, packages and modules, translating C headers) and a page for each part of the toolchain. Where a chapter shows one form of a statement or a few members of a library, the wiki has the rest.

The example programs

Getting Started

tiko is a compiled, statically typed language for applications and systems programming. It produces native executables with no garbage collector, no virtual machine and no runtime for your users to install: a finished program is one file you can copy to a machine and run.

Under the hood the compiler translates your program to LLVM IR and calls clang to assemble and link it. That matters for exactly one reason, which this chapter gets out of the way first: clang has to be on the machine, and tiko is particular about where it looks.

This chapter installs the compiler, builds three small programs, reads a diagnostic, and turns on the runtime checks you should leave on while you work through the book.

Installing tiko

A tiko installation is a directory. Unpack it wherever you like; nothing is written to the registry, no environment variable has to be set, and nothing else on the machine is touched. Moving the directory later is fine — the compiler finds its own headers and libraries relative to the executable.

xattr -dr com.apple.quarantine tiko-macos

Check that it runs:

tiko --version

It prints the compiler's version and the target it builds for by default. Keep that version handy: this book was written against one specific release, and the exact one is named in the preface.

The one thing you must install: clang

tiko contains no assembler and no linker. It emits LLVM IR, then runs clang to turn that into an object file and lld to link it. You need LLVM 18 or newer installed, and it is not bundled.

On Windows, install the LLVM package; on Linux, your distribution's clang package; on macOS, Homebrew's (brew install llvm), together with Xcode's Command Line Tools (xcode-select --install), which hold the SDK and the linker every macOS link uses. Then check:

clang --version

If clang is installed in the usual place, there is nothing to configure. tiko looks for it in four places, and takes the first that exists:

So a clang on PATH is found — but only when there is no clang at the conventional location.

export TIKO_CLANG=/home/you/.local/bin/clang

Hello, tiko

Source files are .tiko, and the headers you write yourself are .ti. Here is the whole program:

'' hello.tiko -- the smallest complete tiko program.
print "hello, tiko"

Output

hello, tiko

Compile and run it:

tiko hello.tiko -o hello
./hello

-o names the output. Without it, the executable takes the source file's name. Two other options are worth knowing on day one: -c compiles to an object file without linking, and -v echoes every clang and linker command tiko runs, which is the quickest way to see what a build actually did.

When a build finishes, the exit status tells you what happened, which matters as soon as you put tiko in a script:

Status 4 is a bug in the compiler. It is worth reporting, with the source that caused it.

Reading the command line

Programs get their arguments through command, which returns one argument by position, counting from 1:

'' greet.tko -- a program that reads its command line.
dim as ustring who = command( 1 )
if len( who ) = 0 then who = "world"
print "hello, " & who

./greet tiko
hello, tiko

Two things in those four lines are worth naming now, because every later chapter assumes them.

ustring is the string type. It holds UTF-16 text, its length is O(1), and it behaves the same on every platform. Chapter 9 covers text properly, including the older string type that holds bytes and the two cases where you still want it.

Types say their width. int32, int64, float64 and their siblings mean what they say on every target — no guessing whether int is 32 or 64 bits today. Chapter 4 lists all ten, plus integer, the pointer-sized type that deliberately has no fixed-width name.

Choosing a target

tiko builds for two targets, and the default is the machine you are on.

Name one explicitly with -target:

tiko hello.tiko -target windows-x86_64 -o hello

Targets are not interchangeable at link time: an object file or static library built for one will not link into a program for the other. Chapter 31 covers what each target links and how to ship the result.

Reading a diagnostic

Every message tiko prints carries a stable identifier: TKO followed by four digits. The number never changes meaning and is never reused, so it is worth searching for, and worth quoting in a bug report. Appendix B lists them all.

This program has a type error:

'' oops.tko -- deliberately broken, to show a diagnostic.
dim as int32 count
count = "seven"

Compiling it says so:

oops.tiko(3,1): error TKO0702: a string and a number do not convert into each other
     |
   3 | count = "seven"
     | ^

The parts are the file, the line and column, the severity, the identifier, the message, and then the offending line with a caret under the position. Five options shape this output:

Turn the checks on now

By default, a tiko program does no bounds checking. Reading past the end of an array is undefined: usually it quietly returns nonsense, occasionally it corrupts something, and it always wastes an afternoon.

-exx turns on array-bounds and null-pointer checking, and the cost is irrelevant while you are learning. Use it for everything in this book.

A program can carry its own build options, which is how the listings in this book stay self-contained — the first source file may hold a #cmdline line, and the compiler restarts itself with those options applied:

'' bounds.tko -- what a checked build catches.
#cmdline "-exx"

dim as int32 a( 1 to 3 )
dim as int32 i = 5

a( i ) = 42          '' index 5 is outside 1..3
print "not reached"

Running it stops at the bad index rather than carrying on with a corrupted array:

Aborting due to runtime error 6 (out of bounds array access) at line 7 of bounds.tiko::(), 
'A' accessed with invalid index = 5, must be between 1 and 3

The process exits with status 6. Without -exx the same program writes 42 somewhere it should not and prints not reached, which is the outcome the check exists to prevent.

The check covers every array, including the ones that grow while the program runs, which Chapter 8 introduces. The null-pointer check is partial: it catches *p, but not yet p->field or p[i], which Chapter 11 covers.

There are three levels, and they stack: -e enables error checking, -ex adds to it, and -exx adds array bounds and null-pointer checks on top. Chapter 18 comes back to them alongside the error-handling model.

An editor

None of this book requires a particular editor; every example is a file and a command line. But two tools come with tiko, and both are worth the five minutes:

• tkEdit is the editor in the tiko-lang folder. It builds and runs your program with one key, lists the compiler's errors and takes you to each one, and shows a procedure's parameters as you type its call.
• tiko-lsp is tiko's language server. It gives the editor you already use — VS Code, Zed, Neovim, Sublime Text and others — errors as you type, completion, go to definition, rename and formatting.

Chapter 2 sets up both.

Looking things up

The reference for everything this book does not spell out — every keyword's full syntax, every compiler option, every function in the standard library — is the tiko wiki, at github.com/PaulSquires/tiko-lang/wiki. Two pages answer most questions: the Feature Index, one line per language feature and library, and the Function Finder, every function grouped by the job it does. The wiki's Quick Start covers building a toolchain from source, which this chapter does not.

Summary

• A tiko installation is a directory you unpack anywhere; the compiler locates its own headers and libraries.
• LLVM 18+ must be installed. tiko finds clang in four places, in order: an explicit path, TIKO_CLANG, the conventional install location, then PATH.
• Source files are .tiko; headers you write are .ti.
• tiko hello.tiko -o hello compiles and links in one step; -c stops at the object file and -v shows every command.
• Two targets: windows-x86_64 (the Windows default, and no Visual Studio needed) and linux-x86_64 (the Linux default). Objects from different targets do not link together.
• Diagnostics carry stable TKOnnnn identifiers; --diagnostics-format gnu suits tools, and -Werror suits discipline.
• Build with -exx while you learn, and let a listing carry its own options with #cmdline.
• ustring is the string type, and numeric types say their width: int32, int64, float64.
• The reference for every keyword, option and function is the tiko wiki; start from its Feature Index or Function Finder.

Exercises

1. Write a program that reports which compiler built it and which version that compiler is. __TIKO_COMPILER__ is defined by tiko and by nothing else, and __TIKO_VERSION__ holds the version as a string. Test the first with #ifdef. {solution="ex01-01"}
2. Make a program greet you by name and then exit with status 3, and check the status from your shell (echo $?, or echo %ERRORLEVEL% in cmd). {solution="ex01-02"}
3. Call a procedure that was never declared, and note the identifier of the diagnostic. Then look it up in Appendix B. {solution="ex01-03"}
4. Compile bounds.tiko again with the #cmdline line deleted, and run it. What does it print, and what is its exit status? That is the difference -exx makes.

Editors: tkEdit and the Language Server

The compiler tells you about a mistake when you build. That is a round trip: save, switch to a terminal, run tiko, read a line number, go back and find the line. For a missing end if the trip costs more than the fix.

An editor that understands tiko shortens it. It underlines the missing end if while you are still on the line, shows a procedure's parameters while you type its call, takes you to a declaration with one key, and builds and runs the program with another.

tiko comes with two tools for this. tkEdit is an editor of its own, in the same folder as the compiler. tiko-lsp, the tiko language server, gives the editor you already use — VS Code, Zed, Neovim, Sublime Text and others — the same understanding. This chapter sets up both. Neither is required by anything else in this book: every example is a file and a command line.

What the editor knows, and what only the compiler knows

Both tools read your code as you type, and both do it without compiling it. They find the declarations — which names a file declares and where, what type a variable was declared with, which members that type has, what an #include or an #import brings in — in well under the time between two keystrokes: the language server reads a 900-line file in about a millisecond. That is enough for completion, codetips and going to a declaration. The language server also checks the code's shape — a function with no end function, a string with no closing quote — and underlines a mistake in it as you type.

Neither knows what only compiling tells you: whether a name used in an expression was ever declared, whether a string is being assigned to a number, whether a call has the right number of arguments. Those errors come from the compiler, when you build. Both tools then show the compiler's errors in the source and in a list, beside their own, so the difference is easy to forget. The Pitfall at the end of the chapter shows what that costs.

This program is the one the chapter edits:

'' The program Chapter 2 edits: a type, a procedure that
'' uses it, and two built-ins.

type Rect
    w as int32
    h as int32
end type

function Area( byref r as Rect ) as int32
    return r.w * r.h
end function

dim as Rect box
box.w = 3
box.h = 4

dim as ustring label = "box"
print ucase( left( label, 1 ) ) & ": " & Area( box )

B: 12

tkEdit

tkEdit is a small, keyboard-driven editor for tiko, on Windows, Linux and macOS. It comes with the toolchain, so there is nothing else to install:

Start it with the files to edit, or with none to restore the last session:

tkedit rect.tiko

The editor fills the window, with a tab for each open file above it and no toolbar. The menu bar appears when you press Ctrl+Shift+M and goes again when you pick a command. On a Mac, every Ctrl chord in this section is Cmd.

tkEdit finds the compiler beside itself, in the same tiko-lang folder, or on your PATH; if it cannot, it asks for it once, when it starts.

Building and running

What F5 builds depends on what you have open. With a project — a folder with a tiko.toml, opened by opening that file — it builds the project's target, as tiko build would. Without one, it builds the file you are editing. Modified source files are saved first.

When the build fails, the compiler's errors open in the output pane at the bottom of the window, each with its TKOnnnn identifier, and are underlined in the source; Alt+PgDn takes you from one to the next. A console program runs in a terminal of its own, which waits for a key when the program ends, so its output stays on the screen.

Debugging

tkEdit can stop your program on a line and show you its variables. It does this through lldb-dap, the LLVM debugger that editors talk to, reading the DWARF debug information that -g writes — the same information Chapter 19 reads with lldb from a terminal.

The same commands are on the Debug menu, after Build.

A breakpoint is a red circle in the margin. F9 sets it; a click in the margin sets a bookmark instead, unless the setting margin_click = "breakpoint" is in the [editor] table of settings.toml. Breakpoints are kept with the session, so they are still there tomorrow.

F6 builds with your configuration's options, without its -O and with -g -exx added, because an optimised program has lost the variables you want to look at. Then it runs the program in its terminal, as F5 does, and stops on the first breakpoint it reaches — or, with no breakpoint set, on the program's first statement. A yellow arrow in the margin marks the line about to run, and the output pane opens on its Debug tab:

• The buttons along the top are the commands in the table: Continue, Break, Stop, Step, Step Over, Step Out, Run to Cursor. The status line at the bottom of the window says why the program stopped and where — Stopped - breakpoint rect.tiko:10 in Area.
• Locals, Globals and Watch share the left side, with their tabs along its bottom. Each variable has a row: its name, its value and its type. A record opens to its fields and an array to its elements, named the way the program names them — rs(1), rs(2), rs(3) for an array declared rs( 1 to 3 ). A large array opens in groups of a hundred elements first. A value that changed since the last stop is shown in red, and what you opened stays open as you step. A click on a column heading sorts by it.
• Call Stack and Breakpoints share the right side. A click on a frame of the call stack shows that procedure's locals and goes to its line. The Breakpoints list has every breakpoint in every open file, with the text of its line; a double click goes to it. Of the two buttons at the end of its tab row, the first removes the selected breakpoint and the second removes them all.

On the Watch tab, the + button adds a row; type a name to watch, written as in the program — r.w, rs(2).sensor, in any case — and press Enter. Its value is read again at every stop. A double click on the row edits it, and emptying it removes it. A watch is a variable, a field or an element, not a calculation: r.w * 2 shows <not in scope>, as does a name the stopped procedure cannot see.

While the program is stopped, rest the mouse on a name in the source and its value appears beside it.

Finding your way

Codetips and completion

Type Area( in rect.tiko and tkEdit shows the procedure's parameter list, read from its declaration. Type left( and it shows Left(string, n): the built-in procedures have codetips too, square brackets marking the arguments you may leave out. Type box. and the members of Rect are offered; type three letters of a name and the names that start with them are offered, and after as, the types. Ctrl+Space completes a word at any time. The setting codetips turns all of this off.

Settings

tkEdit is configured with three text files, and saving one applies it:

The language server

tiko-lsp speaks the Language Server Protocol, which most editors understand. The editor starts it in the background and asks it questions — what is under the cursor, what could be typed here, where is this declared — and it answers in terms of tiko. One server, so every editor gets the same answers.

VS Code

1. In VS Code, open the Extensions view (Ctrl+Shift+X), search for tiko and install Tiko. VSCodium, Cursor and the other editors built on VS Code install it the same way, from their own Extensions view.
2. Open the folder your program is in, with File > Open Folder. The server reads the whole folder, so opening a single file gives you less.
3. Open rect.tiko. Hover Area; press F12 on it; type box. on a new line and see w and h offered.

The extension carries its own copy of the server, for Windows, Linux and macOS on Apple silicon.

Building and running. Press Ctrl+F5, or click the ▶ button at the top right of the editor, to run the program: the extension runs tiko run in VS Code's terminal, so the program can read input. Tiko: Build builds without running; Ctrl+Shift+B offers the same build. As in tkEdit, a tiko.toml at or above the file is built as a project, and otherwise the file itself is. The compiler's errors go to the Problems view (Ctrl+Shift+M); click one to go to it.

Finding the toolchain. For the standard library and the system headers — #import "std/strings", #include "windows.bi" — the server needs the toolchain. It finds it by itself if tiko is on your PATH, or if the tiko-lang folder is in your home folder or your Downloads folder. Otherwise VS Code says so, once, with a Locate Toolchain… button: choose the tiko-lang folder and the extension saves it as the setting tiko.toolchainPath. The command Tiko: Show Language Server Output names the toolchain in use.

Files. .tiko, .ti and .tko files are tiko. The .bas, .bi and .inc extensions of older tiko code are also used by other languages, so the extension leaves them alone until you claim them, in the folder's .vscode/settings.json:

"files.associations": { "*.bas": "tiko", "*.bi": "tiko", "*.inc": "tiko" }

Other editors

Zed, Neovim, Sublime Text, Helix and any other editor that speaks the protocol run the same server, and every tiko release carries it beside the compiler:

So if tiko is on your PATH, tiko-lsp is too. The tiko packages for Zed, Neovim and Sublime Text find it there, or in the tiko-lang folder you name in their settings; the server then takes the headers and packages of that same toolchain. Their instructions are in the editors folder of the tiko-lsp repository. Any other editor needs three things: the command tiko-lsp --stdio, the extensions .tiko, .ti and .tko mapped to a language called tiko, and — if tiko is not on your PATH — the toolchain folder as the option toolchainPath. For Helix, in languages.toml:

[language-server.tiko-lsp]
command = "tiko-lsp"
args = ["--stdio"]
config = { toolchainPath = "/home/you/tiko-lang" }

[[language]]
name = "tiko"
scope = "source.tiko"
file-types = ["tiko", "ti", "tko"]
roots = ["tiko.toml", ".git"]
comment-token = "'"
language-servers = ["tiko-lsp"]

Updating

The server in your other editors is the one in the toolchain, so installing a newer tiko updates it; restart the server, or the editor, afterwards. VS Code updates its extension, and the server inside it, by itself.

When something is wrong

• No colour, no completion. The editor has not recognised the file as tiko. In VS Code, the language mode at the bottom right should say Tiko. If VS Code shows Restricted Mode, the folder is not trusted, and the extension does not start a program in an untrusted folder.
• std/ and windows.bi names do not complete. The server has not found the toolchain. Name the tiko-lang folder (in VS Code, with Tiko: Locate Toolchain…) and restart the server.
• An underline on code that compiles. That is a defect in the server. Report it at the tiko-lsp repository with the smallest program that shows it.
• You want to see what the server is doing. Start it with --log-level debug --log-file and a path; it writes nothing else to disk. tiko-lsp --version names the version you have.

A clean editor is not a clean build

'' Every block is closed and every statement is complete,
'' so the editor underlines nothing. The compiler still
'' refuses it: `totl` was never declared.

dim as int32 total = 10
print totl

typo.tiko(6,7): error TKO0743: 'totl' is not declared
     |
   6 | print totl
     |       ^

What it costs

• The first look at a large header takes seconds. windows.bi and what it includes declare about 190,000 names. The server reads them once, in the background, when a project uses them; after that, completion costs tens of milliseconds a keystroke instead of a few.
• The reader does not run the preprocessor. Both branches of an #ifdef are read, so completion offers MessageBoxA and MessageBoxW alike, and a name made by #define is not offered at all.
• Rename works by name. It follows what each file can see, and refuses where a local variable shares the name it would change, but two unrelated declarations with one name in files that see each other are one name to it.
• Codetips for the built-ins are a fixed list, shipped with the tools; a procedure you declare is described by its own declaration instead, and wins over a built-in of the same name.

Summary

• tkEdit and tiko-lsp read your code as you type, without compiling it. They catch mistakes in its shape; the compiler catches the rest, when you build.
• tkEdit comes with tiko (tiko-lang/tkedit). F5 builds and runs — a tiko.toml project, or else the current file — and Alt+PgDn walks the compiler's errors.
• F12 goes to a declaration, Ctrl+P finds any name, codetips show parameters as you type, and three TOML files hold every setting.
• tiko-lsp gives any editor that speaks the Language Server Protocol completion, hover, signature help, go to definition, references, rename, formatting and errors as you type.
• In VS Code, install the Tiko extension: it carries the server, runs your program with Ctrl+F5, and puts the compiler's errors in Problems.
• Every tiko release carries tiko-lsp beside the compiler, for every other editor; a newer tiko updates it.
• F9 sets a breakpoint and F6 runs to it; F10, F11 and Shift+F11 step. The Debug tab shows locals, globals and watches beside the call stack and the breakpoints. It needs lldb-dap, and on macOS Developer Mode.
• No underline is not a successful build. Build, and read what the compiler says.

Exercises

1. Open rect.tiko in tkEdit and run it with F5. Delete the line end function, build again, and use Alt+PgDn to reach the error. Which line does it point at, and why that one?
2. In tkEdit, put the cursor on Area in the last line, go to its declaration with F12, and come back with Alt+Left. Then, in an editor with the language server, rename Rect to Box2 (in VS Code, F2), and check that the declaration and the dim changed together.
3. In an editor with the language server, start typing print mid( and watch the parameter list. Type an argument and a comma at a time: which parameter is marked after each comma, and what happens after the third?
4. Copy typo.tiko into your editor. Find out, without building, which tool would tell you about totl, then build and compare the identifier you get with the one in this chapter.
5. In tkEdit, set a breakpoint on line 10 of rect.tiko (return r.w * r.h) and press F6. Open r on the Locals tab. Then select the second row of the Call Stack: what does Locals show now, and where is the editor's cursor? Step out with Shift+F11 and watch the arrow.

A Tour of tiko

The last chapter got a compiler working. This one writes something with it — a program that reads a file, counts the words in it, and reports the three most common. It is small enough to hold in your head and big enough to need most of what the language is made of: the standard library, strings, a hash map, iteration, sorting, a lambda, and errors that are values rather than surprises.

Nothing here is explained completely. The point is to see the shape of a tiko program before taking it apart, and every idea below is picked up properly in a later chapter. Read it the way you would read someone else's code: get the sense of it, and let the details wait.

The file we are counting is three lines:

the quick brown fox
jumps over the lazy dog
the dog sleeps

Reaching the standard library

tiko's library comes in modules, and a program says which ones it wants:

'' step1.tiko -- read the file, and say so if it is not there.
#import "std/fs"

var loaded = fs.ReadAllText( "words.txt" )
if loaded.IsOk( ) = false then
    print "cannot read words.txt"
    end 1
end if

var text = loaded.Value( )
print "characters: " & len( text )

characters: 59

#import "std/fs" makes the filesystem module available under the name fs, and every name it provides is reached through that prefix — fs.ReadAllText. That prefix is not decoration; it is how a reader of your code knows where a function came from.

The interesting part is what ReadAllText gives back. Not a string: a Result, which is either the text or the reason there is no text. There is no exception to catch and no error code to remember to check, because the value itself will not hand over its contents until you have asked whether it has any:

• loaded.IsOk( ) — did it work?
• loaded.Value( ) — the text, once you know it worked.

A program that forgets to check does not silently read nonsense; it fails to compile in the shape you would have wanted. Chapter 18 is about designing with Result.

Splitting the text into words

'' step2.tiko -- split the text into words.
#import "std/fs"
#import "std/strings"

var loaded = fs.ReadAllText( "words.txt" )
if loaded.IsOk( ) = false then
    print "cannot read words.txt"
    end 1
end if

var flat = strings.Replace( loaded.Value( ), !"\n", " " )
var words = strings.Split( flat, " " )
print "words: " & words.Count

for each w in words
    if len( w ) > 0 then print "  [" & w & "]"
next

words: 13
  [the]
  [quick]
  [brown]
  [fox]
  [jumps]
  [over]
  [the]
  [lazy]
  [dog]
  [the]
  [dog]
  [sleeps]

Two library calls do the work. strings.Replace flattens the newlines into spaces, and strings.Split cuts the result on spaces and hands back an array. words.Count is how many it found.

for each w in words walks the array without an index, without a bound, and without the off-by-one that an index loop invites. The loop variable's type comes from the array, so w is a ustring — tiko's string type, which holds text rather than bytes and is the same on every platform.

The if len( w ) > 0 is there because splitting on a single space leaves empty pieces wherever two spaces met. Real text is untidy; the program says so rather than pretending.

Counting

'' step3.tiko -- count each word.
#import "std/containers"
#import "std/fs"
#import "std/strings"

var loaded = fs.ReadAllText( "words.txt" )
if loaded.IsOk( ) = false then
    print "cannot read words.txt"
    end 1
end if

var flat = strings.Replace( loaded.Value( ), !"\n", " " )
var words = strings.Split( flat, " " )

dim counts as containers.Map( of ustring, int32 )
for each w in words
    if len( w ) > 0 then counts[ LCase( w ) ] += 1
next

print "distinct words: " & counts.Count
print "the: " & counts[ "the" ]
print "dog: " & counts[ "dog" ]

distinct words: 9
the: 3
dog: 2

containers.Map( of ustring, int32 ) is a hash map from text to a number. The ( of … ) is tiko's generics syntax: this is not a map of "anything", it is a map whose keys are ustring and whose values are int32, checked as thoroughly as if you had written the type by hand. Chapter 17 is about writing your own.

counts[ LCase( w ) ] += 1 does the whole count. Indexing a map with a key that is not there inserts it with a zero value, so the += 1 works the first time a word appears as well as the fifth. LCase folds the case, so The and the are one word.

Ranking, with a lambda

A map has no order, so the last step copies the counts into an array and sorts it:

dim table as containers.Array( of Tally_ )
for each entry in counts
    dim as Tally_ t = ( entry.Key, entry.Value )
    table.Push( t )
next

containers.SortBy( table, _
    function( byref a as Tally_, byref b as Tally_ ) as boolean
        return a.count > b.count
    end function )

for each entry in counts walks the map, and each entry carries a Key and a Value.

containers.SortBy takes the array and a lambda — a function written where it is used, with no name. This one answers a single question: does a come before b? Sorting by descending count is a.count > b.count, and changing the ranking means changing that one line. Chapter 20 is about lambdas, including the ones that capture variables from around them.

The _ at the end of the SortBy line continues the statement onto the next line. tiko has no statement terminator, so a line break ends a statement unless you say otherwise.

Reporting

print "the three most common words"
for i as int32 = 0 to 2
    print using "  \    \ #"; table[ i ].word; table[ i ].count
next

print using formats output against a picture: \ \ is a text field of that width, and # is a digit. It is the one piece of this program that will look unfamiliar to most readers, and it is worth the surprise — lining up columns with string concatenation is miserable.

The whole program

#import "std/containers"
#import "std/fs"
#import "std/strings"

type Tally_
    as ustring word
    as int32 count
end type

var loaded = fs.ReadAllText( "words.txt" )
if loaded.IsOk( ) = false then
    print "cannot read words.txt"
    end 1
end if

var flat = strings.Replace( loaded.Value( ), !"\n", " " )
var words = strings.Split( flat, " " )

dim counts as containers.Map( of ustring, int32 )
for each w in words
    if len( w ) > 0 then counts[ LCase( w ) ] += 1
next

dim table as containers.Array( of Tally_ )
for each entry in counts
    dim as Tally_ t = ( entry.Key, entry.Value )
    table.Push( t )
next

containers.SortBy( table, _
    function( byref a as Tally_, byref b as Tally_ ) as boolean
        return a.count > b.count
    end function )

print "the three most common words"
for i as int32 = 0 to 2
    print using "  \    \ #"; table[ i ].word; table[ i ].count
next

the three most common words
  the    3
  dog    2
  quick  1

Forty lines, no memory management, no build file, and one command to build it:

tiko wordcount.tiko -o wordcount
./wordcount

The result is a native executable that depends on nothing you have to ship beside it.

What you have seen

Each of these has a chapter of its own:

Summary

• A program reaches the library with #import "std/…", and uses the module's name as a prefix.
• Fallible calls return a Result: ask IsOk( ), then take Value( ).
• var infers a variable's type; dim as T states it.
• for each walks an array without an index; the loop variable's type comes from the elements.
• Map( of K, V ) is a checked hash map, and indexing a missing key inserts it.
• A lambda is a function written where it is used — SortBy takes one to decide the order.
• One command compiles and links a native executable.

Exercises

1. Report the longest word in the file as well as the most common ones. {solution="ex03-01"}
2. List the words that appear exactly once, in alphabetical order. OrderedMap keeps its keys sorted, where Map does not. {solution="ex03-02"}
3. Make the program take the file name from the command line, falling back to words.txt when none is given. Chapter 1 showed command( 1 ).
4. The ranking prints three words. Make it print the top three counts, so that words tied for third all appear.

Types, Variables and Constants

Every value in a tiko program has a type, fixed when the program is compiled and never changing afterwards. That is what lets the compiler catch a whole class of mistake before you run anything, and it is why a tiko program runs at the speed it does: the machine code knows exactly what it is working with.

This chapter is the vocabulary the rest of the book assumes — how to write values, how to name them, and what happens when one type becomes another.

Writing the program

A few rules apply everywhere, and they are quickly stated.

Case does not matter to the compiler. Count, count and COUNT are one name. It matters very much to readers, so the book follows one convention throughout: keywords lowercase, and types and procedures in PascalCase.

Comments start with '' and run to the end of the line. A block comment is /' … '/, and those nest.

One statement per line, with no terminator to forget. A line can be continued with a trailing _, and two short statements can share a line with : between them — the book uses that only for setup lines that read better together.

Writing values

'' literals.tiko -- how values are written.
print "hex   &hFF      = " & &hFF
print "octal &o755     = " & &o755
print "binary &b1011   = " & &b1011
print
print "a suffix types the literal: " & sizeof( 1LL ) & " bytes for 1LL"
print
'' A plain string is literal; !"..." reads escapes; $"..." takes none.
print !"escaped:\tone tab, then a newline\n"
print $"raw: \t stays two characters"
print
print "true and false are values: " & true & " " & false

hex   &hFF      = 255
octal &o755     = 493
binary &b1011   = 11

a suffix types the literal: 8 bytes for 1LL

escaped:	one tab, then a newline

raw: \t stays two characters

true and false are values: true false

Numbers can be written in hexadecimal, octal or binary with &h, &o and &b. A suffix pins a literal's type where it matters — 1LL is 64-bit.

Strings come in three spellings, and the difference is what a backslash means:

true and false are ordinary values of type boolean.

Numbers that state their width

tiko's numeric types say how big they are:

'' widths.tiko -- each type states its size in bytes.
print "int8    " & sizeof( int8 )   & "   uint8   " & sizeof( uint8 )
print "int16   " & sizeof( int16 )  & "   uint16  " & sizeof( uint16 )
print "int32   " & sizeof( int32 )  & "   uint32  " & sizeof( uint32 )
print "int64   " & sizeof( int64 )  & "   uint64  " & sizeof( uint64 )
print "float32 " & sizeof( float32 ) & "   float64 " & sizeof( float64 )
print
print "integer " & sizeof( integer ) & "   -- pointer-sized"
print "long    " & sizeof( long ) & "   -- 32 bits, despite the name"

int8    1   uint8   1
int16   2   uint16  2
int32   4   uint32  4
int64   8   uint64  8
float32 4   float64 8

integer 8   -- pointer-sized
long    4   -- 32 bits, despite the name

int8, int16, int32, int64 and the uint forms cover the integers; float32 and float64 cover the rest. Use these. A reader never has to ask how wide int32 is, and the answer is the same on every target.

Two types deliberately have no fixed-width name:

• integer is pointer-sized — 8 bytes on the 64-bit targets. Reach for it when a value has to match the machine's word, such as an index into something that could be huge.
• uinteger is its unsigned twin.

You will also meet the older names in existing code: byte, short, long, longint, single, double, and their unsigned forms. They are what the fixed-width names are made of — int32 is long — so the two mix freely.

The fixed-width names are typedefs the compiler predeclares rather than reserved words, which has a visible consequence:

'' shadow.tiko -- the fixed-width names are typedefs, not keywords,
'' not keywords, so a program is free to take one for itself.
dim as int32 n = 1

scope
    dim as int32 int32 = 7     '' a variable named after its own type
    print "shadowed: " & int32
end scope

print "outside:  " & n

shadowed: 7
outside:  1

A program can declare a variable called int32 and it wins inside its scope. That is not an oversight: existing headers already declare names like INT32, and making them keywords would have broken every one of them.

Naming values

'' declare.tiko -- naming values.
dim as int32 counted = 3            '' the type, stated
var inferred = 3                    '' inferred from the value
var text = "hello"                  '' a ustring
dim as float64 ratio = 1.5

print counted & " " & inferred & " " & text & " " & ratio

'' Several names of one type, and one left deliberately uninitialised.
dim as int32 x, y, z
dim as int32 scratch = any          '' left uninitialised
x = 1 : y = 2 : z = 3
print x + y + z

'' A constant cannot be assigned to, and its type comes from the value.
const MaxRetries = 3
const Greeting as ustring = "hi"
print MaxRetries & " " & Greeting

'' An enumeration names related constants.
enum Colour_
    Red
    Green = 10
    Blue
end enum
print Red & " " & Green & " " & Blue

3 3 hello 1.5
 6
3 hi
0 10 11

dim as T name declares a variable of a stated type. var name = value declares one and takes the type from the value. The book's rule: var when the type is obvious from the line, dim as when it is not.

Other forms in that listing:

• dim as int32 x, y, z declares several at once.
• = any leaves a variable uninitialised, holding whatever was in that memory. It is a deliberate, visible choice, not the default.
• const names a value that cannot be assigned to. Its type comes from the value unless you state one.
• enum names a set of related constants; values follow on from the last one given, so Blue is 11.

Every variable must be declared. There are no implicit variables, which means a typo is caught rather than silently becoming a second variable:

'' implicit.tiko -- every variable is declared. There is no implicit
'' creation, so a typo is a compile error, not a second variable.
dim as int32 total = 0

totl = total + 1      '' a typo, not a new variable
print totl

implicit.tiko(5,1): error TKO0743: 'totl' is not declared
     |
   5 | totl = total + 1      '' a typo, not a new variable
     | ^

One type becoming another

Assigning a value of one numeric type to another converts it, and conversions between integer types are the obvious ones: the value is kept when it fits and wraps when it does not.

Converting a float to an integer is where tiko surprises people who come from C:

'' rounding.tiko -- converting a float to an integer ROUNDS.
dim as float64 half = 2.5
dim as float64 up   = 2.6
dim as float64 neg  = -2.5

print "cint( 2.5 )  = " & cint( half )    '' rounds to even
print "cint( 3.5 )  = " & cint( 3.5 )
print "cint( 2.6 )  = " & cint( up )
print "cint( -2.5 ) = " & cint( neg )
print
print "int( 2.6 )   = " & int( up )       '' towards minus infinity
print "int( -2.5 )  = " & int( neg )
print "fix( 2.6 )   = " & fix( up )       '' towards zero, like C
print "fix( -2.5 )  = " & fix( neg )
print
dim as int32 assigned = up                '' an assignment rounds too
print "dim as int32 = 2.6 gives " & assigned

cint( 2.5 )  = 2
cint( 3.5 )  = 4
cint( 2.6 )  = 3
cint( -2.5 ) = -2

int( 2.6 )   = 2
int( -2.5 )  = -3
fix( 2.6 )   = 2
fix( -2.5 )  = -2

dim as int32 = 2.6 gives 3

The c* family converts explicitly: cint, clng, clngint, csng, cdbl, cbool, cbyte, cshort, and the unsigned forms. cast( T, value ) converts to any type you name, and cunsg / csign reinterpret a value's sign without changing its bits.

Asking about a type

• sizeof( T ) is the size in bytes, as the listing above used.
• len( x ) is the length of a string, or the size of anything else.
• typeof( x ) is the type of an expression, usable where a type is expected.

fblimits.bi names the minimum and maximum of each type, if you would rather not write 2147483647 and hope.

Summary

• Types are fixed at compile time; every variable is declared, so a typo is an error.
• Prefer the width-stating names: int8…int64, uint8…uint64, float32, float64.
• integer and uinteger are pointer-sized and have no fixed-width spelling — that is the reason to use them.
• long is 32 bits. The older names still appear in existing code, and mix freely with the new ones.
• The fixed-width names are typedefs, so a program may shadow one.
• var infers a type from the value; dim as T states it; const names a value; enum names a set.
• Float to integer rounds, half to even — fix( ) truncates like C, int( ) floors.
• Numbers may be written in hex, octal or binary; !"…" reads escapes and $"…" takes none.

Exercises

1. Print the largest value each signed integer type can hold, then add one to the int32 maximum and explain what you see. std/numeric's AddChecked gives a second opinion. {solution="ex04-01"}
2. Build a table comparing cint, int and fix over 2.4, 2.5, 2.6 and their negatives. Which pair agrees on positive numbers but not negative ones? {solution="ex04-02"}
3. Declare a variable with = any, print it before assigning anything, and run the program twice. Is the value the same both times? Should you ever rely on it?
4. const Pi = 3.14159 — what type does Pi get? Use sizeof( typeof( Pi ) ) to find out, then state the type explicitly and compare.

Operators and Expressions

Operators are the part of a language people assume they already know, which is exactly why they are worth a chapter. Most of tiko's behave as you expect. A handful do not, and each of those is a bug waiting in code that was written by assumption rather than by reading.

Everything in this chapter was measured against the compiler. Where tiko's grouping differs from the language you came from, the listing prints both readings so you can see which one won.

Arithmetic

'' arithmetic.tiko -- the operators, and the three that surprise people.
print "7 / 2   = " & ( 7 / 2 )      '' always a float
print "7 \ 2   = " & ( 7 \ 2 )      '' integer division
print "7 mod 2 = " & ( 7 mod 2 )
print
'' mod takes the sign of the LEFT operand: a remainder, not a modulus.
print "-7 mod 2 = " & ( -7 mod 2 )
print " 7 mod -2 = " & ( 7 mod -2 )
print
'' '^' is floating point even when both operands are whole numbers.
print "2 ^ 10 = " & ( 2 ^ 10 ) & _
      "   and it is " & sizeof( typeof( 2 ^ 10 ) ) & " bytes wide"

7 / 2   = 3.5
7 \ 2   = 3
7 mod 2 = 1

-7 mod 2 = -1
 7 mod -2 = 1

2 ^ 10 = 1024   and it is 8 bytes wide

Three of those need naming.

/ is always floating point. 7 / 2 is 3.5 even though both operands are whole numbers. Integer division is a separate operator, \, and it truncates: 7 \ 2 is 3.

mod is a remainder, not a modulus. It takes the sign of its left operand, so -7 mod 2 is −1 rather than 1. When you want the mathematical modulus — always non-negative for a positive divisor — std/numeric's FloorMod is the one to reach for. This matters the moment you use mod to wrap an index that might be negative.

^ produces a float. 2 ^ 10 is 1024, but it is a float64 that happens to hold a whole number, and it was computed in floating point. For integer powers where exactness matters, std/numeric has IPow.

Precedence, and its four surprises

tiko's precedence table was established by compiling 484 operator pairs and reading what the compiler did with each. Four of the answers differ from what a C, Python or Java programmer expects, and this listing prints each one beside both possible groupings:

'' precedence.tiko -- four places the table is not what people expect.
dim as int32 a = 3, b = 0

'' 1. '^' groups to the LEFT. Most languages group it to the right.
print "2 ^ 3 ^ 2   = " & ( 2 ^ 3 ^ 2 )
print "  (2^3)^2   = " & ( ( 2 ^ 3 ) ^ 2 )
print "  2^(3^2)   = " & ( 2 ^ ( 3 ^ 2 ) )
print

'' 2. 'shl' and 'shr' bind TIGHTER than '+'.
print "1 shl 2 + 3 = " & ( 1 shl 2 + 3 )
print "  (1 shl 2) + 3 = " & ( ( 1 shl 2 ) + 3 )
print "  1 shl (2 + 3) = " & ( 1 shl ( 2 + 3 ) )
print

print "not a = b     = " & ( not a = b )
print "  not (a = b) = " & ( not ( a = b ) )
print "not a = b       = " & ( not a = b )
print "  not (a = b)   = " & ( not ( a = b ) )
print "  (not a) = b   = " & ( ( not a ) = b )
print
'' 4. 'andalso' and 'orelse' share one level: they group left to right.

print "true orelse false andalso false = " & _
      ( true orelse false andalso false )
print "  (true orelse false) andalso false = " & _
      ( ( true orelse false ) andalso false )

2 ^ 3 ^ 2   = 64
  (2^3)^2   = 64
  2^(3^2)   = 512

1 shl 2 + 3 = 7
  (1 shl 2) + 3 = 7
  1 shl (2 + 3) = 32

not a = b     = -1
  not (a = b) = -1
not a = b       = -1
  not (a = b)   = -1
  (not a) = b   = 0

true orelse false andalso false = false
  (true orelse false) andalso false = false

Two families of logical operator

'' logic.tiko -- two families of operator that look alike and are not.
declare function Loud( byval v as boolean ) as boolean

function Loud( byval v as boolean ) as boolean
    print "   (Loud was called)"
    return v
end function

print "andalso stops once the answer is known:"
if false andalso Loud( true ) then print "unreachable"

print "and always evaluates both:"
if false and Loud( true ) then print "unreachable"

print
'' On integers the bitwise family works bit by bit.
print "12 and 10 = " & ( 12 and 10 )
print "12 or  10 = " & ( 12 or 10 )
print "12 xor 10 = " & ( 12 xor 10 )
print "not 12    = " & ( not 12 )

andalso stops once the answer is known:
and always evaluates both:
   (Loud was called)

12 and 10 = 8
12 or  10 = 14
12 xor 10 = 6
not 12    = -13

andalso and orelse short-circuit: once the answer is known the right-hand side is never evaluated. and, or, xor and not do not — they are bitwise operators that also serve for logic, and they always evaluate both sides.

The difference matters whenever the right-hand side has an effect, or would fail:

• if p <> 0 andalso p->count > 0 is safe.
• if p <> 0 and p->count > 0 dereferences a null pointer.

Use andalso and orelse for conditions, and reserve and/or/xor/not for bits.

Working with bits

'' bits.tiko -- shifting, masking, and the single-bit helpers.
dim as uint32 flags = 0

flags = bitset( flags, 0 )        '' turn bit 0 on
flags = bitset( flags, 3 )
print "flags        = " & flags & "  (binary " & bin( flags ) & ")"
print "bit 3 set?   = " & cbool( bit( flags, 3 ) <> 0 )
flags = bitreset( flags, 3 )
print "after reset  = " & bin( flags )
print

print "1 shl 8      = " & ( 1 shl 8 )
print "256 shr 4    = " & ( 256 shr 4 )
print

'' Picking halves out of a wider value.
dim as uint32 packed = &hDEADBEEF
print "packed       = " & hex( packed )
print "low word     = " & hex( loword( packed ) )
print "high word    = " & hex( hiword( packed ) )
print "low byte     = " & hex( lobyte( packed ) )

flags        = 9  (binary 1001)
bit 3 set?   = true
after reset  = 1

1 shl 8      = 256
256 shr 4    = 16

packed       = DEADBEEF
low word     = BEEF
high word    = DEAD
low byte     = EF

shl and shr shift; bitset, bitreset and bit turn one bit on, off, or read it; lobyte, hibyte, loword and hiword pull a piece out of a wider value. bin( ) and hex( ) print a value in a base where the bits are visible, which is most of the battle when debugging a mask.

Remember that shifts bind tighter than arithmetic. 1 shl n + 1 shifts by n and then adds — not what the spacing suggests.

Joining, choosing, and shorthand

'' expressions.tiko -- joining text, choosing a value, and shorthand.
dim as ustring who = "world"

'' '&' joins text. '+' also adds numbers, so '&' always means join.
print "hello, " & who
print "1 & 2 gives the text " & ( 1 & 2 )
print "1 + 2 gives the number " & ( 1 + 2 )
print

'' iif picks between two values inside an expression.
dim as int32 stock = 0
print "in stock: " & iif( stock > 0, "yes", "no" )
print

'' Compound assignment, and swapping two variables.
dim as int32 total = 10
total += 5 : total *= 2
print "total = " & total

dim as ustring left_ = "first", right_ = "second"
swap left_, right_
print left_ & " " & right_

hello, world
1 & 2 gives the text 12
1 + 2 gives the number 3

in stock: no

total = 30
second first

& joins text; + adds numbers. + will also join two strings, which is exactly why the book always uses &: 1 & 2 is the text 12, while 1 + 2 is the number 3, and & never has to be guessed at.

iif( condition, a, b ) chooses between two values inside an expression, where an if statement would need several lines. Both arms are evaluated, so do not use it where one of them has an effect.

Compound assignment — +=, -=, *=, /=, \=, &= — and swap a, b, which exchanges two variables of the same type without a temporary.

Summary

• / is floating point, \ is integer division, and ^ produces a float64.
• mod is a remainder and takes its left operand's sign; FloorMod in std/numeric is the mathematical one.
• ^ groups to the left, and shl/shr bind tighter than +.
• not is looser than the comparisons, and andalso/orelse share one level, grouping left to right — parenthesise mixed conditions.
• andalso/orelse short-circuit; and/or/xor/not are bitwise and always evaluate both.
• & joins text and never means anything else; iif chooses within an expression; swap exchanges two variables.

Exercises

1. Build a permissions value from three named bit positions, then read one back. Do it twice — once with bitset/bit, once with 1 shl n masks — and check the two agree. {solution="ex05-01"}
2. Write expressions where parentheses change the answer, and one where they only make the existing grouping obvious. Which of the four surprises above would have bitten you? {solution="ex05-02"}
3. 7 mod -2 and -7 mod 2 give different signs. Predict both before running them, then work out what FloorMod would give instead.
4. Rewrite if p <> 0 and p->count > 0 so it is safe, and explain in one sentence what the original does when p is null.

Control Flow, Scope and defer

Programs choose and repeat. tiko's constructs for both will look familiar, and the reason they get a chapter is the third thing here: defer, which arranges for something to happen on the way out of a block — and which has one rule that catches everybody once.

Choosing

'' branching.tiko -- if, and select case for more than two ways.
dim as int32 score = 72

if score >= 90 then
    print "excellent"
elseif score >= 60 then
    print "a pass"
else
    print "not yet"
end if

'' A whole if on one line needs no 'end if'.
if score > 0 then print "there is a score"

'' select case compares one value against ranges, lists and open tests.
select case score
case 90 to 100
    print "band: top"
case 60 to 89
    print "band: middle"
case 0, 1, 2
    print "band: barely started"
case is < 0
    print "band: impossible"
case else
    print "band: low"
end select

a pass
there is a score
band: middle

if / elseif / else / end if needs no parentheses around the condition. A complete if can sit on one line, and then it takes no end if — useful for a guard, and confusing if you try to extend it later, so keep those short.

select case compares one value against many possibilities, and the cases are more expressive than a C switch:

Cases are tested in order and the first match wins; there is no fall-through and nothing to break out of.

Repeating

'' loops.tiko -- counting, repeating, and leaving early.
for i as int32 = 1 to 3
    print "for " & i
next

for i as int32 = 10 to 0 step -5
    print "down " & i
next

dim as int32 n = 0
do while n < 2
    print "while " & n
    n += 1
loop

n = 0
do
    print "until " & n
    n += 1
loop until n = 2

'' exit leaves the loop; continue skips to the next pass.
for i as int32 = 1 to 6
    if i = 2 then continue for
    if i = 4 then exit for
    print "picked " & i
next

for 1
for 2
for 3
down 10
down 5
down 0
while 0
while 1
until 0
until 1
picked 1
picked 3

for … next counts, and step sets the stride — negative to count down. Declaring the counter in the loop, for i as int32 = 1 to 3, keeps it out of the surrounding scope, which is what you want almost always.

do … loop carries its condition at either end, with either sense:

• do while c … loop — test first, may run zero times
• do until c … loop — the same, inverted
• do … loop while c and do … loop until c — test last, so the body always runs once

exit for and exit do leave a loop; continue for and continue do skip to the next pass. Naming the loop kind makes nested loops readable.

An older form, while … wend, also exists and behaves as do while … loop. Existing code uses it; new code has no reason to.

Blocks that exist to bound a lifetime

'' scopeblock.tiko -- a block that exists only to bound a lifetime.
dim as int32 outer = 1

scope
    dim as int32 inner = 2
    defer print "  leaving the scope"
    print "  inside: outer " & outer & ", inner " & inner
end scope

'' 'inner' is gone here, and the defer has already run.
print "outside: outer " & outer

  inside: outer 1, inner 2
  leaving the scope
outside: outer 1

scope … end scope introduces a block with no condition and no loop — just a boundary. Variables declared inside it end there. It is how you keep a temporary from leaking into the rest of a long procedure, and, as the next section shows, how you control exactly when a defer fires.

defer

defer takes a statement and runs it when the enclosing block is left — however it is left:

'' deferorder.tiko -- defer runs on the way out, in reverse order.
sub Work( byval failEarly as boolean )
    print "  open A"
    defer print "  close A"

    print "  open B"
    defer print "  close B"

    if failEarly then
        print "  leaving early"
        exit sub
    end if

    print "  doing the work"
end sub

print "normal path:"
Work( false )
print
print "early exit:"
Work( true )

normal path:
  open A
  open B
  doing the work
  close B
  close A

early exit:
  open A
  open B
  leaving early
  close B
  close A

Two things to read off that output. Deferred statements run in reverse order, so resources are released in the opposite order to the one they were acquired in — B before A. And they run on every path out, including the early exit sub, which is the entire point: the cleanup is written next to the thing it cleans up, and cannot be skipped by a return you add later.

That makes defer the answer to the oldest bug in resource handling — the path that forgot to close something:

open "data.txt" for input as #1
defer close #1

Now the rule that catches everyone.

'' defertrap.tiko -- defer belongs to the nearest enclosing BLOCK.
sub Guarded( )
    print "  start"
    if true then
        print "    acquired"
        defer print "    released"    '' runs at end if, not end sub
        print "    still inside the if"
    end if
    print "  after the if -- already released"
end sub

sub EachIteration( )
    for i as int32 = 1 to 3
        defer print "    end of pass " & i
        print "  pass " & i
    next
end sub

print "a defer inside an if:"
Guarded( )
print
print "a defer inside a loop body:"
EachIteration( )

a defer inside an if:
  start
    acquired
    still inside the if
    released
  after the if -- already released

a defer inside a loop body:
  pass 1
    end of pass 1
  pass 2
    end of pass 2
  pass 3
    end of pass 3

defer needs a block to be on the way out of, so it cannot appear at module level:

'' modulelevel.tiko -- defer needs a scope to leave, and a module
'' body is not one. Does not build.
defer print "this never had a scope to leave"

print "hello"

modulelevel.tiko(3,1): error TKO0814: 'defer' is not allowed at module level; put it
in a procedure or a block

Labels and goto

tiko has goto and line labels. They exist for two reasons: compatibility with older code, and on error goto, which Chapter 18 covers.

Nothing in this book uses goto for control flow. Every loop construct above, plus exit and continue, covers what it was for — and a goto that jumps across a defer is refused (TKO0827), because the compiler cannot say what should have run on the way.

Summary

• if / elseif / else; a one-line if takes no end if.
• select case matches values, lists, ranges and open comparisons, first match wins, no fall-through. as const builds a jump table and takes only constants.
• A comparison yields an integer, not a boolean — wrap with cbool before comparing one against true.
• for … next with step; do … loop with the test at either end; exit and continue.
• scope … end scope bounds a lifetime deliberately.
• defer runs on the way out of its block, in reverse order, on every path — including the end of each loop pass, and including an if you wrapped it in.

Exercises

1. Write FizzBuzz for 1 to 15. Which reads better here, if/elseif or select case, and why? {solution="ex06-01"}
2. Write a procedure that acquires something inside an if and defers its release, and a second that guards first and defers in the procedure body. Print from both and compare the order. {solution="ex06-02"}
3. Put a defer in a loop body that prints the loop counter. Before running it, predict how many times it fires and with which values.
4. Rewrite a do … loop until as a do while … loop that behaves identically, including when the condition is already true at the start. Which needed more care?

Procedures

A procedure is the unit you give a name to, and naming a piece of work is most of what programming is. tiko's are ordinary — until you look at how arguments get in and results get out, where three things behave differently from what the syntax suggests.

Start with the reason parameters exist at all. A procedure cannot see the variables around it:

'' sealed.tiko -- a procedure cannot see the module's variables.
dim as int32 counter = 5

sub ShowIt( )
    print counter        '' error: procedures start with an empty slate
end sub

ShowIt( )

sealed.tiko(5,11): error TKO0743: 'counter' is not declared
     |
   5 |     print counter        '' error: procedures start with an empty slate
     |           ^

A module-level variable is visible to the module's own statements and to nothing else. Adding shared to the declaration would open it up to every procedure in the file, which is the reason this book almost never does: a procedure whose inputs all arrive as parameters can be read on its own, and tested on its own.

Subs, functions, and three ways to answer

'' basics.tiko -- a sub does something; a function answers something.

sub Banner( byref title as ustring )
    print "== " & title & " =="
end sub

'' Three ways to hand back a result. They compile to the same thing.
function Doubled( byval n as int32 ) as int32
    return n * 2
end function

function Tripled( byval n as int32 ) as int32
    function = n * 3
end function

function Quadrupled( byval n as int32 ) as int32
    Quadrupled = n * 4
end function

Banner( "results" )
print Doubled( 5 ), Tripled( 5 ), Quadrupled( 5 )

'' A function called for its answer and ignored is still called.
function Noisy( ) as int32
    print "Noisy ran"
    return 0
end function

Banner( "called for effect" )
Noisy( )

== results ==
 10            15            20
== called for effect ==
Noisy ran

A sub does something. A function answers something, and has three spellings for handing that answer back:

• return value — sets the result and leaves immediately.
• function = value — sets the result and carries on. Useful when the answer is refined further down.
• Name = value, the function's own name — the oldest form, and the one most likely to be misread as a recursive call.

They compile to the same thing. This book uses return, and function = only where the procedure really does keep working after the answer is settled.

A function may also be called and its answer discarded, as Noisy( ) is above — nothing forces you to use it.

The answer is usually a single value, but it can be a whole array: write ( ) after the result type, as in function Squares( ... ) as int32( ). Chapter 8 covers what such a function may return and who owns the result.

declare, for a name the compiler has not read yet

The compiler reads a file once, top to bottom, and a name has to exist before it is used. Two procedures that call each other cannot both be written first:

'' forward.tiko -- declare names a procedure before it exists.
''
'' IsEven and IsOdd call each other, so whichever is written first
'' refers to a name the compiler has not read yet.

declare function IsOdd( byval n as int32 ) as boolean

function IsEven( byval n as int32 ) as boolean
    if n = 0 then return true
    return IsOdd( n - 1 )
end function

function IsOdd( byval n as int32 ) as boolean
    if n = 0 then return false
    return IsEven( n - 1 )
end function

for i as int32 = 0 to 4
    print i; " even="; IsEven( i ); " odd="; IsOdd( i )
next

 0 even=true odd=false
 1 even=false odd=true
 2 even=true odd=false
 3 even=false odd=true
 4 even=true odd=false

declare states the signature and nothing else. Once it has been seen, the name can be called anywhere below, and the body can arrive whenever it likes. Without the declare, the call is TKO0743: '…' is not declared — the same error a typo gets, because to the compiler it is the same mistake.

How arguments get in

This is the part to read twice. tiko lets you write a parameter with no byval or byref, and what that means depends on the parameter's type:

'' passing.tiko -- what a parameter with no byval or byref means.
''
'' Every procedure here writes to its parameter. Whether the caller
'' sees that write is the whole question.

sub PlainInt( n as int32 )
    n = 99
end sub

sub PlainStr( s as ustring )
    s = "changed"
end sub

'' The same two, written the other way round.
sub ByrefInt( byref n as int32 )
    n = 99
end sub

sub ByvalStr( byval s as ustring )
    s = "changed"
end sub

dim as int32 a = 1
PlainInt( a )
print "int32   with no keyword: "; a

dim as ustring t = "kept"
PlainStr( t )
print "ustring with no keyword: " & t

dim as int32 b = 1
ByrefInt( b )
print "int32   byref:           "; b

dim as ustring u = "kept"
ByvalStr( u )
print "ustring byval:           " & u

int32   with no keyword:  1
ustring with no keyword: changed
int32   byref:            99
ustring byval:           kept

With no keyword, an int32 was copied — the caller still holds 1. A ustring was shared — the caller's text changed under it. Both procedures are written identically.

The rule the compiler applies:

byref is not a wart to be avoided — it is how you pass something large without copying it, and how a procedure reports more than one answer. Exercise 1 builds a MinMax that returns two values that way, which is the tool to reach for until Chapter 20 introduces tuples.

const, for a reference you promise not to write through

byref gives speed and gives up safety: the callee can now write to the caller's variable. Adding const takes the safety back while keeping the speed.

'' constparam.tiko -- const says the procedure will not write to it.
sub Bump( byval n as const int32 )
    n += 1               '' error: that is what const forbids
end sub

Bump( 1 )

constparam.tiko(3,5): error TKO0617: cannot modify 'n': it is a constant
     |
   3 |     n += 1               '' error: that is what const forbids
     |     ^

byref x as const T is the right shape for any parameter you only intend to read, and it is documentation the compiler checks — for every type, text included: byref s as const ustring may not be assigned either.

Arguments you can leave out

'' optional.tiko -- defaults, and leaving an argument out of the middle.

sub Label( byref who as ustring, byval times as int32 = 1, _
           byref mark as ustring = "!" )
    for i as int32 = 1 to times
        print who & mark
    next
end sub

Label( "once" )
Label( "twice", 2 )
Label( "marked", , "?" )      '' times keeps its default

once!
twice!
twice!
marked?

A parameter with = value may be omitted at the call. Optional parameters must come after the required ones, and an argument in the middle can be skipped by leaving its slot empty — Label( "marked", , "?" ) takes the default for times and supplies mark.

The empty slot is easy to misread. Two of them in a row, Label( "x", , ), is legal and tells the reader nothing; when a call needs more than one hole, the parameters are in the wrong order.

One name, several signatures

'' overloads.tiko -- one name, several signatures.

declare function Area overload ( byval side as float64 ) as float64
declare function Area overload ( byval w as float64, _
                                byval h as float64 ) as float64
declare function Area overload ( byref tag as ustring ) as float64

function Area overload ( byval side as float64 ) as float64
    print "  (square)"
    return side * side
end function

function Area overload ( byval w as float64, _
                         byval h as float64 ) as float64
    print "  (rectangle)"
    return w * h
end function

function Area overload ( byref tag as ustring ) as float64
    print "  (by name: " & tag & ")"
    return 0.0
end function

'' Each call prints which overload ran, then its answer.
var square = Area( 3.0 )
print "Area( 3.0 )      ="; square

var rect = Area( 2.0, 5.0 )
print "Area( 2.0, 5.0 ) ="; rect

var widened = Area( 3 )
print "Area( 3 )        ="; widened

var named = Area( "lot" )
print "Area( ""lot"" )    ="; named

  (square)
Area( 3.0 )      = 9
  (rectangle)
Area( 2.0, 5.0 ) = 10
  (square)
Area( 3 )        = 9
  (by name: lot)
Area( "lot" )    = 0

overload says that a name is deliberately shared. It is required: declaring a second Area without it is TKO0600: 'area' is already declared in this scope, the same error as any duplicate name — which is the right default, because most repeated names are accidents.

Note what happened to Area( 3 ). There is no int32 overload, so the argument was converted to float64 and the one-parameter version ran. The compiler scores the conversions each candidate would need and takes the cheapest total. When two candidates tie, it refuses rather than guessing:

'' ambiguous.tiko -- neither overload is a better match than the other.
declare function Pick overload ( byval a as int32, _
                                byval b as float64 ) as int32
declare function Pick overload ( byval a as float64, _
                                byval b as int32 ) as int32

function Pick overload ( byval a as int32, _
                         byval b as float64 ) as int32
    return 1
end function

function Pick overload ( byval a as float64, _
                         byval b as int32 ) as int32
    return 2
end function

print Pick( 1, 1 )       '' error: int32 converts to float64 either way

ambiguous.tiko(17,11): error TKO0660: the call to 'pick' matches more than one overload
     |
  17 | print Pick( 1, 1 )       '' error: int32 converts to float64 either way
     |           ^

Scoring counts conversions; it does not rank them by whether they can lose information. Exercise 2 shows an int64 argument picking an int32 overload — narrowing — in preference to widening to float64. If a width matters, give it an overload of its own.

Recursion, and locals that remember

'' gcd.tiko -- recursion, and a static local that outlives the call.

function Gcd( byval a as int64, byval b as int64 ) as int64
    static as int32 calls = 0
    calls += 1

    if b = 0 then
        print "  reached the base case after"; calls; " calls"
        calls = 0
        return a
    end if

    return Gcd( b, a mod b )
end function

print "Gcd( 1071, 462 ) ="; Gcd( 1071, 462 )
print "Gcd( 270, 192 )  ="; Gcd( 270, 192 )

Gcd( 1071, 462 ) =  reached the base case after 4 calls
 21
Gcd( 270, 192 )  =  reached the base case after 5 calls
 6

A procedure may call itself; each call gets its own parameters and its own locals. static is the exception — a static local is created once, keeps its value between calls, and so is shared by every level of a recursion. That is what makes it usable as a call counter, and it is also why Gcd has to reset it before returning: the count belongs to one top-level call, and nothing else will clear it.

static initialisers run once, on the first call. A static is the smallest thing that gives a procedure memory, and the smallest thing that stops it being safe to call from two threads at once — Chapter 23 returns to that.

Returning a place instead of a value

A function can hand back a reference to storage, which the caller may then assign to. Write byref after the parameter list:

'' byrefresult.tiko -- a function that returns a place, not a value.

function SlotAt( byval i as int32 ) byref as int32
    static as int32 first  = 0
    static as int32 second = 0
    if i = 0 then return first
    return second
end function

print "start:            "; SlotAt( 0 ); SlotAt( 1 )

'' The obvious spelling. Watch what it does.
SlotAt( 0 ) = 42
print "after SlotAt(0)=42:"; SlotAt( 0 ); SlotAt( 1 )

'' Parenthesise the call, and the assignment lands.
( SlotAt( 0 ) ) = 42
print "after (SlotAt(0))=42:"; SlotAt( 0 ); SlotAt( 1 )

'' Or bind the reference to a name first, which reads better.
dim byref as int32 slot = SlotAt( 1 )
slot = 7
print "after slot = 7:   "; SlotAt( 0 ); SlotAt( 1 )

start:             0 0
after SlotAt(0)=42: 0 0
after (SlotAt(0))=42: 42 0
after slot = 7:    42 7

Read the second line of that output before the third.

( SlotAt( 0 ) ) = 42                        '' parenthesise the call

dim byref as int32 slot = SlotAt( 1 )       '' or bind it to a name
slot = 7

Any number of arguments

A Sum that adds two numbers is easy, and so is one that adds three. A Sum that adds however many it is given is the one you actually want, and overloads cannot get there: there would have to be one per count. An array parameter can, but then every caller builds an array before every call.

A typed variadic is the array parameter with the caller's convenience:

'' typedvariadic.tiko -- any number of arguments, every one typed.
''
'' The '...' after the type makes nums an array parameter that the
'' caller fills with a plain list. Each argument is converted to
'' int32 exactly as it would be for an ordinary int32 parameter.

function Sum( nums() as const int32 ... ) as int64
    dim as int64 total = 0
    for i as int32 = lbound( nums ) to ubound( nums )
        total += nums( i )
    next
    return total
end function

'' Fixed parameters come first; the variadic is always the last.
sub Report( byref title as const ustring, _
            nums() as const int32 ... )
    print title; ":"; ubound( nums ) - lbound( nums ) + 1; _
          " values, sum"; Sum( nums( ) )
end sub

print "Sum( 10, 20, 30 ) ="; Sum( 10, 20, 30 )
print "Sum( 7 )          ="; Sum( 7 )
print "Sum( )            ="; Sum( )

Report( "three", 1, 2, 3 )
Report( "none" )

'' A whole array of the element type IS the list -- nothing is
'' copied, and no count has to be passed alongside it.
dim as int32 week( 0 to 6 ) = { 3, 5, 2, 8, 6, 4, 7 }
Report( "week", week( ) )

Sum( 10, 20, 30 ) = 60
Sum( 7 )          = 7
Sum( )            = 0
three: 3 values, sum 6
none: 0 values, sum 0
week: 7 values, sum 35

nums() as const int32 ... is an array parameter with ... after its type. The caller writes a plain list. The compiler converts each argument to int32, exactly as it would for an ordinary int32 parameter, and packs them into a temporary array that lives until the end of the statement. Inside, nums is an ordinary array: lbound is 0, and ubound is -1 when nothing was passed, so the loop above needs no special case for Sum( ).

Fixed parameters come first and the variadic is always last — anywhere else is TKO0973. It takes no default value (TKO0974), because no arguments already means an empty array.

A whole array of the element type is the list itself. Report( "week", week( ) ) passes week as it stands: nothing is copied and no count travels alongside it, because the array carries its own bounds. That is the form to use when the values are already in an array, or when the list is built while the program runs.

Every argument is checked, which is the whole difference from the C mechanism further down:

'' variadictyped.tiko -- every packed argument is checked.

function Sum( nums() as const int32 ... ) as int64
    dim as int64 total = 0
    for i as int32 = lbound( nums ) to ubound( nums )
        total += nums( i )
    next
    return total
end function

print Sum( 1, "two", 3 )    '' error: "two" is no int32

variadictyped.tiko(11,15): error TKO0698: this argument does not convert to the parameter's type
     |
  11 | print Sum( 1, "two", 3 )    '' error: "two" is no int32
     |               ^

It is the same error a non-converting argument to an ordinary parameter gets, at the argument that caused it.

An overload that takes its arguments without packing them always wins over one that needs to:

'' variadicpick.tiko -- an exact match beats a packed one.

sub Show overload ( byval n as int32 )
    print "one int32:"; n
end sub

sub Show overload ( nums() as const int32 ... )
    print "a list of"; ubound( nums ) - lbound( nums ) + 1
end sub

Show( 5 )            '' the plain overload: no packing needed
Show( 5, 6 )         '' only the list can take two
Show( )              '' ... or none

one int32: 5
a list of 2
a list of 0

So adding a variadic overload never changes which procedure an existing call runs. Among candidates that all pack, each packed argument is scored like any other argument.

'' variadicalias.tiko -- a packed list is a copy; a spread array
'' is the caller's own.

sub Bump( nums() as int32 ... )      '' no const: it may write
    for i as int32 = lbound( nums ) to ubound( nums )
        nums( i ) += 100
    next
end sub

dim as int32 a = 1, b = 2
Bump( a, b )                         '' packed into a temporary
print "after Bump( a, b ):    "; a; b

dim as int32 both( 0 to 1 ) = { 1, 2 }
Bump( both( ) )                      '' the array itself
print "after Bump( both( ) ): "; both( 0 ); both( 1 )

after Bump( a, b ):     1 2
after Bump( both( ) ):  101 102

The element type can be one of your own, and the conversion then goes through that type's constructors — Chapter 12. That is how strings.Format in Chapter 9 takes values of every type in one list: its element type has a constructor for each.

C's ...

'' varargs.tiko -- a procedure taking any number of arguments.
''
'' This is C's mechanism, so it needs C's calling convention and
'' none of the type checking you get everywhere else.

#include once "crt/stdarg.bi"

function SumOf cdecl ( byval count as int32, ... ) as int64
    dim as cva_list args
    dim as int64 total = 0

    cva_start( args, count )
    for i as int32 = 1 to count
        total += cva_arg( args, int32 )
    next
    cva_end( args )

    return total
end function

print "SumOf( 3, 10, 20, 30 ) ="; SumOf( 3, 10, 20, 30 )
print "SumOf( 1, 7 )          ="; SumOf( 1, 7 )
print "SumOf( 0 )             ="; SumOf( 0 )

SumOf( 3, 10, 20, 30 ) = 60
SumOf( 1, 7 )          = 7
SumOf( 0 )             = 0

A bare ... as the last parameter, with no name and no type, is C's mechanism, with C's consequences: the procedure must be cdecl, there is no type checking, and the count is recorded nowhere — SumOf is told how many to read because there is no other way for it to find out. Reading one argument too many reads whatever happened to be next.

Declaring the procedure without cdecl is refused: TKO0781: '...' needs a 'cdecl' procedure, and this one is 'stdcall'.

C's variadics exist so that C libraries can be called; printf is the reason the machinery is there. In tiko code, use a typed variadic. Chapter 29 covers cva_list in full, alongside the rest of C interoperation.

Procedures as values

'' callbacks.tiko -- a procedure held in a variable.

function Twice( byval x as float64 ) as float64
    return x * 2.0
end function

function Square( byval x as float64 ) as float64
    return x * x
end function

'' The parameter type names a shape, not a procedure.
sub Tabulate( byref title as ustring, _
              byval f as function( byval as float64 ) as float64 )
    print title
    for i as int32 = 1 to 3
        print "  f("; i; " ) ="; f( i )
    next
end sub

Tabulate( "doubling", @Twice )
Tabulate( "squaring", procptr( Square ) )

'' A variable of that type can be reassigned like any other.
dim as function( byval as float64 ) as float64 op = @Twice
print "op( 4 ) ="; op( 4.0 )
op = @Square
print "op( 4 ) ="; op( 4.0 )

doubling
  f( 1 ) = 2
  f( 2 ) = 4
  f( 3 ) = 6
squaring
  f( 1 ) = 1
  f( 2 ) = 4
  f( 3 ) = 9
op( 4 ) = 8
op( 4 ) = 16

A procedure has a type, spelled the way it is declared minus the names: function( byval as float64 ) as float64. A variable or parameter of that type holds any procedure matching the shape, and is called with ordinary call syntax.

Two spellings get the address: @Twice and procptr( Twice ). They are the same thing, and procptr is the one to use for an overloaded name, where the signature picks which one you meant.

A procedure pointer is the plainest form of the idea that the caller decides what to do and the procedure decides when. Chapter 21 extends it to lambdas, which add the ability to capture variables from around them — the one thing a procedure pointer cannot do, because a pointer is a single address with no room for anything else.

What it costs

A call is cheap and not free: arguments are placed, a return address is pushed, the callee may save registers. None of that matters at ordinary granularity, and all of it matters in a loop running millions of times.

The costs worth knowing are the ones you choose:

• byval on a large value copies it, every call. That is what byref ... as const is for.
• byref on a small value adds an indirection to save copying eight bytes. It is a loss.
• A procedure pointer cannot be inlined, because the target is not known until it runs. A direct call can be.
• A packed variadic call builds a temporary array, every call. Passing an array you already have costs nothing extra.
• static locals make a procedure stateful, which is a correctness cost before it is ever a performance one.

Summary

• A procedure sees only what it is given; module-level variables need shared, which is rarely worth it.
• return, function = and assigning the procedure's own name are three spellings of one thing. A function that sets no result returns zero, silently.
• declare introduces a name before its body, which mutual recursion requires.
• With no keyword, numbers are passed byval, and strings and your own types byref. Write the keyword on every parameter.
• byref x as const T is a reference you cannot write through, whatever T is.
• Optional parameters follow the required ones, and a middle argument can be left empty.
• overload is required to share a name; resolution scores conversions and refuses ties with TKO0660. It does not avoid narrowing.
• static locals persist between calls and are shared by every level of a recursion.
• Assigning to a one-argument byref function result needs ( f( x ) ) = v or a dim byref name — the bare form parses as an equality test, and TKO0931 warns that it did.
• A typed variadic, nums() as const T ..., takes any number of arguments, each converted to T and checked; the procedure sees an ordinary array. A whole array of T is the list itself, with no copy.
• An overload that needs no packing beats one that does. A packed list is a copy and a spread array is the caller's own — declare the variadic const.
• C's bare ... needs cdecl and does no type checking; it is for calling C.
• Procedure pointers hold any procedure of a matching shape; @P and procptr( P ) are the same.
• The costs you choose: a byval copy of something large, a byref indirection to save eight bytes, a procedure pointer that cannot be inlined, and a static that makes a procedure stateful.

Exercises

1. Write MinMax, which takes three int32 values and reports the smallest and largest through byref parameters. Which of its five parameters are byval and which byref, and what happens if you omit the keywords entirely? {solution="ex07-01"}
2. Overload Describe for int32, float64 and ustring, then call it with an int16 and an int64. Predict which overload each picks before you run it. {solution="ex07-02"}
3. Write a function with a byref result, then assign to it with the bare f( i ) = v form. Before running the program, say what you expect; afterwards, explain what the compiler actually compiled.
4. Give a recursive procedure a static call counter and don't reset it. Call the procedure twice and explain the second number.
5. Take Tabulate from callbacks.tiko and add a third operation without touching Tabulate. Then do the same with an overload instead of a procedure pointer. Which change would you rather review?
6. Write Average( xs() as const float64 ... ). Decide what Average( ) should return, and make it do that. Then pass it a whole int32 array: why is that refused, when Average( 1, 2, 3 ) with int32 arguments is not?

Arrays

An array is many values of one type under one name, reached by number. It is the oldest data structure there is and still the one to reach for first: the elements sit next to each other in memory, so walking an array is the fastest thing a processor does.

tiko's arrays carry their bounds with them — not a length, but a first and last index, which the compiler and the runtime both know. That one decision explains most of what follows.

Bounds, not length

'' bounds.tiko -- an array's bounds are part of its declaration.

dim as int32 a( 10 )           '' how many elements is that?
dim as int32 b( 1 to 5 )
dim as int32 c( -2 to 2 )

print "a( 10 )      "; lbound( a ); " to"; ubound( a ); _
      "  elements:"; ubound( a ) - lbound( a ) + 1
print "b( 1 to 5 )  "; lbound( b ); " to"; ubound( b ); _
      "  elements:"; ubound( b ) - lbound( b ) + 1
print "c( -2 to 2 ) "; lbound( c ); " to"; ubound( c ); _
      "  elements:"; ubound( c ) - lbound( c ) + 1

'' Every element starts at zero. Nothing is left uninitialised.
print "a( 7 ) before anything is written:"; a( 7 )

'' An initialiser gives the values in order.
dim as int32 d( 1 to 3 ) = { 10, 20, 30 }
print "d ="; d( 1 ); d( 2 ); d( 3 )

a( 10 )       0 to 10  elements: 11
b( 1 to 5 )   1 to 5  elements: 5
c( -2 to 2 ) -2 to 2  elements: 5
a( 7 ) before anything is written: 0
d = 10 20 30

lbound( a ) and ubound( a ) are the first and last valid index. Every loop over an array in this book is written between them, and never between 0 and some remembered count.

Elements start at zero — or empty, for a string — so there is no such thing as an uninitialised array element. An initialiser in braces fills them in order.

A lower bound other than 0 is worth using when the index means something. band( 0 to 9 ) in the histogram below is indexed by tens, and temperature( 1900 to 2000 ) indexed by year needs no arithmetic at the point of use, which is where arithmetic goes wrong.

Arrays that change size

'' dynamic.tiko -- an array whose size is decided while running.

dim as int32 a( )              '' declared, but with no room yet

print "empty:  lbound="; lbound( a ); " ubound="; ubound( a )
print "  ubound < lbound is how an empty array reads"

redim a( 1 to 3 )
a( 1 ) = 10 : a( 2 ) = 20 : a( 3 ) = 30
print "filled:"; a( 1 ); a( 2 ); a( 3 )

'' redim on its own does NOT keep what was there.
redim a( 1 to 5 )
print "after redim( 1 to 5 ):";
for i as int32 = lbound( a ) to ubound( a ) : print a( i ); : next
print

redim a( 1 to 3 )
a( 1 ) = 10 : a( 2 ) = 20 : a( 3 ) = 30
redim preserve a( 1 to 5 )
print "after redim preserve:";
for i as int32 = lbound( a ) to ubound( a ) : print a( i ); : next
print

'' Shrinking with preserve keeps the elements that still fit.
redim preserve a( 1 to 2 )
print "after shrinking:";
for i as int32 = lbound( a ) to ubound( a ) : print a( i ); : next
print

erase a
print "after erase: lbound="; lbound( a ); " ubound="; ubound( a )

empty:  lbound= 0 ubound=-1
  ubound < lbound is how an empty array reads
filled: 10 20 30
after redim( 1 to 5 ): 0 0 0 0 0
after redim preserve: 10 20 30 0 0
after shrinking: 10 20
after erase: lbound= 0 ubound=-1

dim as int32 a( ) declares an array with no room yet. Its ubound is below its lbound, which is how an empty array reads — and why for i = lbound( a ) to ubound( a ) over an empty array runs zero times instead of needing a guard.

redim gives it a size. redim preserve gives it a size and keeps what was there, growing with zeros and shrinking by truncation.

erase does two different things, and which one depends on the array:

'' erasefixed.tiko -- erase means two different things.

dim as int32 fixed_( 1 to 3 ) = { 7, 8, 9 }
dim as int32 grown( )
redim grown( 1 to 3 )
grown( 1 ) = 7 : grown( 2 ) = 8 : grown( 3 ) = 9

erase fixed_
erase grown

print "fixed after erase: ubound="; ubound( fixed_ ); _
      "  values:"; fixed_( 1 ); fixed_( 2 ); fixed_( 3 )
print "dynamic after erase: ubound="; ubound( grown ); _
      "  (the storage is gone)"

fixed after erase: ubound= 3  values: 0 0 0
dynamic after erase: ubound=-1  (the storage is gone)

On a dynamic array, erase frees the storage and leaves it empty. On a fixed array there is nothing to free, so it resets the elements and the bounds stay. One keyword, two meanings, neither surprising once you know which array you are holding.

An array indexed by what it counts

'' histogram.tiko -- an array indexed by the thing you are counting.

'' Sixteen measurements, each a score from 0 to 100.
dim as int32 scores( 1 to 16 ) = { 93, 71, 55, 88, 64, 79, 100, 42, _
                                   67, 85, 90, 73, 58, 96, 81, 77 }

'' One counter per band of ten. The bounds say what the index MEANS:
'' band 4 counts 40-49. A perfect score is clamped into the top band.
dim as int32 band( 0 to 9 )

for i as int32 = lbound( scores ) to ubound( scores )
    dim as int32 b = scores( i ) \ 10
    if b > ubound( band ) then b = ubound( band )
    band( b ) += 1
next

for b as int32 = ubound( band ) to lbound( band ) step -1
    dim as ustring bar = ""
    for n as int32 = 1 to band( b ) : bar &= "#" : next
    print b * 10; "-"; b * 10 + 9; " "; bar
next

 90- 99 ####
 80- 89 ###
 70- 79 ####
 60- 69 ##
 50- 59 ##
 40- 49 #
 30- 39 
 20- 29 
 10- 19 
 0- 9 

This is the shape worth recognising: the index is the data. band( score \ 10 ) needs no search and no comparison chain — the value computes its own slot, and counting is one addition.

The bounds document the meaning. band( 0 to 9 ) says there are ten bands of ten; the clamp says what happens to a perfect score. Both are visible at the declaration rather than implied by a loop somewhere below.

More than one subscript

'' matrix.tiko -- more than one subscript, and how it sits in memory.

dim as int32 m( 0 to 1, 0 to 2 )

dim as int32 n = 0
for r as int32 = lbound( m, 1 ) to ubound( m, 1 )
    for c as int32 = lbound( m, 2 ) to ubound( m, 2 )
        m( r, c ) = n
        n += 1
    next
next

print "dimension 1:"; lbound( m, 1 ); " to"; ubound( m, 1 )
print "dimension 2:"; lbound( m, 2 ); " to"; ubound( m, 2 )

print "as a grid:"
for r as int32 = lbound( m, 1 ) to ubound( m, 1 )
    print "  ";
    for c as int32 = lbound( m, 2 ) to ubound( m, 2 )
        print m( r, c );
    next
    print
next

'' The same storage, read as one flat run of int32.
dim as int32 ptr p = @m( 0, 0 )
print "in memory:";
for i as int32 = 0 to 5
    print p[ i ];
next
print

dimension 1: 0 to 1
dimension 2: 0 to 2
as a grid:
   0 1 2
   3 4 5
in memory: 0 1 2 3 4 5

Dimensions are separated by commas, and lbound/ubound take the dimension number as a second argument: ubound( m, 2 ) is the last column index. An array may have up to eight dimensions.

The last line of that output is the one to keep. Reading the same storage as a flat run of int32 gives 0 1 2 3 4 5 — the order the elements were written, row by row. The last subscript varies fastest. A row is contiguous; a column is strided.

That is not a detail of the implementation you are free to ignore, because it decides how fast the loop runs. Iterating with the last subscript innermost walks memory forwards and every cache line that is fetched gets used. Swapping the loops reads one element per cache line, and on a large array that is the difference between fast and slow, with identical source.

Filling an array with braces

'' braces.tiko -- a brace list fills an array: one value per element,
'' and one level of braces per dimension.

dim as int32 primes( 0 to 4 ) = { 2, 3, 5, 7, 11 }
dim as int32 grid( 0 to 1, 0 to 2 ) = { { 1, 2, 3 }, { 4, 5, 6 } }

'' '...' as the upper bound: the list decides how many there are.
dim as int32 squares( 1 to ... ) = { 1, 4, 9, 16 }

print "primes:";
for i as int32 = lbound( primes ) to ubound( primes )
    print primes( i );
next
print

print "grid, row 1:";
for c as int32 = lbound( grid, 2 ) to ubound( grid, 2 )
    print grid( 1, c );
next
print

print "squares:"; ubound( squares ); " of them, the last";
print squares( ubound( squares ) )

primes: 2 3 5 7 11
grid, row 1: 4 5 6
squares: 4 of them, the last 16

A brace list gives the elements in index order, from lbound up. An array with more than one dimension takes one level of braces per dimension, in the order the last section described: the first inner list is row 0, so { { 1, 2, 3 }, { 4, 5, 6 } } puts 4 5 6 in row 1.

... as the upper bound lets the list decide the size. squares( 1 to ... ) has four elements because four values were given, so adding a fifth value cannot leave the bounds behind.

Braces are for arrays, and only at the declaration. A single value takes = and the value. A brace list on anything that is not an array is refused:

'' bracescalar.tiko -- a brace list initializes an ARRAY. A single
'' value takes '=' and the value.

dim as int32 total = { 0 }          '' refused
print total

bracescalar.tiko(4,22): error TKO0845: 'total' is not an array, so it cannot take a '{ ... }' initializer
     |
   4 | dim as int32 total = { 0 }          '' refused
     |                      ^

A brace list is not a value either, so it cannot be assigned later:

'' bracelater.tiko -- braces belong to the declaration. Filling an
'' array again is a loop, or one element at a time.

dim as int32 scores( 0 to 2 ) = { 70, 85, 90 }
scores = { 0, 0, 0 }                '' refused
print scores( 0 )

bracelater.tiko(5,10): error TKO0846: a '{ ... }' list can only initialize an array, and nothing here is one
     |
   5 | scores = { 0, 0, 0 }                '' refused
     |          ^

To refill an array, loop over it, or assign the elements one at a time. A container from the standard library is the one other thing a brace list can fill; Chapter 17 shows how.

fbc refuses both lines too, with its errors 63 and 73.

Passing an array to a procedure

'' params.tiko -- an array argument carries its bounds with it.

sub FillWith( a( ) as int32, byval v as int32 )
    for i as int32 = lbound( a ) to ubound( a )
        a( i ) = v
    next
end sub

function SumOf( a( ) as int32 ) as int64
    dim as int64 total = 0
    for i as int32 = lbound( a ) to ubound( a )
        total += a( i )
    next
    return total
end function

'' A parameter's rank is written with commas, and nothing else.
sub Describe2D( m( , ) as int32 )
    print "  rows"; lbound( m, 1 ); " to"; ubound( m, 1 ); _
          ", columns"; lbound( m, 2 ); " to"; ubound( m, 2 )
end sub

dim as int32 a( 5 to 8 )
FillWith( a( ), 3 )
print "bounds inside were"; lbound( a ); " to"; ubound( a )
print "sum ="; SumOf( a( ) )

dim as int32 grid( 1 to 2, 0 to 3 )
print "Describe2D saw:"
Describe2D( grid( ) )

bounds inside were 5 to 8
sum = 12
Describe2D saw:
  rows 1 to 2, columns 0 to 3

A parameter is written a( ) as int32 and the argument is written a( ) — the empty parentheses are required at both ends. An array is always passed by reference; there is no copy, and writes inside the procedure are writes to the caller's array.

The bounds travel with it. FillWith was given an array indexed 5 to 8 and saw exactly that, which is why a procedure that works from lbound and ubound works on any array of the right type without being told anything else.

A multi-dimensional parameter states its rank with commas and nothing more: m( , ) as int32 takes any two-dimensional int32 array. Rank is fixed at compile time; bounds are not.

Returning an array

A function that builds a table has to give it to its caller somehow, and the old answers are both awkward. The caller can create the array first and let the function fill it in, which means the caller must know the size before the function has decided it. Or the array can be wrapped in a container just to cross one return. tiko lets a function return the array itself: write ( ) after the result type.

'' returning.tiko -- a function can hand back a dynamic array.

'' A brace list: the array is sized to the list and filled.
function FirstPrimes( ) as int32( )
    return { 2, 3, 5, 7, 11 }
end function

'' A local array, built in a loop and moved out. Nothing is copied,
'' and the bounds the function chose travel with it.
function Squares( byval n as int32 ) as int32( )
    dim r( ) as int32
    redim r( 1 to n )
    for i as int32 = 1 to n
        r( i ) = i * i
    next
    return r
end function

'' Any element type works.
function Greetings( ) as ustring( )
    return { "hello", "bonjour", "hola" }
end function

sub ShowSum( a( ) as int32 )
    dim as int32 total = 0
    for i as int32 = lbound( a ) to ubound( a )
        total += a( i )
    next
    print "sum of " & lbound( a ) & " to " & ubound( a ) & " = " & total
end sub

'' var takes the result as it is: type, bounds and all.
var p = FirstPrimes( )
print "primes " & lbound( p ) & " to " & ubound( p ) & _
      ", p(4) = " & p( 4 )

var sq = Squares( 5 )
print "squares " & lbound( sq ) & " to " & ubound( sq ) & _
      ", sq(5) = " & sq( 5 )

'' A result can go straight to a parameter that takes an array.
ShowSum( Squares( 4 ) )

'' for each walks a result, and the array is freed afterwards.
for each g as ustring in Greetings( )
    print g
next

'' Assigning moves the new array in and frees the old one.
dim as int32 m( )
m = FirstPrimes( )
m = Squares( 3 )
print "m is now " & lbound( m ) & " to " & ubound( m )

primes 0 to 4, p(4) = 11
squares 1 to 5, sq(5) = 25
sum of 1 to 4 = 30
hello
bonjour
hola
m is now 1 to 3

The result is an ordinary dynamic array. It is what dim a( ) makes, it is indexed the same way, and a parameter written a( ) accepts it. It is not a container. Only an empty ( ) means an array: as int32( 3 ) is refused (TKO0836), where it used to be quietly read as a plain int32.

return takes three things:

• A brace list. The array is sized to the list, indexed from 0, and filled. A list inside a list is refused (TKO0839); write it flat. Any element type works, and each element is built once.
• A local array, as Squares does. It is moved, not copied: the function's own variable is left empty and the caller receives the array it built, bounds included. sq runs from 1 to 5 because Squares said so.
• A call that returns an array, handed straight through.

function = value takes the same three and carries on afterwards, as it does for any result. A function that never sets one returns an empty array: ubound is −1 and a loop over it runs zero times.

The caller owns what comes back. var p = FirstPrimes( ) takes the array itself. m = Squares( 3 ) frees what m held and moves the new array in. ShowSum( Squares( 4 ) ) and for each over a call use the array and free it when the statement ends, and a result nobody uses is freed there too. Two calls never share a buffer, so there is nothing to release by hand.

Only an array that belongs to the function can be moved out. A parameter, a shared or static array and a field all belong to somebody else: handing one back would leave its owner pointing at freed memory, and the other way to do it is a copy, which tiko does not make for you yet.

'' returnparam.tiko -- an array that is not the function's own cannot
'' be handed back: its owner would be left holding freed memory.

function Same( a( ) as int32 ) as int32( )
    return a
end function

returnparam.tiko(5,5): error TKO0840: 'a' is not a local array of this procedure, so returning it would need a copy, which is not supported yet; return a local array or a brace list
     |
   5 |     return a
     |     ^

To hand back a copy, build a local array element by element and return that.

The same care applies at the call. The name of a function that returns an array needs its parentheses, so var b = FirstPrimes is TKO0841 rather than a call. An array is not a number, so dim as int32 x = FirstPrimes( ) is TKO0842. And dim a( ) as int32 = FirstPrimes( ) is TKO0612, as it is for any initialised dynamic array: var takes the result, dim does not.

Walking it

'' walking.tiko -- three ways across an array, and what each can do.

dim as int32 a( 1 to 4 ) = { 10, 20, 30, 40 }

print "by index:";
for i as int32 = lbound( a ) to ubound( a )
    print a( i );
next
print

print "for each:";
for each v as int32 in a
    print v;
next
print

'' for each hands you a COPY. Writing to it changes nothing.
for each v as int32 in a
    v = 0
next
print "after writing to the copy:";
for each v as int32 in a : print v; : next
print

'' for each byref hands you the element itself.
for each byref v as int32 in a
    v *= 2
next
print "after for each byref:";
for each v as int32 in a : print v; : next
print

by index: 10 20 30 40
for each: 10 20 30 40
after writing to the copy: 10 20 30 40
after for each byref: 20 40 60 80

Three ways across, and the difference between the last two matters:

• By index, for i = lbound( a ) to ubound( a ). You have the position as well as the value, which you need whenever the answer depends on where you are.
• for each v as T in a hands you a copy of each element. Writing to v changes nothing — the listing proves it.
• for each byref v as T in a hands you the element itself, so writes land in the array.

Use for each when you only need the values, and byref when you mean to modify them. The byref form also avoids copying each element, which matters when the elements are large.

for each will not walk a grid:

'' foreach2d.tiko -- there is no single order to walk a grid in.
dim as int32 m( 0 to 1, 0 to 2 )

for each v as int32 in m        '' refused: which way, rows or columns?
    print v;
next

foreach2d.tiko(4,1): error TKO0608: 'for each' over a multi-dimensional array has no defined order
     |
   4 | for each v as int32 in m        '' refused: which way, rows or columns?
     | ^

The compiler could have picked row-major and said nothing. Refusing is the better answer: an iteration order over a grid is a decision the code should state, not one to inherit silently from the memory layout.

When the index is wrong

An index outside the bounds reads or writes memory that belongs to something else. Chapter 1 introduced -exx, which adds a check to every array access and stops the program with runtime error 6 rather than letting it continue with nonsense.

The check covers a dynamic array as well as a fixed one. Its bounds are not in the source, so the check reads them from the array's descriptor at run time, on every access, and a redim that shrinks the array moves the limit with it:

'' exxdynamic.tiko -- -exx checks a dynamic array too: its bounds are
'' read from the descriptor, at run time, on every access.
#cmdline "-exx"

dim as int32 a( )
redim a( 1 to 3 )
a( 3 ) = 30                     '' fine: 3 is inside 1 to 3

redim preserve a( 1 to 2 )      '' the limit moves with the array
a( 3 ) = 42                     '' stops here, with runtime error 6
print "not reached"

Aborting due to runtime error 6 (out of bounds array access) at line 10 of exxdynamic.tiko::(), 
'A' accessed with invalid index = 3, must be between 1 and 2

The program stops with exit code 6 at the first bad write, before it can corrupt anything.

In a build without -exx, the check you write yourself is the one you can rely on:

'' guard.tiko -- checking an index yourself, which is always available.

function ValueAt( a( ) as int32, byval i as int32, _
                  byval fallback as int32 ) as int32
    if i < lbound( a ) orelse i > ubound( a ) then return fallback
    return a( i )
end function

dim as int32 a( )
redim a( 1 to 3 )
a( 1 ) = 10 : a( 2 ) = 20 : a( 3 ) = 30

print "ValueAt( 2 )  ="; ValueAt( a( ), 2, -1 )
print "ValueAt( 9 )  ="; ValueAt( a( ), 9, -1 )
print "ValueAt( 0 )  ="; ValueAt( a( ), 0, -1 )

redim preserve a( 1 to 2 )
print "after shrinking, ValueAt( 3 ) ="; ValueAt( a( ), 3, -1 )

ValueAt( 2 )  = 20
ValueAt( 9 )  =-1
ValueAt( 0 )  =-1
after shrinking, ValueAt( 3 ) =-1

Comparing against lbound and ubound costs two comparisons and works on every array, of every kind, in every build. Where an index comes from outside the program — a file, an argument, a calculation you are not certain of — that guard is not optional.

Text stored in an array

'' textarray.tiko -- an array of text that grows as rows arrive.

dim as ustring names( )

'' Each append makes room for one more and writes into the new slot.
dim as ustring incoming( 1 to 3 ) = { "Ada", "Grace", "Barbara" }

for i as int32 = lbound( incoming ) to ubound( incoming )
    redim preserve names( 0 to ubound( names ) + 1 )
    names( ubound( names ) ) = incoming( i )
next

'' Take the count into a variable of its own before printing it.
dim as int32 count = ubound( names ) - lbound( names ) + 1
print "count ="; count

for i as int32 = lbound( names ) to ubound( names )
    print i; ": " & names( i )
next

count = 3
 0: Ada
 1: Grace
 2: Barbara

Growing an array one element at a time — redim preserve to ubound + 1, then write the new slot — is the pattern for building a list whose length is not known in advance. It is correct, and it is also the slow way to do it: each redim preserve may copy the whole array, so appending n items can cost n² work.

For a handful of rows that does not matter. For a file of them it does, and the answer is either to grow in chunks — double the size when it fills — or to use the container that already does exactly that, which is the next section.

The descriptor

'' descriptor.tiko -- what the compiler keeps alongside the elements.
#include once "fbc-int/array.bi"

dim as int32 a( 1 to 4 )
dim as int32 grid( 0 to 1, 0 to 2 )
dim as int32 empty_( )

print "a( 1 to 4 )       len ="; FB.ArrayLen( a( ) ); _
      " bytes ="; FB.ArraySize( a( ) )
print "grid( 0..1, 0..2 ) len ="; FB.ArrayLen( grid( ) ); _
      " bytes ="; FB.ArraySize( grid( ) )
print "an empty array     len ="; FB.ArrayLen( empty_( ) )

dim as FBC.FBARRAY ptr d = FBC.ArrayDescriptorPtr( a( ) )
print "descriptor: dimensions ="; d->dimensions; _
      " element_len ="; d->element_len
print "            dimTb( 0 ) ="; d->dimTb( 0 ).lbound; " to"; _
      d->dimTb( 0 ).ubound

a( 1 to 4 )       len =4 bytes =16
grid( 0..1, 0..2 ) len =6 bytes =24
an empty array     len =0
descriptor: dimensions =1 element_len =4
            dimTb( 0 ) = 1 to 4

An array variable is a descriptor: a pointer to the elements plus the bounds, the element size and the number of dimensions. That is what travels when an array is passed, and what lbound reads.

fbc-int/array.bi exposes it. FB.ArrayLen is the total element count across all dimensions and FB.ArraySize the total bytes; FBC.ArrayDescriptorPtr hands over the record itself. You will not often need these, and when you do — writing a procedure that takes an array of any type, checking what a redim actually allocated — nothing else will do.

When to reach for something else

An array is the right answer when the index is meaningful and the size is known, or grows rarely. It is the wrong answer when you find yourself writing the bookkeeping around it: a separate count of how many slots are really in use, a doubling strategy, a search to find where something is.

That bookkeeping is what containers.Array( of T ) already contains, along with Map, Set and the rest of the containers in Chapter 24. The rule is the ordinary one: use the raw array while it is simpler, and stop as soon as it is not.

What it costs

• Indexing is an address calculation, not a search. Cost does not depend on which element.
• redim preserve may copy everything. Growing one at a time is quadratic; grow in chunks.
• Row-major order is a performance fact, not trivia — put the last subscript in the inner loop.
• -exx costs a comparison per access. It is worth it while you are writing the program; measure before you decide it is not worth it afterwards.
• for each byref avoids copying each element, which is free to write and matters for large ones.
• Returning an array moves it. The cost is one descriptor, a fixed few hundred bytes, whatever the length of the array; the elements are never copied.

Summary

• An array carries bounds, not a length: lbound and ubound are the first and last index.
• a( 10 ) is eleven elements, indexed 0 to 10. Write both bounds.
• Elements are zero-initialised; an initialiser in braces fills them in order, one level of braces per dimension, and 1 to ... lets the list set the size.
• Braces initialize arrays and nothing else: not a single value (TKO0845), and not an assignment after the declaration (TKO0846).
• dim a( ) is empty, with ubound below lbound, so loops over it run zero times.
• redim discards the contents; redim preserve keeps them. erase frees a dynamic array and zeroes a fixed one.
• Dimensions are comma-separated, ubound( m, 2 ) names one, and the last subscript varies fastest.
• Arrays pass by reference as a( ), carrying their bounds; a rank-2 parameter is written m( , ).
• A function returns an array with as int32( ). return takes a brace list, a local array (moved, never copied) or another call, and the caller owns the result: var takes it, m = f( ) replaces what m held. A parameter or shared array cannot be returned (TKO0840).
• for each gives copies, for each byref gives the elements, and neither will walk a multi-dimensional array (TKO0608).
• -exx checks every subscript, fixed or dynamic, against the bounds the array has at that moment. Without it, compare against lbound and ubound yourself when the index is not obviously safe.
• An array is not assigned or swapped whole (TKO0930); copy it element by element.
• The array variable is a descriptor; fbc-int/array.bi exposes FB.ArrayLen, FB.ArraySize and the record itself.
• Stop using a raw array when you start writing bookkeeping around it — a live count, a doubling strategy, a search. containers.Array( of T ) and the containers in Chapter 24 already contain it, and growing one element at a time with redim preserve is quadratic.

Exercises

1. Write Reverse, which reverses an array in place and works whatever its bounds — test it on arrays indexed from 1, from 0 and from −2, and on an empty one. Use no literal 0 and no literal length. {solution="ex08-01"}
2. Transpose a two-dimensional array into a second array whose dimensions are the original's, swapped. Where do the new bounds come from? {solution="ex08-02"}
3. Predict how many elements dim as int32 a( 5 ) has before running anything, then print lbound, ubound and the count. Rewrite the declaration so the answer is obvious.
4. Take the redim line in dynamic.tiko, add preserve, and say in one sentence what each version is for.
5. Build a 200 × 200 array and sum it twice: once with the last subscript innermost, once with the loops swapped. Both give the same total. Which do you expect to be faster, and why?
6. Write Evens( byval n as int32 ), returning the first n even numbers as an array indexed from 1, and Doubled( a( ) as int32 ), returning a new array of the same bounds with every element doubled. Take var e = Evens( 4 ) and print Doubled( e( ) ) with for each. Why can Doubled not simply return a?

Strings and Text

Text is where a program meets people, and it is where most of the assumptions a programmer carries turn out to be wrong. A string is not a sequence of characters. A character is not a byte. The length of a word depends on what you are counting.

This chapter is long because that is the honest size of the subject, and because getting it wrong is silent: text that looks right on your machine arrives as nonsense on someone else's.

The short version, which the rest of the chapter earns: use ustring, and keep bytes and text apart in your head.

Five ways to hold text

'' types.tiko -- five ways to hold text, and what len counts in each.

dim as string       a = "hello"     '' bytes, length-counted, grows
dim as ustring      b = "hello"     '' UTF-16 code units, grows
dim as zstring * 16 c = "hello"     '' bytes, NUL-terminated, fixed room
dim as wstring * 16 d = "hello"     '' wide characters, fixed room
dim as string * 8   e = "hello"     '' bytes, fixed length, PADDED

print "string       len ="; len( a )
print "ustring      len ="; len( b )
print "zstring * 16 len ="; len( c )
print "wstring * 16 len ="; len( d )
print "string * 8   len ="; len( e ); "   <- not 5"
print "string * 8   = ["; e; "]"

'' A string variable is a descriptor, not the characters themselves.
print "sizeof( string )  ="; sizeof( string )
print "sizeof( ustring ) ="; sizeof( ustring )

string       len = 5
ustring      len = 5
zstring * 16 len = 5
wstring * 16 len = 5
string * 8   len = 8   <- not 5
string * 8   = [hello   ]
sizeof( string )  = 24
sizeof( ustring ) = 24

ustring is the one to reach for. It is the type this book uses everywhere, it holds text rather than bytes, and len on it is a single read rather than a walk.

string is the byte type. It is right for file contents, network payloads and anything you do not intend to interpret — and wrong for anything a person will read, unless you also know its encoding.

zstring and wstring exist for talking to C, which Chapter 29 covers. They are the shapes a C library expects: a pointer to bytes ending in a NUL, or the platform's wide character.

Text that spans lines

A line ends a statement in tiko, so an ordinary string literal ends where its line does. A query, a usage message or a block of JSON is several lines of text, and writing it as one literal per line, glued with & and chr( 10 ), buries the text under the punctuation. A block string holds it the way it reads:

dim as ustring query = """
    select id, name
      from users
     where name = "Ada"
    """
print query
print len( query )

dim as ustring poem = """
  roses
    are
  red

  """
print poem;
print "---"

dim as ustring shifted = """
    indented
"""
print "["; shifted; "]"

dim as ustring row = !"""
    name\tvalue
    one long line, \
    continued
    """
print row
print instr( row, chr( 9 ) )

select id, name
  from users
 where name = "Ada"
 48
roses
  are
red
---
[    indented]
name	value
one long line, continued
 5

""" at the end of a line opens one, and the first line that starts with """ closes it. Only spaces may follow the opening """. Anything after the closing one is ordinary code, so a block string can sit inside an expression or an argument list like any other literal.

The closing line's indentation is the margin. It is removed from the front of every line, so the text indents with the code around it, while indentation that belongs to the text — the two extra spaces before from — is kept. A blank line is always allowed. Any other line that starts left of the margin is an error:

'' A line that starts left of the closing line is an error.
dim as ustring note = """
    first
  second
    """
print note

blockmargin.tiko(4,1): error TKO0208: block string line does not start with the closing line's indentation
     |
   4 |   second
     | ^~~~~~~~~

The line breaks at the two ends are not part of the text. query is 48 characters and does not end in a line feed. To end with one, leave an empty line before the closing line, as poem does — which is why print poem; still put --- on a line of its own.

Inside the block, nothing needs escaping. A " is an ordinary character, so "Ada" is written as it reads. Every line break is a single line feed, chr( 10 ), whatever line endings the file was saved with, so the same source gives the same string on every machine.

!""" reads escapes, exactly as !"…" does: \t became the tab instr found at position 5. A backslash at the very end of a line joins it to the next with no line break, for a long line you want to wrap in the source.

The operations that come with the language

'' intrinsics.tiko -- the operations that come with the language.

dim as ustring s = "The quick brown fox"

print "len            ="; len( s )
print "left( s, 3 )   = ["; left( s, 3 ); "]"
print "right( s, 3 )  = ["; right( s, 3 ); "]"
print "mid( s, 5, 5 ) = ["; mid( s, 5, 5 ); "]"
print "mid( s, 11 )   = ["; mid( s, 11 ); "]"

print "instr( s, ""brown"" )  ="; instr( s, "brown" )
print "instr( 5, s, ""o"" )   ="; instr( 5, s, "o" )
print "instrrev( s, ""o"" )   ="; instrrev( s, "o" )
print "instr( s, ""zebra"" )  ="; instr( s, "zebra" ); "   <- not found"

print "ucase = ["; ucase( s ); "]"
print "lcase = ["; lcase( s ); "]"

dim as ustring padded = "   spaces   "
print "trim  = ["; trim( padded ); "]"
print "ltrim = ["; ltrim( padded ); "]"
print "rtrim = ["; rtrim( padded ); "]"
print "trim any = ["; trim( "xxcorexx", any "x" ); "]"

print "space( 3 )        = ["; space( 3 ); "]"
print "string( 5, ""-"" )   = ["; string( 5, "-" ); "]"
print "chr( 65 )         = ["; chr( 65 ); "]"
print "asc( ""A"" )        ="; asc( "A" )

len            = 19
left( s, 3 )   = [The]
right( s, 3 )  = [fox]
mid( s, 5, 5 ) = [quick]
mid( s, 11 )   = [brown fox]
instr( s, "brown" )  = 11
instr( 5, s, "o" )   = 13
instrrev( s, "o" )   = 18
instr( s, "zebra" )  = 0   <- not found
ucase = [THE QUICK BROWN FOX]
lcase = [the quick brown fox]
trim  = [spaces]
ltrim = [spaces   ]
rtrim = [   spaces]
trim any = [core]
space( 3 )        = [   ]
string( 5, "-" )   = [-----]
chr( 65 )         = [A]
asc( "A" )        =65

Two conventions to absorb, because everything else follows from them.

Positions start at 1. mid( s, 5, 5 ) means five characters starting at the fifth. This is not an accident to be worked around — it is consistent across every intrinsic here, and mixing it with zero-based thinking is the single commonest source of off-by-one bugs in string code.

Not found is 0. instr returns 0 when the needle is absent, which is never a valid position, so if instr( s, x ) > 0 is the test. There is no separate "not found" value to remember.

mid also works on the left of an assignment, where it overwrites in place:

'' midstatement.tiko -- mid on the left of an assignment overwrites
'' in place, without rebuilding the string.

dim as string s = "abcdef"
mid( s, 2, 3 ) = "XYZ"
print "mid( s, 2, 3 ) = ""XYZ"" -> ["; s; "]"

dim as string t = "abcdef"
mid( t, 2 ) = "XY"              '' no length: as much as will fit
print "mid( t, 2 )    = ""XY""  -> ["; t; "]"

'' It never changes the length. A replacement longer than the room
'' available is truncated rather than growing the string.
dim as string u = "abc"
mid( u, 2 ) = "ZZZZZZ"
print "overlong replacement   -> ["; u; "]  len ="; len( u )

mid( s, 2, 3 ) = "XYZ" -> [aXYZef]
mid( t, 2 )    = "XY"  -> [aXYdef]
overlong replacement   -> [aZZ]  len = 3

The mid statement never changes the length. A replacement longer than the room available is truncated, as the third case shows. That is what makes it cheap: nothing is reallocated. It works the same on a ustring, counting UTF-16 units.

Numbers and text

'' numbers.tiko -- text to numbers and back.

print "str( 42 )     = ["; str( 42 ); "]"
print "str( -7 )     = ["; str( -7 ); "]"
print "str( 3.5 )    = ["; str( 3.5 ); "]"
print "print puts a space where the sign would be:"; 42

print "val( ""42"" )       ="; val( "42" )
print "val( ""42abc"" )    ="; val( "42abc" ); "  <- stops at 'a'"
print "val( ""abc"" )      ="; val( "abc" ); "  <- and a word gives 0"
print "valint( ""42"" )    ="; valint( "42" )
print "vallng( ""4200000000"" ) ="; vallng( "4200000000" )
print "valuint( ""&hFF"" ) ="; valuint( "&hFF" )

print "hex( 255 )    = ["; hex( 255 ); "]"
print "hex( 255, 4 ) = ["; hex( 255, 4 ); "]"
print "oct( 8 )      = ["; oct( 8 ); "]"
print "bin( 5 )      = ["; bin( 5 ); "]"

print using "pi to two places: ###.##"; 3.14159
print using "a whole number:   ###"; 42

str( 42 )     = [42]
str( -7 )     = [-7]
str( 3.5 )    = [3.5]
print puts a space where the sign would be: 42
val( "42" )       = 42
val( "42abc" )    = 42  <- stops at 'a'
val( "abc" )      = 0  <- and a word gives 0
valint( "42" )    = 42
vallng( "4200000000" ) = 4200000000
valuint( "&hFF" ) =255
hex( 255 )    = [FF]
hex( 255, 4 ) = [00FF]
oct( 8 )      = [10]
bin( 5 )      = [101]
pi to two places:   3.14
a whole number:    42

str converts a number to text with no padding. print is different — it reserves a column for the sign, which is why printed positive numbers appear with a leading space throughout this book.

hex, oct and bin render a value in another base, and a second argument sets the width: hex( 255, 4 ) is 00FF.

print using lays out a line against a picture: ###.## is a number with two decimals, & is a string. It is the quickest way to get a column to line up. For a line that mixes widths, bases and decimals, strings.Format, later in this chapter, says it in one place.

What len is actually counting

Here is the idea the rest of the chapter depends on. A string has three different lengths, and which one you get depends on the type holding it.

'' codeunits.tiko -- what len actually counts, in each type.
''
'' The text is built from explicit bytes so that nothing here depends
'' on how this file was saved: these ARE the UTF-8 bytes for the two
'' strings, listed one by one.

'' "cafe" with an e-acute: 63 61 66 C3 A9
dim as string cafeBytes = chr( &h63, &h61, &h66, &hC3, &hA9 )

'' "a", U+1F600 GRINNING FACE, "b": 61 F0 9F 98 80 62
dim as string faceBytes = chr( &h61, &hF0, &h9F, &h98, &h80, &h62 )

dim as ustring cafe = cafeBytes
dim as ustring face = faceBytes

print "cafe: as string (UTF-8 bytes) len ="; len( cafeBytes )
print "      as ustring (code units) len ="; len( cafe )
print "      code units:";
for i as int32 = 1 to len( cafe ) : print asc( cafe, i ); : next
print

print "face: as string (UTF-8 bytes) len ="; len( faceBytes )
print "      as ustring (code units) len ="; len( face )
print "      code units:";
for i as int32 = 1 to len( face ) : print asc( face, i ); : next
print
print "      -- 55357 and 56832 are ONE character, stored as a pair"

'' Converting back is lossless.
dim as string roundTrip = face
print "round trip back to string: len ="; len( roundTrip ); _
      "  equal = "; cbool( roundTrip = faceBytes )

cafe: as string (UTF-8 bytes) len = 5
      as ustring (code units) len = 4
      code units: 99 97 102 233
face: as string (UTF-8 bytes) len = 6
      as ustring (code units) len = 4
      code units: 97 55357 56832 98
      -- 55357 and 56832 are ONE character, stored as a pair
round trip back to string: len = 6  equal = true

The listing builds its text from explicit byte values, so nothing depends on how the file was saved — those really are the UTF-8 bytes.

Read the two cases:

• café is 4 characters. As UTF-8 bytes it is 5 — é needs two. As a ustring it is 4 code units. Here characters and code units agree.
• a + U+1F600 + b is 3 characters. As UTF-8 bytes it is 6. As a ustring it is 4 code units — because U+1F600 is outside the Basic Multilingual Plane and UTF-16 stores it as a surrogate pair, two units that mean one character.

Conversion between string and ustring is UTF-8 in both directions, on every target, and it is lossless: the round trip in the listing compares equal.

That is the rule worth memorising. A ustring assigned to a string is encoded as UTF-8; a string assigned to a ustring is decoded as UTF-8. There is no locale involved and no target dependence.

Bytes that are not UTF-8

'' ansibytes.tiko -- a byte string is not automatically UTF-8.
''
'' Both of these hold "cafe" with an e-acute. One was saved by a
'' program that wrote Latin-1; the other is UTF-8. Converting each to
'' ustring shows what the difference costs.

dim as string latin1 = chr( &h63, &h61, &h66, &hE9 )
dim as string utf8   = chr( &h63, &h61, &h66, &hC3, &hA9 )

dim as ustring fromLatin1 = latin1
dim as ustring fromUtf8   = utf8

print "Latin-1 bytes:"; len( latin1 ); " -> ustring units:"; _
      len( fromLatin1 )
print "  code units:";
for i as int32 = 1 to len( fromLatin1 )
    print asc( fromLatin1, i );
next
print

print "UTF-8 bytes:  "; len( utf8 ); " -> ustring units:"; _
      len( fromUtf8 )
print "  code units:";
for i as int32 = 1 to len( fromUtf8 ) : print asc( fromUtf8, i ); : next
print
print
print "65533 is U+FFFD, the replacement character: the byte was not"
print "valid UTF-8, so the text it stood for is gone."

Latin-1 bytes: 4 -> ustring units: 4
  code units: 99 97 102 65533
UTF-8 bytes:   5 -> ustring units: 4
  code units: 99 97 102 233

65533 is U+FFFD, the replacement character: the byte was not
valid UTF-8, so the text it stood for is gone.

Both of those strings hold the word café. One was written by something that used Latin-1, where é is the single byte E9. The other is UTF-8, where it is C3 A9.

Converting each to ustring decodes as UTF-8 — because that is the only rule — so the Latin-1 byte is not valid input and becomes 65533, U+FFFD, the replacement character. The text it stood for is gone, and nothing was reported.

The standard library

'' stdstrings.tiko -- what the standard library adds.
#import "std/strings"

dim as ustring s = "alpha,beta,gamma"

'' Split hands back an Array( of ustring ) -- the container from
'' Chapter 24. Here it is only asked for its count and its elements.
var parts = strings.Split( s, "," )
print "Split gave"; parts.Count( ); " parts:";
for i as int32 = 0 to parts.Count( ) - 1
    print " ["; parts[ i ]; "]";
next
print

print "Join       = ["; strings.Join( parts, " | " ); "]"
print "Replace    = ["; strings.Replace( s, "beta", "BETA" ); "]"
print "Between    = ["; strings.Between( "name(value)", "(", ")" ); "]"
print "StartsWith = "; strings.StartsWith( s, "alpha" )
print "EndsWith   = "; strings.EndsWith( s, "gamma" )
print "Contains   = "; strings.Contains( s, "bet" )
print "IsNumeric  = "; strings.IsNumeric( "42" ); _
      strings.IsNumeric( "4a" )
print "PadLeft    = ["; strings.PadLeft( "7", 4, "0" ); "]"
print "PadRight   = ["; strings.PadRight( "ab", 5, "." ); "]"
print "PadCenter  = ["; strings.PadCenter( "ab", 6, "." ); "]"
print "Repeat     = ["; strings.Repeat( 3, "ab" ); "]"
print "Reverse    = ["; strings.Reverse( "abc" ); "]"
print "Wrap       = ["; strings.Wrap( "hi", "<", ">" ); "]"
print "Escape     = ["; strings.Escape( !"a\tb" ); "]"

Split gave 3 parts: [alpha] [beta] [gamma]
Join       = [alpha | beta | gamma]
Replace    = [alpha,BETA,gamma]
Between    = [value]
StartsWith = true
EndsWith   = true
Contains   = true
IsNumeric  = truefalse
PadLeft    = [0007]
PadRight   = [ab...]
PadCenter  = [..ab..]
Repeat     = [ababab]
Reverse    = [cba]
Wrap       = [<hi>]
Escape     = [a\tb]

#import "std/strings" brings in around forty-five operations that the language itself does not have. The ones above are the everyday set; the module also carries Extract, Remain, Tally, InsertAt, DeleteAt, Shrink, VerifySet, the *Chars family that works on sets of characters rather than substrings, and Unescape/Unwrap to undo their opposites.

Split hands back an Array( of ustring ) — the container Chapter 24 covers. Here it is only asked for .Count( ) and [ i ], which is all you need to walk it.

Join( Split( s, d ), d ) gives back s, for every s and every non-empty d. That round-trip property is worth knowing, because it tells you what Split does with the awkward cases: "a,,c" is three fields with an empty one in the middle, and "a," is two.

Part of a string, without a copy

mid, left, right and Split all hand back new strings. Each one allocates and copies, and a program that takes a line apart word by word pays that once per word. Most of those words are looked at once and thrown away. A slice names part of a string where it already is:

'' slices.tiko -- naming part of a string without copying it.

dim as ustring s = "hello, world"

dim as uslice word = s[7 to 11]    '' units 7 to 11, both included
print "word           = ["; word; "]"
print "len( word )    = "; str( len( word ) )
print "word[0]        = "; str( word[0] )   '' a code unit
print "word[1 to 3]   = ["; word[1 to 3]; "]"
print "s[0 to 4]      = ["; s[0 to 4]; "]"
print "mid( s, 1, 5 ) = ["; mid( s, 1, 5 ); "]"

'' A slice compares, joins and prints like the text it names.
print "word = ""world"" is "; cbool( word = "world" )
print "<" & word & ">"

'' Assigning it to a ustring makes an owned copy.
dim as ustring copy = word
copy &= "!"
print "copy = ["; copy; "]  s = ["; s; "]"

'' A slice of a mutable string writes straight into it.
word[0] = asc( "W" )
print "s is now ["; s; "]"

'' An empty slice: hi is one less than lo.
print "s[3 to 2] has "; str( len( s[3 to 2] ) ); " units"

word           = [world]
len( word )    = 5
word[0]        = 119
word[1 to 3]   = [orl]
s[0 to 4]      = [hello]
mid( s, 1, 5 ) = [hello]
word = "world" is true
<world>
copy = [world!]  s = [hello, world]
s is now [hello, World]
s[3 to 2] has 0 units

s[7 to 11] is the units from 7 to 11, both ends included, counting from 0 — the same counting as s[i], which reads one unit. That is the one place in this chapter where positions start at 0, and it is why the listing prints mid( s, 1, 5 ) beside s[0 to 4]: the same five characters, one counted from 1 and copied, the other counted from 0 and not.

There are three slice types, one for each width of text:

A slice is small — a pointer, a length and one more word the runtime checks use — and it is never a copy. It prints, compares and joins with & as the text it names, and len on it is a single read. Indexing it gives a code unit, a number, as indexing a ustring does. Slicing a slice gives a smaller view of the same text.

A slice is as writable as the string it came from. word[0] = asc( "W" ) changed s, because word is part of s. A slice of a const string, a literal or a temporary is a const uslice, and writing through it is refused. Going the other way — from const back to writable — is error TKO1052.

To keep the text, assign the slice to a ustring. That makes the copy, once, at the point where you decide the text is worth keeping.

s[3 to 2] is empty: a slice whose end is one before its start has no units. So s[0 to len( s ) - 1] is the whole of s even when s is empty. Anything else outside the string is runtime error 6 — checked whenever a slice is made, in every build — and a range the compiler can see is wrong, such as 5 to 2 or past the end of a string * 4, is TKO1051 before the program runs.

Slices in the standard library

std/strings has a set of functions that take and return slices, so a whole parse can run without a single copy:

'' slicestd.tiko -- reading settings with no copies at all.
#import "std/strings"

dim as ustring config = !"  name = tiko \n\n version=0.1 \n  bad line"

for each line_ as const uslice in _
        strings.SplitSlices( config[0 to len( config ) - 1], !"\n" )
    dim as const uslice t = strings.TrimSpace( line_ )
    if len( t ) = 0 then continue for

    dim as int32 eq = strings.Find( t, "=" )
    if eq < 0 then
        print "skipped: ["; t; "]"
        continue for
    end if

    dim as const uslice key = strings.TrimSpace( t[0 to eq - 1] )
    dim as const uslice value = _
        strings.TrimSpace( t[eq + 1 to len( t ) - 1] )
    print "["; key; "] -> ["; value; "]"
next

[name] -> [tiko]
[version] -> [0.1]
skipped: [bad line]

TrimSpace, TrimStart and TrimEnd return a slice of their argument with the spaces, tabs, carriage returns and line feeds removed from the ends. Find returns the position of the first match, counting from 0, or -1 when there is none — slice counting, not instr's. Equals, StartsWith and EndsWith compare, with an optional true to ignore the case of A to Z.

SplitSlices is Split without the array. It hands the parts to a for each one at a time, each a slice of the original. An empty part is still a part, so the blank line in config arrives as an empty slice — that is what the len( t ) = 0 test is for.

Every one of these takes a const slice, and any string can be passed for one. A string of the same width is viewed where it is; any other is converted once, for the call.

A slice cannot outlive its text

A slice points into memory it does not own. If that memory is freed or moved while the slice is still in use, the slice reads whatever is there now. The compiler keeps track of where every slice came from and refuses each way that could happen:

'' sliceescape.tiko -- a slice of a local cannot leave its function.

function FirstWord( byval s as ustring ) as const uslice
    return s[0 to 2]      '' error: s is this call's own copy
end function

print FirstWord( "the end" )

sliceescape.tiko(4,13): error TKO1059: cannot return a slice of local 's'
     |
   4 |     return s[0 to 2]      '' error: s is this call's own copy
     |             ^

s is passed byval, so it belongs to the call and is gone when the call returns. Make it byref s as const ustring and the function compiles: the text then belongs to the caller, which still has it after the call.

The other rules follow from the same idea. A slice may be a local variable, a parameter or a result. It may not be shared or static, an array element, an element of a container, or allocated with new (TKO1054), because any of those can outlive the text with nothing to say when. A slice of a string declared in an inner block cannot be kept in a variable declared outside it (TKO1057). A slice of a temporary — (a & b)[0 to 3] — can be printed or passed in the same statement, and cannot be kept at all (TKO1058).

The last rule is about the text changing underneath the slice:

'' slicelock.tiko -- a string cannot grow while a slice reads it.

dim as ustring s = "hello"
dim as uslice first3 = s[0 to 2]

s &= ", world"        '' error: may move the text first3 points into
print first3

slicelock.tiko(6,1): error TKO1062: 's' cannot be changed while slice 'first3' borrows from it (last used at line 7)
     |
   6 | s &= ", world"        '' error: may move the text first3 points into
     | ^

Appending to a ustring may move its text to a bigger block of memory, which would leave first3 pointing at the old one. So from the moment a slice is made until the last line that reads it, the string it came from cannot be assigned, appended to, swapped, read into, or passed byref to a procedure that might do any of those. Move the print above the &= and the program compiles: the slice is finished with before the string grows. Writing single units in place — s[i] = x, the mid statement — never moves the text, and stays allowed.

Types that hold a slice

A field of an ordinary type can outlive anything, so a slice field is refused there (TKO1055). A type declared as view may hold one, and is then held to the same rules as a slice:

'' lexer.tiko -- a type that holds a slice is a view.

type Lexer as view
    src as const uslice      '' the text being read, not a copy
    pos as int32
end type

'' The next run of non-blank units, or an empty slice at the end.
function NextWord( byref lx as Lexer ) as const uslice
    while lx.pos < len( lx.src ) andalso lx.src[lx.pos] = 32
        lx.pos += 1
    wend
    dim as int32 start = lx.pos
    while lx.pos < len( lx.src ) andalso lx.src[lx.pos] <> 32
        lx.pos += 1
    wend
    return lx.src[start to lx.pos - 1]
end function

dim as ustring text = "  let total = price * 3  "
dim as Lexer lx
lx.src = text[0 to len( text ) - 1]

do
    dim as const uslice w = NextWord( lx )
    if len( w ) = 0 then exit do
    print "<"; w; ">";
loop
print

<let><total><=><price><*><3>

Lexer walks the text without copying any of it, and NextWord returns a slice of the text the lexer holds — which is the caller's, so it may. A Lexer cannot be shared, put in an array or stored in an ordinary type, exactly as a slice cannot, and a type that holds a Lexer must say as view too.

What the compiler cannot see

The compiler follows a string through one procedure. It does not follow it into another, so it cannot know that Note appends to the same logText that head points into:

'' slicecheck.tiko -- what -exx catches that the compiler cannot.
#cmdline "-exx"

dim shared as ustring logText

sub Note( byref msg as const ustring )
    logText &= msg & " -- long enough to move the text elsewhere"
end sub

sub Report( )
    logText = "start"
    dim as const uslice head = logText[0 to 4]
    Note( "grown" )      '' grows logText behind head's back
    print head           '' head now points at freed memory
end sub

Report( )

Aborting due to runtime error 6 (out of bounds array access) at line 14 of slicecheck.tiko::REPORT()

Every slice of a string or ustring remembers which string it came from. Under -exx, each read of the slice first checks that its text is still inside that string, and stops the program with runtime error 6 when it is not. A v[i] past the end of a slice is caught the same way.

Laying out a line

str, hex and print using each do one job. A line that wants a name padded to a width, a number in hexadecimal and a price to two decimals becomes three calls glued with &, and the shape of the line disappears into them. strings.Format takes the shape as one string — a format string — and the values to put in it:

'' format.tiko -- text laid out by a format string.
#import "std/strings"

dim as ustring who = "Quinn"
dim as int32 n = 42
dim as float64 x = 3.14159265358979

'' {} takes the next value; {n} takes value n, counting from 0.
print strings.Format( "Hello, {}!", who )
print strings.Format( "{1} before {0}", "b", "a" )

'' Fill, alignment and width, after a ':'.
print strings.Format( "[{:>6}] [{:<6}] [{:^6}]", n, n, n )
print strings.Format( "[{:*^9}] [{:.3}]", "hi", who )

'' Integers in other bases, a sign, zero padding.
print strings.Format( "{:x} {:#x} {:08b} {:+}", 255, 255, 5, n )
print strings.Format( "[{:+06}] [{:c}]", n, 65 )

'' Floats: fixed, exponent, and the default -- the shortest text
'' that reads back as the same value.
print strings.Format( "{:.2f} {:e} {}", x, x, 0.1 + 0.2 )

print strings.Format( "{} {:d}", true, true )
print strings.Format( "{{}} is a literal pair of braces" )

Hello, Quinn!
a before b
[    42] [42    ] [  42  ]
[***hi****] [Qui]
ff 0xff 00000101 +42
[+00042] [A]
3.14 3.141593e+00 0.30000000000000004
true 1
{} is a literal pair of braces

Each {} is replaced by the next value. {0}, {1} and so on name a value by position, counting from 0, so the same value can appear twice or the order can change; a format string uses one style or the other, never both. {{ and }} are literal braces. Values left over are not an error.

After a : comes the spec, a run of optional fields in a fixed order:

Numbers align right and everything else left unless the spec says otherwise. A boolean is the word true or false, or 1 and 0 under d.

The spec language is C++'s std::format, so a format string written for one reads the same in the other.

A float with no type is printed as the shortest text that reads back as the same value. That is why 0.1 + 0.2 appears with its last digit, 0.30000000000000004: the sum is not 0.3, and Format does not hide that. Give it a precision — {:.2f} — when a reader wants two decimals.

A width can come from a value as well, with {:{}}. That is what makes a table line up when the widest entry is known only at run time:

'' formattable.tiko -- columns that line up, whatever the values.
#import "std/strings"

dim as ustring item( 0 to 2 ) = { "tea", "biscuits", "jam" }
dim as int32 qty( 0 to 2 ) = { 2, 12, 1 }
dim as float64 price( 0 to 2 ) = { 3.5, 0.25, 4.75 }

'' The width can come from a value: {:{}} takes it from the next one.
dim as int32 w = 10
print strings.Format( "{:<{}}{:>5}{:>9}", "item", w, "qty", "each" )
for i as int32 = 0 to 2
    print strings.Format( "{:<{}}{:>5}{:>9.2f}", _
                          item( i ), w, qty( i ), price( i ) )
next

item        qty     each
tea           2     3.50
biscuits     12     0.25
jam           1     4.75

Checked when it compiles

A format string written as a literal is read by the compiler, and checked against the values that follow it:

'' formatcheck.tiko -- a literal format string is checked when the
'' program is compiled.
#import "std/strings"

dim as int32 done = 3
print strings.Format( "{} of {} done", done )   '' error: one value

formatcheck.tiko(6,30): error TKO0851: format string: no argument 1 (1 given)
     |
   6 | print strings.Format( "{} of {} done", done )   '' error: one value
     |                              ^

The caret is on the field that has no value. Every way a format string can be wrong has its own id, TKO0847 to TKO0858 in Appendix B: a { with no }, a spec that does not parse, a type the value cannot take — {:x} on text, {:f} on an integer — and a precision on an integer, among others.

Format strings that are data

A format string read from a translation file or a user's settings is data, and a bad one is not a bug in the program. strings.TryFormat returns the text or the reason, and never stops anything:

'' tryformat.tiko -- a format string that is data, not code.
#import "std/strings"
#import "std/outcome"

sub Render( byref template as const ustring )
    var r = strings.TryFormat( template, "Quinn", 3 )
    if r.IsOk( ) then
        print "ok:      "; r.Value( )
    else
        print "refused: "; r.Failure( ).message; _
              " (offset"; r.Failure( ).offset; ")"
    end if
end sub

'' Templates as they might arrive from a translation file.
Render( "{} has {} messages" )
Render( "{1} messages for {0}" )
Render( "{} has {} messages, {}" )
Render( "{:d} has {} messages" )

ok:      Quinn has 3 messages
ok:      3 messages for Quinn
refused: no argument 2 (2 given) (offset 20)
refused: 'd' is no presentation type for text (offset 2)

The answer is a Result — Chapter 18's type for a value or the reason there isn't one. IsOk( ) says which, Value( ) is the text, and Failure( ) carries a message, an offset into the format string counted from 0, and a code to test against. TryFormat takes a plain ustring, so even a literal passed to it is left for run time.

Reading a table

'' csvreport.tiko -- reading a table and printing a report.
#import "std/strings"

dim as ustring text = _
    "name,city,year" & chr( 10 ) & _
    "Ada,""London, UK"",1815" & chr( 10 ) & _
    "Grace,New York,1906" & chr( 10 ) & _
    "Barbara,""Wilmington, DE"",1925"

var rows = strings.SplitCsv( text )
print "rows read:"; rows.Count( ); " (including the header)"
print

'' Row 0 is the header. Its field names become the column labels.
var header = rows[ 0 ].fields

for r as int32 = 1 to rows.Count( ) - 1
    var f = rows[ r ].fields
    for c as int32 = 0 to f.Count( ) - 1
        print strings.PadRight( header[ c ], 6, " " ); ": "; f[ c ]
    next
    print
next

'' A quoted field keeps its comma. Writing one back out re-quotes it.
print "field 1 of row 1 = ["; rows[ 1 ].fields[ 1 ]; "]"

rows read: 4 (including the header)

name  : Ada
city  : London, UK
year  : 1815

name  : Grace
city  : New York
year  : 1906

name  : Barbara
city  : Wilmington, DE
year  : 1925

field 1 of row 1 = [London, UK]

CSV looks like a job for Split and is not. A field may be quoted, and a quoted field may contain the delimiter — "London, UK" is one field, not two. Anything that splits on commas gets that wrong, and gets it wrong on exactly the rows a test file does not contain.

strings.SplitCsv( text ) parses a whole table and returns rows, each with a .fields array. strings.SplitCsvRow( line ) parses a single record and returns the fields directly. Both handle quoting, embedded delimiters and embedded newlines. Going the other way, strings.JoinCsvRow quotes each field that needs it, and strings.CsvWriter writes rows to a file (Chapter 25).

A string as bytes

'' binary.tiko -- a string as a container of bytes.

'' mk* packs a number into its raw bytes; cv* reads it back.
dim as string packed = mkl( 70000 ) & mkd( 3.5 ) & mkshort( 300 )
print "packed bytes ="; len( packed )

print "cvl   ="; cvl( mid( packed, 1, 4 ) )
print "cvd   ="; cvd( mid( packed, 5, 8 ) )
print "cvshort ="; cvshort( mid( packed, 13, 2 ) )

'' Fixed-length strings can be aligned within their room.
dim as string * 10 slot
lset slot, "left"
print "lset  = ["; slot; "]"
rset slot, "right"
print "rset  = ["; slot; "]"

'' strptr and sadd reach the bytes themselves.
dim as string s = "hello"
print "first byte via sadd ="; peek( ubyte, sadd( s ) )
print "asc of it           ="; asc( s, 1 )

packed bytes = 14
cvl   = 70000
cvd   = 3.5
cvshort = 300
lset  = [left      ]
rset  = [     right]
first byte via sadd =104
asc of it           =104

The mk* family packs a number into its raw bytes and cv* reads it back: mkl/cvl for 32-bit, mkshort/cvshort for 16, mkd/cvd and mks/cvs for floating point, mki/cvi for the pointer-sized integer. This is how a record gets written to a binary file, which Chapter 25 comes back to.

lset and rset align text within a fixed-length string. strptr and sadd give the address of the characters themselves, which is what a C function needs — and what makes it your responsibility to keep the string alive while the pointer is in use.

Bytes as text: base64, hex and URLs

Some places take only text: a JSON field, an e-mail body, a URL, a configuration file. Bytes that are not text — a hash, a key, an image — have to be spelled in characters that survive the trip. std/codec has the three common spellings:

#import "std/codec"
#import "std/containers"

'' Four bytes that no text format can carry as they are.
dim raw as containers.Array( of ubyte )
dim b( 0 to 3 ) as uint8 = { &h00, &hFB, &hFF, &h7F }
for i as int32 = 0 to 3
    raw.Push( b( i ) )
next

print "hex:     "; codec.HexEncode( raw )
print "base64:  "; codec.Base64Encode( raw )
print "url-safe "; codec.Base64Encode( raw, codec.B64_URL, false )

'' Text is encoded as its UTF-8 bytes, and decodes back the same.
dim word as ustring = "na" & wchr( &hEF ) & "ve"
dim enc as ustring = codec.Base64Encode( word )
print "naive:   "; enc
dim back as ustring = codec.Base64DecodeText( enc ).Value( )
print "same:    "; cbool( back = word )

hex:     00fbff7f
base64:  APv/fw==
url-safe APv_fw
naive:   bmHDr3Zl
same:    true

Hex writes each byte as two digits: twice the size, and readable. Base64 packs three bytes into four characters, a third bigger than the bytes; its URL-safe alphabet swaps + and / for - and _, and usually drops the = padding. Base32 (Base32Encode) uses only capital letters and digits, for codes that people type. Each has an …Encode, a …Decode that fills a containers.Array( of ubyte ), and a …DecodeText that returns a ustring — text is encoded as its UTF-8 bytes, so …DecodeText gives back exactly the text that went in.

A decoder can be lenient or exact. These are exact:

#import "std/codec"

'' A decoder that guesses hides corrupt data. This one refuses.
dim inputs( 0 to 3 ) as ustring = _
    { "aGVsbG8=", "aGVsbG8", "aGVs bG8=", "aGVsbG9=" }

for each t in inputs
    var r = codec.Base64DecodeText( t )
    if r.IsOk( ) then
        print t; " -> "; r.Value( )
    else
        print t; " -> "; r.Failure( ).Message( )
    end if
next

aGVsbG8= -> hello
aGVsbG8 -> Base64DecodeText: invalid padding
aGVs bG8= -> Base64DecodeText: invalid character at 5
aGVsbG9= -> Base64DecodeText: non-zero trailing bits at 7

The second input lost its padding, the third gained a space, and the fourth has a final character whose unused low bits are not zero — three different damages, each refused with the position of the first bad character. A lenient decoder would have returned something for all three, and the fourth would have decoded to hello, so a corrupted value would have compared equal to the original. When the input is known to be loose, say so: DECODE_IGNORE_WHITESPACE and DECODE_ALLOW_MISSING_PAD, combined with or, relax exactly those two rules.

Text inside a URL needs a different treatment. Percent-encoding keeps letters, digits and - . _ ~, and writes every other byte of the UTF-8 as % and two hex digits:

#import "std/codec"

dim dish as ustring = "fish & chips"
print "component: "; codec.PercentEncode( dish )
print "form:      "; codec.PercentEncode( dish, codec.PCT_QUERY )
print "path:      "; _
    codec.PercentEncode( "menu/" & dish, codec.PCT_PATH )

var r = codec.PercentDecode( "fish+%26+chips", codec.PCT_QUERY )
print "decoded:   "; r.Value( )

component: fish%20%26%20chips
form:      fish+%26+chips
path:      menu/fish%20&%20chips
decoded:   fish & chips

The mode decides what else survives. PCT_COMPONENT, the default, is for one value placed inside a URL, and escapes / and & because they would end it. PCT_PATH keeps the path's own /. PCT_QUERY is the encoding an HTML form submits, with a space as +. Decode with the mode you encoded with: + means a space only in PCT_QUERY.

Working with letters

'' caesar.tiko -- shifting letters, which is arithmetic on asc codes.

function Shift( byref text as ustring, byval by as int32 ) as ustring
    dim as ustring built = ""

    for i as int32 = 1 to len( text )
        dim as int32 c = asc( text, i )

        if c >= asc( "a" ) andalso c <= asc( "z" ) then
            c = asc( "a" ) + ( ( c - asc( "a" ) + by + 26 ) mod 26 )
        elseif c >= asc( "A" ) andalso c <= asc( "Z" ) then
            c = asc( "A" ) + ( ( c - asc( "A" ) + by + 26 ) mod 26 )
        end if

        built &= chr( c )
    next

    return built
end function

dim as ustring plain = "Attack at dawn!"
dim as ustring coded = Shift( plain, 3 )

print "plain   : "; plain
print "shifted : "; coded
print "back    : "; Shift( coded, -3 )
print "round trip holds: "; cbool( Shift( coded, -3 ) = plain )

plain   : Attack at dawn!
shifted : Dwwdfn dw gdzq!
back    : Attack at dawn!
round trip holds: true

asc and chr move between a character and its number, which turns text into arithmetic. The + 26 before the mod is there because Chapter 5's rule still applies: mod takes the sign of its left operand, so shifting backwards past a would otherwise give a negative answer.

This works for ASCII letters and stops there. It is a cipher exercise, not a text-processing technique — ucase on a ustring already knows about every alphabet, and hand-rolled arithmetic does not.

A value with no printable form

print "X["; Box( 7 ); "]Y"      '' refused

A type that has no conversion to text has nothing for print to show, whether it arrives as a variable or as a temporary like Box( 7 ): both are refused with TKO0871: a 'box' has no printable form. Give the type an operator cast( ) as ustring — Chapter 13 — and it prints.

What it costs

• len on a ustring is O(1). So is len on a string. Neither walks the text.
• ustring uses 2 bytes per code unit, so ASCII text costs twice what it would as a string. That is the price of not having to think about encoding, and it is almost always worth paying.
• & builds a new string. Appending in a loop copies what is there each time; for a large result, build into one string and overwrite with the mid statement, or let the library do it.
• A slice costs nothing to make — no allocation, no copy — and len on it is O(1). Keeping its text, by assigning it to a ustring, is the copy you skipped; under -exx each read of a slice is also checked against its source.
• Converting between string and ustring is a re-encode, not a cast. Do it at the boundary, once, not inside a loop.
• instr is a scan. Finding n things in a long string costs n scans unless you carry the position forward with instr( start, s, x ).
• Format reads its format string on every call, and copies each text value it is given. In a loop over millions of lines that is measurable; for a report it is not.

Summary

• Five types: ustring for text, string for bytes, zstring/wstring for C, string * n fixed and space-padded.
• wstring is 2 bytes per character on Windows and 4 on Linux; ustring is UTF-16 everywhere.
• A block string opens with """ at the end of a line and closes at the first line starting with """; that closing line's indentation is the margin removed from every line. !""" reads escapes, and every line break inside is chr( 10 ).
• Positions start at 1, and "not found" is 0.
• The mid statement overwrites in place and never changes the length, on string and ustring alike.
• str has no padding; print reserves a column for the sign. hex/oct/bin take a width.
• val( ) cannot fail — it returns 0 for text that is not a number. Use IsNumeric, or Chapter 18's Outcome, when it matters.
• len on a ustring counts code units. Outside the BMP one character is two units, so arithmetic positions can cut a character in half.
• Converting between string and ustring is UTF-8 in both directions, on every target, and lossless.
• Bytes that are not UTF-8 become U+FFFD, silently. Convert at the boundary, knowing what you have.
• #import "std/strings" adds ~45 operations including Split/Join, Replace, the Pad* family, Between and the CSV parsers. Join( Split( s, d ), d ) is s.
• strings.Format lays out a line from a format string: {} or {n} for each value, then : and a spec — fill, alignment, sign, width, precision, type. A float with no type is the shortest text that reads back exactly.
• A literal format string is checked when the program compiles (TKO0847 to TKO0858). One held in a variable is checked when it runs, and a bad one stops the program; for format strings that are data, TryFormat returns a Result instead.
• s[lo to hi] is a slice: part of s, counted from 0, both ends included, never copied. uslice, slice and wslice match ustring, string and wstring. A slice is as writable as its source; assigning it to a ustring keeps a copy.
• strings.TrimSpace, Find (0-based, -1 for none), Equals and SplitSlices work on slices without copying.
• A slice cannot outlive its text. It cannot be returned from the function that owns the text (TKO1059), stored where it could outlive it (TKO1054, TKO1057, TKO1058), or kept while its string grows (TKO1062). A type that holds one is declared as view.
• -exx catches what crosses a procedure: a slice read after its string moved stops with runtime error 6. Without it, the slice reads freed memory.
• CSV is not a Split problem — use SplitCsv or SplitCsvRow, which understand quoting.
• #import "std/codec": HexEncode, Base64Encode (standard or URL-safe) and Base32Encode spell bytes as text; …Decode and …DecodeText are strict and report the position of the first bad character. PercentEncode makes text safe in a URL — component, path or form query.
• mk*/cv* pack numbers into bytes; lset/rset align in a fixed-length string; strptr/sadd reach the characters for C.
• A value with no printable form is refused by print (TKO0871), variable or temporary.

Exercises

1. Write TitleCase, which uppercases the first letter of each word and lowercases the rest, then check it against strings.MCase on a sentence, a shouted one, one with double spaces and an empty string. {solution="ex09-01"}
2. Parse a block of key = value lines into pairs, skipping blanks and lines with no =. A value may itself contain an =; make sure yours survives that. {solution="ex09-02"}
3. Take the café bytes from codeunits.tiko and print len of the string and of the ustring. Then do the same for a string of pure ASCII. When do the two numbers agree, and why?
4. mid( face, 2, 1 ) on the emoji text returns half a character. Write a loop that walks a ustring and prints each character, treating a surrogate pair as one. What does it need to check?
5. Read a CSV line with SplitCsvRow, then split the same line on commas with Split, and print both field counts. Construct a line where they differ.
6. Print a multiplication table for 1 to 12 with strings.Format, every column right-aligned. Then make the column width depend on the largest product, so the table still lines up for 1 to 99.
7. Rewrite lexer.tiko's NextWord so that it also returns =, * and the other punctuation as words of their own, still without copying. Then move print "<"; w; ">"; into a procedure that takes byval w as const uslice, and call it with a ustring and with a literal as well as with a slice.
8. Encode the bytes 0 to 255 in base64, then decode the result with one character changed in each position in turn. Does every change get refused? Which ones decode, and to what?

User-Defined Types and Layout

Part I worked with values one at a time: a number, a string, an array of numbers. Real data comes in groups — a point has an x and a y, a reading has a station and a temperature — and a program that keeps the parts in separate variables keeps them in step by hand, which means sometimes out of step.

A type puts the parts under one name. This chapter covers what that gives you, and then what it costs in memory, because a type is also a decision about where bytes go. That second half is what lets tiko talk to file formats, network protocols and C libraries without translation.

Chapter 12 turns these types into objects with methods. Everything here still applies then; it is the foundation, not a beginner's version.

Grouping values

'' pointrect.tiko -- a type groups values that belong together.

type Point
    x as int32
    y as int32
end type

type Rect
    topLeft as Point                  '' a type inside a type
    size    as Point = ( 10, 20 )     '' fields can start with a value
    title   as ustring = "untitled"
    visible as boolean = true
end type

dim as Rect r
print "a fresh Rect: size"; r.size.x; r.size.y; _
      ", title "; r.title; ", visible "; r.visible

r.topLeft.x = 5                       '' reach in with dots
r.title = "window"
print "after edits:  topLeft"; r.topLeft.x; r.topLeft.y; _
      ", title "; r.title

'' A literal in parentheses fills the fields in order.
dim as Point origin = ( 0, 0 )
dim as Point corner = ( 640, 480 )
print "corner:"; corner.x; corner.y; "  origin:"; origin.x; origin.y

a fresh Rect: size 10 20, title untitled, visible true
after edits:  topLeft 5 0, title window
corner: 640 480  origin: 0 0

A type lists its fields, each with a name and a type. Fields are reached with a dot, and a type can contain another type, so r.topLeft.x reads the way the data is shaped.

A field may carry an initialiser — size as Point = ( 10, 20 ) — and every variable of the type starts with it. Fields without one start at zero, or empty for a string, exactly as variables do. A value in parentheses, ( 640, 480 ), fills a type's fields in declaration order.

Parentheses, not braces. Braces are how an array is filled (Chapter 8), and a type is not an array:

'' bracesudt.tiko -- a type is filled with parentheses. Braces are for
'' arrays, and a Point is not one.

type Point
    as int32 x, y
end type

dim as Point good = ( 3, 4 )
dim as Point bad = { 3, 4 }         '' refused
print good.x, bad.x

bracesudt.tiko(9,20): error TKO0845: 'bad' is not an array, so it cannot take a '{ ... }' initializer
     |
   9 | dim as Point bad = { 3, 4 }         '' refused
     |                    ^

fbc refuses it in the same place, with its error 63. Braces are for arrays, parentheses for types.

A type is a value

'' copying.tiko -- assignment copies the whole value, text included.

type Account
    owner   as ustring
    balance as int64
end type

dim as Account a
a.owner = "Ada"
a.balance = 100

dim as Account b = a            '' a complete, independent copy
b.owner = "Grace"
b.balance = 250

print "a: "; a.owner; a.balance
print "b: "; b.owner; b.balance

a: Ada 100
b: Grace 250

Assigning one variable of a type to another copies every field, including the text: changing b.owner afterwards leaves a.owner alone. The two are independent values, the way two int32s are, and nothing is shared.

Comparison is a different matter:

'' noequal.tiko -- a type can be copied, but not compared.
type Point
    x as int32
    y as int32
end type

dim as Point a = ( 1, 2 ), b = ( 1, 2 )
if a = b then print "equal"     '' refused: what would equal mean?

noequal.tiko(8,6): error TKO0720: no operator = compares this type
     |
   8 | if a = b then print "equal"     '' refused: what would equal mean?
     |      ^

The compiler will not guess what equality means for your type — whether two accounts with the same owner are equal, or only ones with the same balance too. Chapter 13 shows how to say.

Passing and returning

Chapter 7's table said that a type you define is passed by reference when no keyword is given. Here is the measurement:

'' passing.tiko -- a type you define is passed by reference unless
'' you say otherwise. Chapter 7 said so; this is the proof.

type Point
    x as int32
    y as int32
end type

sub NoKeyword( p as Point )         '' the default
    p.x = 99
end sub

sub Copied( byval p as Point )      '' a private copy
    p.x = 77
end sub

function Shifted( byval p as Point, byval dx as int32 ) as Point
    p.x += dx                       '' changes the copy ...
    return p                        '' ... and hands it back
end function

dim as Point a = ( 1, 2 )

NoKeyword( a )
print "after NoKeyword:"; a.x; "   <- the caller's Point changed"

a.x = 1
Copied( a )
print "after Copied:   "; a.x

dim as Point b = Shifted( a, 10 )
print "Shifted gave";  b.x; ", and a is still"; a.x

after NoKeyword: 99   <- the caller's Point changed
after Copied:    1
Shifted gave 11, and a is still 1

With no keyword, the procedure wrote straight into the caller's Point. With byval it worked on a copy. A function can return a type, and Shifted returns its modified copy while the original stays put.

The rule from Chapter 7 holds with more force here, because a type can be large: write byref or byval on every parameter. byref ... as const is the right shape for a type you only read — no copy, and the compiler stops you writing to it.

Where the bytes go

'' layout.tiko -- where the bytes go, and what decides it.

'' Each field sits at a multiple of its own size, so a 4-byte int32
'' after a 1-byte field leaves a 3-byte gap.
type Loose
    a as uint8
    b as int32
    c as uint8
end type

'' The same fields, largest first: the gaps disappear.
type Ordered
    b as int32
    a as uint8
    c as uint8
end type

'' field = n caps the alignment at n bytes.
type Packed field = 1
    a as uint8
    b as int32
    c as uint8
end type

print "Loose   sizeof"; sizeof( Loose ); _
      "  b at"; offsetof( Loose, b ); "  c at"; offsetof( Loose, c )
print "Ordered sizeof"; sizeof( Ordered ); _
      "  a at"; offsetof( Ordered, a ); "  c at"; offsetof( Ordered, c )
print "Packed  sizeof"; sizeof( Packed ); _
      "  b at"; offsetof( Packed, b ); "  c at"; offsetof( Packed, c )

'' field = packs the INSIDE of a type. Put that type inside another,
'' and the outer one still aligns it.
type Holder
    tag as uint8
    p   as Packed
end type
type PackedHolder field = 1
    tag as uint8
    p   as Packed
end type
print "Holder       p at"; offsetof( Holder, p ); _
      "  sizeof"; sizeof( Holder )
print "PackedHolder p at"; offsetof( PackedHolder, p ); _
      "  sizeof"; sizeof( PackedHolder )

Loose   sizeof 12  b at 4  c at 8
Ordered sizeof 8  a at 4  c at 5
Packed  sizeof 6  b at 1  c at 5
Holder       p at 4  sizeof 12
PackedHolder p at 1  sizeof 7

sizeof is a type's size in bytes, and offsetof( T, field ) is where a field starts. Read the first three lines together.

Loose holds 6 bytes of data and takes 12. Each field is placed at a multiple of its own size, so the int32 after a one-byte field starts at offset 4, leaving three unused bytes — and the whole type is rounded up to a multiple of its largest field so that an array of them keeps every element aligned. That padding is not waste for its own sake: the processor reads aligned values in one step and misaligned ones in two, or not at all.

Ordered has the same fields, largest first, and takes 8. Ordering fields from largest to smallest is the cheapest optimisation in this book — no code changes, and every array of the type gets smaller. Exercise 2 puts a number on it.

field = 1 turns alignment off: Packed is 6 bytes with no gaps. Reach for it when the layout is dictated from outside — a file format, a network header, a C struct declared with packing — and not otherwise, because the misaligned reads it creates are slower.

Seeing the layout

The compiler will print the layout it chose. From the directory holding the listing:

tiko --dump-udt layout.tiko

loose size 12 align 4
  loose off a 0
  loose off b 4
  loose off c 8
ordered size 8 align 4
  ordered off b 0
  ordered off a 4
  ordered off c 5
packed size 6 align 4
  packed off a 0
  packed off b 1
  packed off c 5
holder size 12 align 4
  holder off tag 0
  holder off p 4
packedholder size 7 align 4
  packedholder off tag 0
  packedholder off p 1
object size 8 align 8
  object off vptr 0
udts: 6

Names appear in lower case, because tiko's names are not case-sensitive. The align column is the alignment a type asks for when it is placed inside another — which is why packed still says 4, and why the pitfall above happens. object is the built-in base type Chapter 14 introduces; it is listed whether or not you use it.

--dump-udt stops after printing, without building anything. This output was captured at the pin and is identical on both targets.

Several views of the same bytes

'' unions.tiko -- several views of the same bytes.

union Bits32
    asFloat as float32
    asInt   as uint32
    bytes( 0 to 3 ) as uint8
end union

dim as Bits32 u
u.asFloat = 1.0
print "sizeof( Bits32 ) ="; sizeof( Bits32 )
print "1.0 as float32 is the bit pattern &h"; hex( u.asInt, 8 )
print "its highest byte is "; u.bytes( 3 )

'' An anonymous union inside a type: one slot, two meanings, and a
'' field beside it that says which meaning is current.
type Value
    isFloat as boolean
    union
        i as int64
        f as float64
    end union
end type

dim as Value v
v.isFloat = true
v.f = 2.5
print "Value: sizeof"; sizeof( Value ); ", i and f both at offset"; _
      offsetof( Value, i ); offsetof( Value, f )
if v.isFloat then print "holding the float"; v.f

sizeof( Bits32 ) = 4
1.0 as float32 is the bit pattern &h3F800000
its highest byte is 63
Value: sizeof 16, i and f both at offset 8 8
holding the float 2.5

A union is declared like a type, but its fields all start at offset 0: they are different interpretations of one block of memory, and the union is as large as its largest field. Writing asFloat and reading asInt shows the IEEE 754 bit pattern of 1.0, which is the classic use — and also the only kind of use where reading a different field than the one last written is meaningful.

An anonymous union inside a type gives one slot two meanings. Value stores either an int64 or a float64 in the same eight bytes, and the isFloat field beside it records which one is current. That pairing is the whole discipline of a union: something must say which field is live, because the union cannot.

Fields narrower than a byte

'' packetheader.tiko -- fields narrower than a byte.

'' The first 32 bits of an IPv4 header, as bitfields. Each field
'' says how many bits it occupies; together they fill one uint32.
type Header
    version : 4  as uint32
    ihl     : 4  as uint32
    tos     : 8  as uint32
    length  : 16 as uint32
end type

dim as Header h
h.version = 4
h.ihl     = 5
h.tos     = 0
h.length  = 1500

print "sizeof( Header ) ="; sizeof( Header )
print "version "; h.version; "  ihl "; h.ihl; "  length "; h.length

'' The whole thing, read as one number. The first field declared
'' is in the LOWEST bits: version is the last hex digit.
dim as uint32 ptr whole = cast( uint32 ptr, @h )
print "as one uint32: &h"; hex( *whole, 8 )

'' A value too wide for its field is cut down to fit, silently.
h.version = 17
print "after version = 17, version is "; h.version

sizeof( Header ) = 4
version 4  ihl 5  length 1500
as one uint32: &h05DC0054
after version = 17, version is 1

version : 4 as uint32 declares a bitfield: a field four bits wide, carved out of a uint32. Consecutive bitfields share their containing integer, so four of them describe the first 32 bits of an IPv4 header in exactly 4 bytes.

Reading the whole word back shows the allocation order: &h05DC0054 has version (4) in the lowest four bits and length (1500, &h5DC) in the highest sixteen. The first field declared takes the least significant bits. That is the order the targets in this book use, and the order C compilers use on them — but it is not the order a network protocol transmits, which is why a real header parser reads bytes and shifts rather than overlaying a bitfield type on the wire.

--dump-udt reports bitfields at the byte offset of their container, so it cannot show bit positions; the listing measures them by reading the word back.

Naming a type once

'' withblock.tiko -- naming the thing once.

type Settings
    width   as int32
    height  as int32
    title   as ustring
    resizable as boolean
end type

dim as Settings s

with s
    .width     = 800
    .height    = 600
    .title     = "Report"
    .resizable = false
end with

print s.title; ":"; s.width; " x"; s.height; ", resizable "; s.resizable

'' An alias is another name for the same type, not a new type.
type Dimensions as Settings
dim as Dimensions d = s
print "through the alias: "; d.title; d.width

Report: 800 x 600, resizable false
through the alias: Report 800

Inside with s … end with, a field written with a leading dot belongs to s. It is shorthand for filling in a type with many fields, and it reads well for exactly that. Inside a long block it reads less well, because the reader has to remember which variable the dots refer to.

type Dimensions as Settings makes a second name for an existing type. It is the same type, not a new one — a Dimensions and a Settings can be assigned to each other freely. Aliases are for giving a type a name that fits the context, or a shorter one, and Chapter 17 relies on them to name instantiated generics.

Types that refer to each other

'' linked.tiko -- types that point at each other.

'' A type may point at itself: the compiler only needs to know the
'' size of a pointer, which it always does.
type Node
    value as int32
    next_ as Node ptr
end type

dim as Node a, b, c
a.value = 1 : b.value = 2 : c.value = 3
a.next_ = @b : b.next_ = @c : c.next_ = nullptr

dim as Node ptr p = @a
print "the list:";
do while p <> nullptr
    print p->value;
    p = p->next_
loop
print

'' Two types that point at EACH OTHER need one of them named before
'' it is declared. A forward alias does that.
type ChildFwd as Child

type Parent
    firstChild as ChildFwd ptr
end type

type Child
    owner as Parent ptr
end type

dim as Parent mum
dim as Child kid
mum.firstChild = @kid
kid.owner = @mum
print "the child's owner is the parent: "; cbool( kid.owner = @mum )

the list: 1 2 3
the child's owner is the parent: true

A type can hold a pointer to its own kind, which is how linked structures are built. It cannot hold a value of its own kind — that would be infinitely large — but a pointer is 8 bytes on every target this book uses, so the compiler needs nothing more. Chapter 11 covers pointers properly; here they are only the arrows between records.

Two types that point at each other are a different problem. Whichever is written first mentions the other before it is declared, and the compiler refuses that with TKO0784: the type 'child' is used above its declaration. A forward alias — type ChildFwd as Child — names the type before it exists, and a pointer to it can be used straight away.

A forward alias that is never completed is an opaque type:

'' opaque.tiko -- a type known only by name.
type Handle as HandleImpl          '' never declared in this file

dim as Handle ptr h = nullptr      '' a pointer is fine
dim as Handle whole                '' a value is not: how big is it?

opaque.tiko(5,8): error TKO0810: 'handleimpl' is an incomplete type: nothing declares it, so only a pointer to it may be used here
     |
   5 | dim as Handle whole                '' a value is not: how big is it?
     |        ^

That refusal is the feature. A library can hand out Handle ptr values while keeping the contents of HandleImpl to itself, and callers cannot depend on what they cannot see. Chapter 29 meets the same pattern from C's side.

Arrays of records

'' udtarray.tiko -- an array whose elements are records.

type Reading
    station as ustring
    celsius as float64
end type

'' A small function that builds one record keeps the call sites to a
'' single readable line each.
function MakeReading( byref station as ustring, _
                      byval celsius as float64 ) as Reading
    dim as Reading r
    r.station = station
    r.celsius = celsius
    return r
end function

dim as Reading log_( 1 to 4 )
log_( 1 ) = MakeReading( "north", 11.5 )
log_( 2 ) = MakeReading( "south", 18.0 )
log_( 3 ) = MakeReading( "east",  14.25 )
log_( 4 ) = MakeReading( "west",  9.75 )

'' By index, when the position matters.
dim as int32 warmest = lbound( log_ )
for i as int32 = lbound( log_ ) to ubound( log_ )
    if log_( i ).celsius > log_( warmest ).celsius then warmest = i
next
print "warmest: "; log_( warmest ).station; log_( warmest ).celsius

'' for each byref, when it does not. Each record is used in place,
'' without being copied.
dim as float64 total = 0
for each byref r as Reading in log_
    total += r.celsius
next
print "mean:"; total / ( ubound( log_ ) - lbound( log_ ) + 1 )

'' byref also lets the loop change them.
for each byref r as Reading in log_
    r.celsius = r.celsius * 9 / 5 + 32
next
print "north in fahrenheit:"; log_( 1 ).celsius

warmest: south 18
mean: 13.375
north in fahrenheit: 52.7

An array of a type works like any array, and Chapter 8's advice carries over: index when the position matters, for each byref when it does not. For records, byref is more than a style choice, because the alternative copies every field — text included — on every pass.

MakeReading builds one record per call, which keeps each line of setup readable. Chapter 12 turns that function into a constructor, which is where it belongs.

What it costs

• A type costs exactly its sizeof. There is no hidden header and no per-object allocation until Chapter 12 adds virtual methods, and even then it is one pointer.
• Padding is real memory. Ordering fields largest first is free and can save a third.
• field = 1 trades speed for exactness. Use it where a format demands it, nowhere else.
• Assignment copies everything, and a ustring field is a separate allocation copied each time. Pass large types byref.
• A union costs its largest member, and the field that says which member is live.

Summary

• type groups named fields; a field may have an initialiser, and may itself be a type. ( a, b ) fills fields in order. Braces are for arrays — { a, b } on a type is TKO0845
• Assignment copies the whole value, text included. = does not compare types — TKO0720 — until Chapter 13 defines it.
• With no keyword, a type is passed byref. byval copies; a function may return a type.
• sizeof and offsetof measure layout. Fields align to their own size, and the type rounds up to its largest field — order fields largest first.
• field = n caps alignment — for the inside of that type only; nested types need their own.
• tiko --dump-udt file.tiko prints the layout the compiler chose.
• A union overlays its fields at offset 0; something beside it must record which one is live.
• Bitfields pack narrow fields into an integer, first field in the lowest bits, and truncate an oversized value silently.
• with shortens repeated access; type A as B is an alias for the same type.
• A type may point at itself; types that point at each other need a forward alias, and an alias never completed is an opaque type usable only through a pointer.
• Arrays of types: index for position, for each otherwise — for each byref for records, which the by-value form would copy field by field.
• type( ... ) cannot build a record holding text (TKO0929); build it through a function or a constructor.

Exercises

1. Declare a 16-byte record header — magic, version, flags, length, checksum — that must match a published binary format exactly, and have the program verify every offset against the spec instead of trusting it. {solution="ex10-01"}
2. Take a type with six fields of mixed sizes declared carelessly, reorder them largest first, and print both sizes and what a million of each would cost. {solution="ex10-02"}
3. Put a field = 1 type inside an ordinary one and print the offset of the inner type. Then fix the outer type so the whole thing has no padding. How many field = did it take?
4. Write a union holding a float64 and an array of eight uint8, store −0.0 and 0.0, and print the bytes of each. Which byte differs, and why does = on the two floats still say they are equal?
5. Build a three-element linked list of Nodes and write a procedure that prints it given only the first node's address. What should the procedure do if it is handed 0?

Pointers, References and Memory

Every variable so far has lived somewhere the compiler chose and lasted as long as the compiler decided. That is the right default, and most of a tiko program never needs anything else.

This chapter is about the rest: memory you ask for while the program runs, structures that link to each other, and addresses you hand to code that expects them — C libraries, the operating system, a buffer shared between two parts of a program. A pointer is the address of a value, and with it comes the responsibility the compiler was carrying for you: knowing that the address is valid, and giving the memory back.

Chapter 22 shows the tools that carry most of that responsibility again — destructors, defer, shared pointers. Learn the raw form first, because every one of those tools is built on it.

An address, and what lives at it

'' basics.tiko -- an address, and what lives at it.

dim as int32 n = 10
dim as int32 ptr p = @n          '' @ takes the address

print "*p ="; *p                 '' * reads what p points at
*p = 20                          '' ... or writes it
print "n is now"; n

print "varptr( n ) is the same address: "; cbool( varptr( n ) = p )

'' A pointer to a type reaches its fields with ->.
type Point
    x as int32
    y as int32
end type

dim as Point pt = ( 3, 4 )
dim as Point ptr pp = @pt
pp->x = 30
print "pt.x ="; pt.x; ", and (*pp).y ="; ( *pp ).y

'' nullptr is the null pointer: it points at nothing.
dim as int32 ptr nothing_ = nullptr
print "nothing_ is null: "; cbool( nothing_ = nullptr )

*p = 10
n is now 20
varptr( n ) is the same address: true
pt.x = 30, and (*pp).y = 4
nothing_ is null: true

Writing through *p changed n, because p holds n's address — there is one value and two ways to reach it. That sharing is the whole point of a pointer, and the source of every pointer bug.

nullptr, not 0

0 works as a null pointer too: p = 0 and if p = 0 are legal, and existing code is full of them. But 0 is an integer, and wherever the compiler has to choose a type for it, it chooses an integer's:

'' nullptr.tiko -- the null pointer, spelled as a pointer.

sub Show overload ( byval n as integer )
    print "  Show( integer )"
end sub

sub Show overload ( byval p as any ptr )
    print "  Show( any ptr )"
end sub

print "0 is an integer, so it picks the integer overload:"
Show( 0 )
print "nullptr is a pointer:"
Show( nullptr )

'' var takes the type of what it is given.
var a = 0
var b = nullptr
print "var from 0, then var from nullptr:"
Show( a )
Show( b )

0 is an integer, so it picks the integer overload:
  Show( integer )
nullptr is a pointer:
  Show( any ptr )
var from 0, then var from nullptr:
  Show( integer )
  Show( any ptr )

nullptr is an any ptr whose value is 0, so it converts to every pointer type — typed pointers, zstring ptr, procedure pointers — and an overload, var and a generic all see a pointer. Against two pointer overloads it matches both equally and the call is an error; cptr( zstring ptr, nullptr ) names the one you mean. This book writes nullptr everywhere.

Arithmetic counts elements

'' arithmetic.tiko -- pointer arithmetic counts elements, not bytes.

dim as int32 a( 0 to 4 ) = { 10, 20, 30, 40, 50 }
dim as int32 ptr p = @a( 0 )

print "p[2] ="; p[2]; ",  *( p + 3 ) ="; *( p + 3 )

'' How far does + 1 move? It depends on what the pointer points at.
dim as uint8 ptr b = cast( uint8 ptr, p )
dim as any ptr v = p

print "int32 ptr + 1 moves"; cast( integer, p + 1 ) - cast( integer, p )
print "uint8 ptr + 1 moves"; cast( integer, b + 1 ) - cast( integer, b )
print "any ptr   + 1 moves"; cast( integer, v + 1 ) - cast( integer, v )

'' Walking a run of elements with a pointer.
dim as int32 ptr last_ = @a( 4 )
dim as int32 ptr q = p
print "walked:";
do while q <= last_
    print *q;
    q += 1
loop
print

p[2] = 30,  *( p + 3 ) = 40
int32 ptr + 1 moves 4
uint8 ptr + 1 moves 1
any ptr   + 1 moves 1
walked: 10 20 30 40 50

p + 1 does not mean "one byte later". It means the next element, so on an int32 ptr it moves four bytes, on a uint8 ptr one, and on an any ptr — which has no element size — one. p[2] is *( p + 2 ), which is why a pointer to an array's first element can be indexed like the array.

Comparison works as you would expect — q <= last_ walks a run of elements — and subtracting two pointers into the same array gives the distance between them in elements. Comparing pointers into different blocks of memory tells you nothing useful.

Changing what a pointer points at

'' conversions.tiko -- any ptr, and changing what a pointer points at.

dim as int32 n = 70000

'' any ptr holds an address of anything. Every pointer converts to
'' it, and it converts back to any pointer type, without a cast.
dim as any ptr raw = @n
dim as int32 ptr back = raw
print "through any ptr and back:"; *back

'' Between two typed pointers, say what you mean with cptr or cast.
dim as uint8 ptr bytes = cptr( uint8 ptr, @n )
print "the bytes of 70000:";
for i as int32 = 0 to 3 : print " "; bytes[i]; : next
print

dim as uint16 ptr halves = cast( uint16 ptr, @n )
print "as two uint16: "; halves[0]; " "; halves[1]
print "and 4464 + 1 * 65536 = 70000 -- the low half comes first"

through any ptr and back: 70000
the bytes of 70000: 112 17 1 0
as two uint16: 4464 1
and 4464 + 1 * 65536 = 70000 -- the low half comes first

any ptr is an address with no type attached. Every pointer converts to it, and it converts back to any pointer type without a cast — which is what lets one procedure accept the address of anything, and what allocate returns.

Between two typed pointers the compiler wants you to say what you mean. cptr( T ptr, p ) and cast( T ptr, p ) reinterpret the address; without one, assigning a float64 ptr from an int32 ptr draws TKO0830: suspicious pointer assignment — a warning, because sometimes it is exactly what you want, and usually it is a bug.

The listing reads the four bytes of 70000 and then the two 16-bit halves. Both show the low part first: on every target in this book the least significant byte has the lowest address.

const on either side

'' constptr.tiko -- const on either side of ptr.
dim as int32 n = 5, m = 6

dim as const int32 ptr readOnly = @n    '' can move; cannot write
dim as int32 const ptr fixed_ = @n      '' cannot move; can write

readOnly = @m                           '' fine
*fixed_ = 9                             '' fine

*readOnly = 1                           '' refused
fixed_ = @m                             '' refused

constptr.tiko(11,1): error TKO0617: cannot modify 'fixed_': it is a constant
     |
  11 | fixed_ = @m                             '' refused
     | ^
constptr.tiko(10,1): error TKO0617: cannot modify '*readonly': it is a constant
     |
  10 | *readOnly = 1                           '' refused
     | ^

const int32 ptr is a pointer to constant data: it may be pointed elsewhere, but not written through. int32 const ptr is a constant pointer: it always points at the same place, but what is there may change. Read the declaration from the right — "a const pointer to an int32", "a pointer to a const int32" — and the rule falls out.

Asking for memory

'' buffer.tiko -- memory the program asks for, and gives back.

'' allocate returns raw memory: uninitialised, and yours to free.
dim as int32 ptr p = allocate( 4 * sizeof( int32 ) )
if p = nullptr then
    print "out of memory"
    end 1
end if

for i as int32 = 0 to 3 : p[i] = ( i + 1 ) * 10 : next
print "allocated:"; p[0]; p[1]; p[2]; p[3]

'' reallocate grows (or shrinks) a block, keeping its contents. It
'' may MOVE the block, so the old pointer must not be used again.
dim as int32 ptr grown = reallocate( p, 6 * sizeof( int32 ) )
if grown = nullptr then
    deallocate( p )             '' on failure the old block survives
    end 1
end if
p = grown
p[4] = 50 : p[5] = 60
print "reallocated:"; p[0]; p[3]; " and new"; p[4]; p[5]
deallocate( p )
p = nullptr                     '' so nothing can use it by mistake

'' callocate zeroes the memory it returns.
dim as int32 ptr z = callocate( 4, sizeof( int32 ) )
print "callocated:"; z[0]; z[1]; z[2]; z[3]

'' clear sets every BYTE to a value -- here 255, so each int32 is -1.
clear( *z, 255, 4 * sizeof( int32 ) )
print "after clear( 255 ): "; z[0]; " "; z[3]
deallocate( z )

deallocate( nullptr )           '' freeing null does nothing, safely
print "done"

allocated: 10 20 30 40
reallocated: 10 40 and new 50 60
callocated: 0 0 0 0
after clear( 255 ): -1 -1
done

Three habits are in that listing, and each prevents a class of bug:

• Check for null. Allocation can fail, and a null pointer used as a buffer fails much later and much less clearly.
• Keep reallocate's result in a new variable. On failure it returns null and leaves the old block alone; writing the result straight over the only pointer you had leaks that block.
• Set the pointer to nullptr after deallocate. A freed pointer still holds an address, and using it is the worst kind of bug: it usually works.

clear sets bytes, so clear( *z, 255, … ) makes each int32 −1, not 255. It is for zeroing and for byte buffers, not for filling an array with a value.

Freeing on every way out

Those habits make each call correct. They do not make the procedure correct: a procedure with three returns after its allocate needs three deallocates, and the fourth return someone adds later leaks. defer, from Chapter 6, writes the release once, beside the allocation, and runs it on every way out of the block:

'' deferfree.tiko -- defer pairs each allocate with its deallocate.

'' The sum of the squares 1, 4, 9 ... of the first n numbers, built in
'' a scratch buffer. Every return after the allocate frees it.
function SumOfSquares( byval n as int32 ) as int64
    if n < 1 then return 0              '' nothing allocated yet

    dim as int64 ptr squares = allocate( n * sizeof( int64 ) )
    defer deallocate( squares )         '' runs on every return below
    if squares = nullptr then return -1 '' freeing null does nothing

    for i as int32 = 0 to n - 1
        squares[i] = cast( int64, i + 1 ) * ( i + 1 )
    next
    dim as int64 total = 0
    for i as int32 = 0 to n - 1 : total += squares[i] : next
    return total
end function

'' A deferred statement reads its variables when it RUNS, not when it
'' is written: when reallocate moves the block, defer frees the new one.
function Collect( byval count as int32, byref slots as int32 ) as int32
    slots = 2
    dim as int32 ptr items = allocate( slots * sizeof( int32 ) )
    defer deallocate( items )
    if items = nullptr then return -1

    for i as int32 = 0 to count - 1
        if i = slots then
            dim as int32 wanted = slots * 2
            dim as int32 ptr grown = _
                reallocate( items, wanted * sizeof( int32 ) )
            if grown = nullptr then return -1 '' items still freed
            items = grown
            slots = wanted
        end if
        items[i] = i * i
    next

    dim as int32 total = 0
    for i as int32 = 0 to count - 1 : total += items[i] : next
    return total
end function

print "Sum of squares through a scratch buffer"
print "  n = 0:  " & SumOfSquares( 0 )
print "  n = 3:  " & SumOfSquares( 3 )
print "  n = 10: " & SumOfSquares( 10 )

dim as int32 slots
dim as int32 total = Collect( 5, slots )
print "Collect 5 squares, starting with 2 slots"
print "  total " & total & " in " & slots & " slots"

Sum of squares through a scratch buffer
  n = 0:  0
  n = 3:  14
  n = 10: 385
Collect 5 squares, starting with 2 slots
  total 30 in 8 slots

• Write the defer on the line after the allocate, before the check for null. deallocate( nullptr ) does nothing, so the failure path is covered too, and the pair cannot drift apart.
• A deferred statement reads its variables when it runs, at the end of the block, not when the defer line is reached. That is why Collect can store reallocate's result in items and still free the right block. It also means a block you free early by hand needs its pointer set to nullptr afterwards, or the defer frees it a second time.
• Allocate and defer in the procedure's own body, never inside an if. Chapter 6's pitfall applies with full force: the block is freed at end if, and the pointer still holds its address.

Typed allocation

'' newdelete.tiko -- typed allocation.

'' new knows the type: no sizeof, no cast, and the memory is zeroed.
dim as int32 ptr one = new int32( 42 )
print "new int32( 42 ):"; *one
delete one

dim as float64 ptr several = new float64[ 3 ]
several[0] = 1.5 : several[2] = 2.5
print "new float64[ 3 ]:"; several[0]; several[1]; several[2]
delete[] several                '' [] matches the [] in new

type Point
    x as int32
    y as int32
end type

dim as Point ptr pt = new Point
print "new Point starts at"; pt->x; pt->y
pt->x = 3 : pt->y = 4
print "and is now"; pt->x; pt->y
delete pt

new int32( 42 ): 42
new float64[ 3 ]: 1.5 0 2.5
new Point starts at 0 0
and is now 3 4

new T allocates one T, zeroed; new T[ n ] allocates n of them. No sizeof, no cast, and the pointer comes back typed. Pair them exactly: memory from new goes back with delete, memory from new[] with delete[], and memory from allocate with deallocate. Mixing the families is undefined, and Chapter 12 shows the version where it visibly breaks — once a type has a destructor, delete runs it and deallocate does not.

new takes a value too. For a type with no constructor the arguments fill the fields in order, as dim … = ( 3, 4 ) does, and any left out are zero:

'' newargs.tiko -- new with a field list.
''
'' For a type with no constructor, the arguments to new fill its
'' fields in order, as they do in a dim. This prints 3 4.

type Point
    x as int32
    y as int32
end type

dim as Point ptr pt = new Point( 3, 4 )
print pt->x; pt->y
delete pt

 3 4

new int32( 42 ) works the same way for a single value. A type with a constructor takes its arguments through that instead — Chapter 12.

Structures that link

'' linkednodes.tiko -- a list built on the heap, walked, and freed.

type Node
    value as int32
    next_ as Node ptr
end type

'' Push a value on the front. The new node points at the old head,
'' and becomes the head.
function Push( byval head as Node ptr, byval v as int32 ) as Node ptr
    dim as Node ptr n = new Node
    n->value = v
    n->next_ = head
    return n
end function

dim as Node ptr head = nullptr
for i as int32 = 1 to 5
    head = Push( head, i * i )
next

print "the list:";
dim as Node ptr p = head
do while p <> nullptr
    print p->value;
    p = p->next_
loop
print

'' Free every node. Read next_ BEFORE deleting the node it is in.
dim as int32 freed = 0
do while head <> nullptr
    dim as Node ptr after = head->next_
    delete head
    head = after
    freed += 1
loop
print "freed"; freed; " nodes"

the list: 25 16 9 4 1
freed 5 nodes

A linked list is the canonical heap structure: each node allocated separately, each pointing at the next, the end marked by a null. Push puts a new node at the front in constant time, which is why the list prints in reverse order of insertion.

Freeing it shows the rule that matters for every linked structure: read the link before you delete the node that holds it. head->next_ after delete head reads freed memory. It will often produce the right answer, which is exactly why it ships.

References

'' references.tiko -- a second name for an existing variable.

'' A byref parameter is a reference the caller sets up.
sub Double_( byref n as int32 )
    n *= 2
end sub

dim as int32 a = 21
Double_( a )
print "after Double_:"; a

'' dim byref makes a reference inside a procedure or module. It must
'' be bound when it is declared, and it stays bound to that variable.
dim as int32 b = 5
dim byref as int32 alias_ = b
alias_ = 9
print "b ="; b; ", same address: "; cbool( @alias_ = @b )

'' A reference reads like a variable; a pointer reads like an
'' address. Same machine code, different promises.
dim as int32 ptr ptr_ = @b
*ptr_ = 11
print "b ="; b; " via a pointer, and"; alias_; " via the reference"

after Double_: 42
b = 9, same address: true
b = 11 via a pointer, and 11 via the reference

A reference is a second name for an existing variable. byref parameters are references the caller sets up; dim byref makes one directly, and it must be bound when declared — there is no null reference and no rebinding.

Underneath, a reference and a pointer are the same machine address. The difference is the promise: a reference always refers to something, and is used like the variable it names. Prefer a reference whenever "nothing" is not a valid answer, and a pointer when it is.

The memory behind a string

'' emptystrptr.tiko -- an empty string has no characters to point at.

dim as ustring full = "hi"
dim as ustring empty_ = ""

print "strptr( full )   is null: "; cbool( strptr( full ) = nullptr )
print "strptr( empty_ ) is null: "; cbool( strptr( empty_ ) = nullptr )

'' Emptying a string by assignment releases its characters too.
dim as ustring was = "something"
was = ""
print "after was = """", strptr is null: "; _
      cbool( strptr( was ) = nullptr )

strptr( full )   is null: false
strptr( empty_ ) is null: true
after was = "", strptr is null: true

strptr( s ) is the address of a string's characters, which is what a C function wants. A string owns that memory, and moves or frees it whenever it changes — so the pointer is valid only until the next modification of the string, and only while the string exists.

Reading and writing raw memory

'' peekpoke.tiko -- reading and writing a value of a stated type at
'' an address.

dim as int32 v = &h01020304

'' The lowest-addressed byte is the least significant: little-endian.
'' @v is an int32 ptr, so @v + 3 would be 12 bytes on. Step in bytes
'' through a uint8 ptr instead.
dim as uint8 ptr at = cast( uint8 ptr, @v )
print "byte 0 = "; peek( uint8, at ); _
      "  byte 3 = "; peek( uint8, at + 3 )

poke uint8, @v, &hFF                '' overwrite one byte
print "after poking byte 0: &h"; hex( v, 8 )

poke int32, @v, 7                   '' overwrite all four
print "after poking an int32:"; v

byte 0 = 4  byte 3 = 1
after poking byte 0: &h010203FF
after poking an int32: 7

peek( T, address ) reads a T at an address, and poke T, address, value writes one. They are the explicit spellings of *cptr( T ptr, address ), useful when the type at an address is decided by the code rather than by a declaration — parsing a binary format, for instance.

The output confirms what conversions.tiko showed: byte 0 of &h01020304 is 4. These targets are little-endian.

When the pointer is null

'' nullcheck.tiko -- following a null pointer, under -exx.
#cmdline "-exx"

dim as int32 ptr p = nullptr
print *p
print "not reached"

Aborting due to runtime error 7 (null pointer access) at line 5 of nullcheck.tiko::()

Under -exx, reading or writing through a null pointer with * stops the program with runtime error 7, and the process exits with status 7, instead of crashing somewhere further on. Without -exx the same program crashes with an access violation.

The same check covers -> and indexing — p->x and p[i] through null stop with error 7 too. That is the common case: a null object pointer is reached through -> far more often than through *.

'' nullmember.tiko -- a field through a null pointer, under -exx.
''
'' -exx checks p->x as it checks *p, and stops with the same runtime
'' error 7 that nullcheck.tiko shows.
#cmdline "-exx"

type Point
    x as int32
end type

dim as Point ptr p = nullptr
print p->x
print "not reached"

Aborting due to runtime error 7 (null pointer access) at line 12 of nullmember.tiko::()

What it costs

• A pointer costs 8 bytes and one indirection on every access. Following a chain of them — a->b->c — is a chain of dependent memory loads, which is slower than walking an array.
• Every allocation has overhead in time and bookkeeping. A thousand small news cost more than one new[] of a thousand.
• A linked list scatters its nodes across memory, so walking it misses the cache where an array would not. Use one when you need constant-time insertion in the middle, not by default.
• Memory you allocate is memory you must free. Every exit path, including errors. defer covers the paths out of one procedure; Chapter 22 covers memory that outlives it.

Summary

• T ptr is an address; @ takes one, * follows it, -> reaches a field, and nullptr is null.
• Write nullptr, not 0. Both are null, but 0 is an integer to an overload, var and a generic.
• Pointer arithmetic counts elements: p + 1 moves sizeof( T ) bytes. Step in bytes through a uint8 ptr.
• any ptr converts to and from every pointer type; between typed pointers use cptr or cast.
• An any ptr is not dereferenced (TKO0933); convert it to a typed pointer first.
• const T ptr cannot write through; T const ptr cannot be moved.
• allocate/callocate/reallocate/deallocate: check for null, keep reallocate's result separately, null the pointer after freeing. clear sets bytes.
• defer deallocate( p ) on the line after the allocate frees on every way out. It reads p when it runs, so it follows a reallocate.
• new/delete and new[]/delete[] allocate typed and zeroed memory — pair each family with its own release. new T( a, b ) fills a constructor-less type's fields in order.
• Read a node's link before deleting the node.
• References are bound once and never null; prefer them when "nothing" is not a valid answer.
• strptr of an empty string is null, and any strptr is valid only until the string changes.
• peek/poke read and write a stated type at an address. The targets are little-endian.
• -exx traps *p, p->x and p[i] through null with runtime error 7. Still test a pointer that may be null before you follow it.

Exercises

1. Write a growable int32 buffer that doubles its capacity when it fills, using reallocate, and report each growth. Append twenty items and say how many times the buffer was copied. {solution="ex11-01"}
2. Reverse a singly linked list in place by changing the links, not the values. Make sure an empty list and a one-node list both survive. {solution="ex11-02"}
3. Print the address of each element of an int64 array and of a uint8 array. What is the difference between consecutive addresses in each, and why?
4. Allocate a block, free it, and then read through the old pointer. Run it several times. What does it print, and why is "it printed the right value" not evidence that the code is correct?
5. Write a procedure that takes a string and passes strptr of it to a procedure expecting a zstring ptr, which prints the text. What must it do when the string is empty?

Objects

Chapter 10's types held data and left every rule about that data to the code around them. An account balance could be set negative from anywhere; a buffer's length could disagree with its memory; a record could be built half-filled. Nothing in the type said otherwise, because the type could not say anything.

An object is a type that carries its rules with it. It has procedures of its own, it decides how it is created and destroyed, and it can keep its fields to itself so that the only way to change them is through code that keeps them consistent. None of that costs anything at run time: a method is an ordinary procedure with the object passed in, and an object is exactly as large as its fields.

A type with its own procedures

'' account.tiko -- a type that looks after its own data.

type Account
    declare constructor( byref owner as ustring, _
                         byval opening as int64 )
    declare destructor( )
    declare sub Deposit( byval amount as int64 )
    declare function Withdraw( byval amount as int64 ) as boolean
    declare function Balance( ) as int64

    owner_   as ustring
    balance_ as int64
end type

'' Bodies live outside the type, named Type.Member.
constructor Account( byref owner as ustring, byval opening as int64 )
    this.owner_ = owner          '' this is the object being built
    balance_ = opening           '' ... and "this." may be left off
    print "  opened "; owner_
end constructor

destructor Account( )
    print "  closed "; owner_; " with"; balance_
end destructor

sub Account.Deposit( byval amount as int64 )
    balance_ += amount
end sub

function Account.Withdraw( byval amount as int64 ) as boolean
    if amount > balance_ then return false
    balance_ -= amount
    return true
end function

function Account.Balance( ) as int64
    return balance_
end function

scope
    dim as Account ada = Account( "Ada", 100 )
    dim as Account grace = Account( "Grace", 20 )

    ada.Deposit( 50 )
    print "Ada withdraws 500: "; ada.Withdraw( 500 )
    print "Ada withdraws 30:  "; ada.Withdraw( 30 )
    print "Ada's balance:"; ada.Balance( )
end scope                        '' both destructors run here
print "after the scope"

  opened Ada
  opened Grace
Ada withdraws 500: false
Ada withdraws 30:  true
Ada's balance: 120
  closed Grace with 20
  closed Ada with 120
after the scope

A method is declared inside the type and defined outside it, named Type.Member. Inside, the object it was called on is this, and this. may be left off when nothing else has the same name — balance_ and this.balance_ are the same field.

The constructor runs when an object is created and the destructor when it is destroyed. For an object in a scope, that is at the end of the scope, in the reverse order of creation: Grace was opened after Ada and closed before her. That ordering is deliberate — something created later may depend on something created earlier, never the other way round.

The trailing underscore on the fields is a convention, not syntax. It marks the data the methods look after, and keeps names free for the methods that expose it.

Constructors

'' constructors.tiko -- several ways in, one of them implicit.

type Temp
    declare constructor( )
    declare constructor( byval c as float64 )
    declare constructor( byref text as ustring )
    celsius as float64
end type

constructor Temp( )
    celsius = 20
    print "  default constructor"
end constructor

constructor Temp( byval c as float64 )
    celsius = c
    print "  from a float64"
end constructor

'' Delegation: hand the work to another constructor, then add to it.
constructor Temp( byref text as ustring )
    this.constructor( val( text ) )
    print "  ... reached through the text constructor"
end constructor

sub Show( byref t as const Temp )
    print "  Show sees"; t.celsius
end sub

print "dim as Temp a"
dim as Temp a

print "dim as Temp b = Temp( 5.0 )"
dim as Temp b = Temp( 5.0 )

print "dim as Temp c = Temp( ""31.5"" )"
dim as Temp c = Temp( "31.5" )

'' A one-argument constructor is also a CONVERSION: the compiler will
'' call it on its own whenever a float64 is offered where a Temp is
'' wanted.
print "dim as Temp d = 7.5"
dim as Temp d = 7.5
print "Show( 12.0 )"
Show( 12.0 )

print "values:"; a.celsius; b.celsius; c.celsius; d.celsius

dim as Temp a
  default constructor
dim as Temp b = Temp( 5.0 )
  from a float64
dim as Temp c = Temp( "31.5" )
  from a float64
  ... reached through the text constructor
dim as Temp d = 7.5
  from a float64
Show( 12.0 )
  from a float64
  Show sees 12
values: 20 5 31.5 7.5

A type may have as many constructors as it needs, distinguished by their parameters like overloaded procedures. The one with no parameters is the default constructor, used by a plain dim.

One constructor may hand its work to another: this.constructor( val( text ) ) runs the float64 constructor and then carries on. Put shared setup in one constructor and delegate to it, rather than repeating it.

Copying an object that owns something

Chapter 10's types copied themselves correctly: every field was copied, strings included. That stops being true the moment a type holds a pointer to memory it owns. Copying the pointer gives two objects the same block — and two destructors that each free it.

Measured, with a Buffer that has a constructor and destructor but nothing else, after dim as Buffer b = a:

a.data_[0] = 99   same block: true
  freeing 31008
  freeing 31008

The write through b appeared in a, both destructors freed the same address — the number printed differs from run to run; that it is the same number twice does not — and the process then died — 0xC0000374, heap corruption, on Windows, and free(): double free on Linux. fbc behaves identically: this is the language's rule, not a defect.

The fix is to say what copying means:

'' copying.tiko -- a type that owns memory must say how to copy it.

type Buffer
    declare constructor( byval n as int32 )
    declare constructor( byref other as const Buffer )    '' copy
    declare operator let( byref other as const Buffer )   '' assign
    declare destructor( )
    items as int32 ptr
    size  as int32
end type

constructor Buffer( byval n as int32 )
    items = callocate( n, sizeof( int32 ) )
    size = n
end constructor

'' Copy construction: a brand-new object, so there is nothing to free.
constructor Buffer( byref other as const Buffer )
    size = other.size
    items = allocate( size * sizeof( int32 ) )
    for i as int32 = 0 to size - 1 : items[i] = other.items[i] : next
    print "  copy constructor"
end constructor

'' Assignment: this object already owns memory. Free it first -- but
'' not when assigning an object to itself, or the source goes too.
operator Buffer.let( byref other as const Buffer )
    if @this = @other then
        print "  let: assigned to itself, nothing to do"
        exit operator
    end if
    deallocate( items )
    size = other.size
    items = allocate( size * sizeof( int32 ) )
    for i as int32 = 0 to size - 1 : items[i] = other.items[i] : next
    print "  let"
end operator

destructor Buffer( )
    deallocate( items )
end destructor

scope
    dim as Buffer a = Buffer( 4 )
    a.items[0] = 1

    dim as Buffer b = a           '' copy constructor
    b.items[0] = 99
    print "a[0] ="; a.items[0]; "  b[0] ="; b.items[0]; _
          "  separate memory: "; cbool( a.items <> b.items )

    dim as Buffer c = Buffer( 2 )
    c = a                         '' operator let
    print "c now has size"; c.size

    c = c                         '' must not destroy itself
end scope
print "every buffer freed exactly once"

  copy constructor
a[0] = 1  b[0] = 99  separate memory: true
  let
c now has size 4
  let: assigned to itself, nothing to do
every buffer freed exactly once

Two procedures are involved, because there are two ways an object gets a copy:

• The copy constructor builds a new object from an existing one — dim as Buffer b = a, or passing byval. There is nothing to clean up first, because the object did not exist.
• operator let assigns into an object that already exists — c = a. It must release what the object holds before taking the copy, and it must do nothing when assigned to itself, or it frees the very data it is about to copy.

Properties

'' properties.tiko -- methods that read like fields.

type Thermostat
    declare property Celsius( ) as float64                '' get
    declare property Celsius( byval c as float64 )        '' set
    declare property Fahrenheit( ) as float64             '' get only
    declare property Reading( byval i as int32 ) as float64
    declare property Reading( byval i as int32, byval v as float64 )
  private:
    c_ as float64
    readings( 0 to 2 ) as float64
end type

property Thermostat.Celsius( ) as float64
    return c_
end property

'' The setter is where a type defends its own rules.
property Thermostat.Celsius( byval c as float64 )
    if c < -273.15 then c = -273.15
    c_ = c
end property

property Thermostat.Fahrenheit( ) as float64
    return c_ * 9 / 5 + 32
end property

property Thermostat.Reading( byval i as int32 ) as float64
    return readings( i )
end property

property Thermostat.Reading( byval i as int32, byval v as float64 )
    readings( i ) = v
end property

dim as Thermostat t
t.Celsius = 21.5
print "celsius"; t.Celsius; ", fahrenheit"; t.Fahrenheit

t.Celsius = -500
print "asked for -500, got "; t.Celsius

t.Reading( 1 ) = 3.5               '' an indexed property
print "reading 1 is"; t.Reading( 1 )

celsius 21.5, fahrenheit 70.7
asked for -500, got -273.15
reading 1 is 3.5

A property is a pair of methods that read and write like a field. t.Celsius = 21.5 calls the setter; t.Celsius in an expression calls the getter. The point is the setter: it is where the type enforces its own rules, so asking for −500 °C gives absolute zero rather than an impossible temperature.

A property with only a getter is read-only — Fahrenheit is computed from the same field rather than stored, so it can never disagree with Celsius. A property may take an index, as Reading( i ) does, which reads like an array whose elements the type controls.

A property with only a setter is write-only: reading it is refused, TKO0934.

t.Celsius += 2.5 works on a property with both accessors: it calls the getter, applies the operator and calls the setter. So does swap, and a property passed to a byref parameter is written back through its setter when the call returns.

Auto-properties

Most properties only store a value. An auto-property says so in its declaration, by ending it with its accessors; the compiler writes both over a hidden field:

type Person
    declare property Name as ustring get, set           '' read and written like a field
    declare property Age as int32 = 18 get, private set '' anyone reads, only Person writes
    declare property Id as int32 get, init              '' set once, by the constructor
    declare constructor( byval newId as int32 )
    declare sub Birthday( )
end type

constructor Person( byval newId as int32 )
    Id = newId
end constructor

sub Person.Birthday( )
    Age += 1
end sub

dim as Person p = Person( 42 )
p.Name = "Ann"
p.Birthday( )
print p.Name; ", age"; p.Age; ", id"; p.Id

Ann, age 19, id 42

get, set behaves like a plain field. get, private set can be read anywhere but written only by the type's own code — the setter may be less visible than the getter, never more. get, init (or get alone) can be assigned only by a constructor, which makes an identifier that never changes after the object is built. = 18 gives the hidden field its first value. An auto-property has no code of its own: as soon as the setter has a rule to keep, write an ordinary property.

Property blocks

A property block keeps the getter and setter together in one definition:

type Thermostat
    declare property Celsius( ) as float64
    declare property Celsius( byval value as float64 )
    declare property Fahrenheit( ) as float64
  private:
    celsius_ as float64
end type

'' Both accessors in one block. In set, value is what was assigned;
'' in get, value is what will be returned.
property Thermostat.Celsius as float64
    get
        value = celsius_
    end get
    set
        if value < -273.15 then value = -273.15
        celsius_ = value
    end set
end property

property Thermostat.Fahrenheit as float64
    get
        return celsius_ * 9 / 5 + 32
    end get
end property

dim as Thermostat t
t.Celsius = 21.5
t.Celsius += 2.5                    '' get, add, set
print "celsius"; t.Celsius; ", fahrenheit"; t.Fahrenheit
t.Celsius = -500
print "asked for -500, got "; t.Celsius

celsius 24, fahrenheit 75.2
asked for -500, got -273.15

Inside both sections, value is the value crossing the boundary: in set the value being assigned, which may be adjusted before it is stored; in get the result, which starts at the type's default and is returned at end get. return works too, and either section may be left out. get and set open a section only when they stand alone on their line, so get #1, , buffer in an ordinary getter is still the file statement.

A property may also take more than one index, be static (Grid.Created, belonging to the type), or live in a namespace with no object at all. These forms, and the auto-property and block above, are tiko's own: FreeBASIC has none of them (Appendix D).

Keeping fields private

Everything after private: in a type is reachable only from the type's own methods:

'' access.tiko -- private means private.
type Thermostat
    declare property Celsius( ) as float64
  private:
    c_ as float64
end type

property Thermostat.Celsius( ) as float64
    return c_
end property

dim as Thermostat t
t.c_ = 5                           '' refused: c_ is private

access.tiko(13,2): error TKO0625: 'c_' is not accessible here
     |
  13 | t.c_ = 5                           '' refused: c_ is private
     |  ^

That refusal is what made the setter's rule real. Without it, t.c_ = -500 would bypass the clamp and the type's guarantee would be a suggestion. public: reopens access, and protected: — private to everyone except types derived from this one — matters from Chapter 14.

Access applies to the calls the compiler makes for you as well as the ones you write:

'' privatector.tiko -- access applies to the calls you never write.
type Fortress
    x as int32
  private:
    declare constructor( )
    declare destructor( )
end type

constructor Fortress( )
end constructor

destructor Fortress( )
end destructor

dim as Fortress f                 '' needs both, and may call neither

privatector.tiko(15,17): error TKO0633: 'f' needs a constructor or destructor that is not accessible here
     |
  15 | dim as Fortress f                 '' needs both, and may call neither
     |                 ^
privatector.tiko(15,17): error TKO0633: 'f' needs a constructor or destructor that is not accessible here
     |
  15 | dim as Fortress f                 '' needs both, and may call neither
     |                 ^

A plain dim calls the constructor and, at the end of the scope, the destructor — so a type with both private cannot be declared from outside at all. The message appears once for each. Making a constructor private is how a type insists on being created through a function of its own choosing.

Members that belong to the type

'' statics.tiko -- members that belong to the type, and methods that
'' promise not to change the object.

type Ticket
    declare constructor( )
    declare static function Issued( ) as int32
    declare const function Describe( ) as ustring
    static issued_ as int32           '' one, shared by every Ticket
    number_ as int32                  '' one per Ticket
end type

dim as int32 Ticket.issued_ = 0      '' a static field is defined once

constructor Ticket( )
    issued_ += 1
    number_ = issued_
end constructor

'' A static method has no this: it is called on the type.
static function Ticket.Issued( ) as int32
    return issued_
end function

'' A const method may read the object but not change it.
const function Ticket.Describe( ) as ustring
    return "ticket #" & number_
end function

dim as Ticket a, b, c
print "issued so far:"; Ticket.Issued( )
print b.Describe( )
print c.Describe( )

issued so far: 3
ticket #2
ticket #3

A static field exists once, shared by every object of the type, and needs a definition outside the type — dim as int32 Ticket.issued_ = 0 — in exactly one place. A static method has no this and is called on the type itself, Ticket.Issued( ).

A const method promises not to change the object. It may read fields and call other const methods, and the compiler holds it to the promise:

'' constmethod.tiko -- a const method that tries to change the object.
type Tally
    declare const sub Bump( )
    n as int32
end type

const sub Tally.Bump( )
    n += 1                            '' refused: Bump promised not to
end sub

constmethod.tiko(8,5): error TKO0664: 'n' cannot be written from a const method
     |
   8 |     n += 1                            '' refused: Bump promised not to
     |     ^

Objects on the heap

'' heapobjects.tiko -- objects that do not live in a scope.

type Noisy
    declare constructor( )
    declare constructor( byval n as int32 )
    declare destructor( )
    n as int32
end type

constructor Noisy( )
    n = -1
    print "  default constructor"
end constructor

constructor Noisy( byval n as int32 )
    this.n = n
    print "  constructor"; n
end constructor

destructor Noisy( )
    print "  destructor"; n
end destructor

'' new runs the constructor; delete runs the destructor, then frees.
print "new and delete:"
dim as Noisy ptr one = new Noisy( 7 )
delete one

'' An array in a scope constructs every element, and destroys them
'' all when the scope ends.
print "an array in a scope:"
scope
    dim as Noisy pair( 1 to 2 )
    pair( 2 ).n = 20
end scope

'' Placement new builds an object in memory you already have, and you
'' must run the destructor yourself before freeing that memory.
print "placement new:"
dim as any ptr raw = allocate( sizeof( Noisy ) )
dim as Noisy ptr placed = new( raw ) Noisy( 42 )
print "  placed->n ="; placed->n
placed->destructor( )
deallocate( raw )
print "done"

new and delete:
  constructor 7
  destructor 7
an array in a scope:
  default constructor
  default constructor
  destructor 20
  destructor-1
placement new:
  constructor 42
  placed->n = 42
  destructor 42
done

new T( … ) allocates and runs the matching constructor; delete runs the destructor and then frees. An array of objects in a scope constructs every element, using the default constructor, and destroys them all — last first — when the scope ends.

Placement new — new( address ) T( … ) — runs a constructor in memory you already have, without allocating. It is for memory pools and for objects that must live at a particular address. Because nothing was allocated, delete is wrong here: you call the destructor yourself, explicitly, and free the memory the way you got it.

new T[ n ] and delete[] do the same for an array on the heap: a constructor for each element, then a destructor for each, last first.

'' deletearray.tiko -- delete[] on an array of objects.
''
'' new[] runs a constructor for each element. delete[] must run the
'' destructor for each, or everything the objects own is leaked.

type Noisy
    declare constructor( )
    declare destructor( )
    n as int32
end type

constructor Noisy( )
    n = -1
    print "  constructor"
end constructor

destructor Noisy( )
    print "  destructor"; n
end destructor

dim as Noisy ptr three = new Noisy[ 3 ]
three[1].n = 10
delete[] three
print "done"

  constructor
  constructor
  constructor
  destructor-1
  destructor 10
  destructor-1
done

delete[] knows how many elements there are because, for a type with a destructor, new[] recorded the count just before the first one. That is one more reason the families must not be mixed: delete or deallocate on such memory starts at the wrong address.

A builder that needs none of the three

'' stringbuilder.tiko -- building text without copying it every time.
''
'' Joining with &= in a loop copies everything built so far on every
'' pass. A builder keeps spare room and copies only when it runs out.

type StringBuilder
    declare sub Add( byref piece as string )
    declare property Length( ) as int32
    declare property Capacity( ) as int32
    declare function Text( ) as string
  private:
    buf_ as string                '' the room, mostly unused
    len_ as int32                 '' how much of it holds text
end type

sub StringBuilder.Add( byref piece as string )
    dim as int32 n = len( piece )
    if n = 0 then exit sub

    '' Out of room: at least double it, so growth stays rare.
    if len_ + n > len( buf_ ) then
        dim as int32 grow = iif( len( buf_ ) = 0, 16, len( buf_ ) )
        do while len_ + n > len( buf_ ) + grow : grow *= 2 : loop
        buf_ &= space( grow )
    end if

    mid( buf_, len_ + 1, n ) = piece  '' overwrite in place: no copy
    len_ += n
end sub

property StringBuilder.Length( ) as int32
    return len_
end property

property StringBuilder.Capacity( ) as int32
    return len( buf_ )
end property

function StringBuilder.Text( ) as string
    return left( buf_, len_ )
end function

dim as StringBuilder sb
for i as int32 = 1 to 12
    sb.Add( "item" & i & ";" )
next
print "length"; sb.Length; ", capacity"; sb.Capacity
print left( sb.Text( ), 40 ); "..."

'' Every field copies itself correctly, so the type needs no copy
'' constructor, no operator let and no destructor of its own.
dim as StringBuilder copy_ = sb
copy_.Add( "more" )
print "original"; sb.Length; ", copy"; copy_.Length

length 75, capacity 128
item1;item2;item3;item4;item5;item6;item...
original 75, copy 79

Joining text with &= in a loop copies everything built so far on every pass. StringBuilder keeps spare room in its buffer, writes each piece into it with Chapter 9's mid statement — which overwrites in place and copies nothing — and grows by doubling, so growth is rare.

The copy at the end is the point. StringBuilder owns a buffer, and yet it has no copy constructor, no operator let and no destructor — because its buffer is a string, which already knows how to copy and free itself. That is the rule of zero in practice: let each field manage its own resource, and the type composed of them needs no management at all.

What it costs

• A method call is a procedure call with the object passed by reference. No lookup, no indirection — until Chapter 14 adds virtual, which adds one.
• An object is the size of its fields. Methods, properties and static fields add nothing to each instance.
• A property is a call, but a short one that the optimiser usually inlines. Use them freely.
• A copy constructor runs on every byval pass and every return. For a type that owns memory, that is an allocation each time — pass it byref ... as const.
• Constructors and destructors run in scope order — free at run time, but only correct if they are written to be.

Summary

• Methods are declared in the type and defined as Type.Member; this is the object, and this. is optional.
• Constructors run on creation, destructors at the end of the scope, in reverse order of creation.
• Constructors overload; this.constructor( … ) delegates. A one-parameter constructor is an implicit conversion, with no way to switch it off.
• Rule of three: a type with a destructor needs a copy constructor and an operator let, and the let must survive self-assignment. Rule of zero: better still, build from fields that copy themselves.
• Properties are methods that read like fields; the setter enforces the type's rules. A property with only a setter cannot be read (TKO0934). An auto-property (get, private set, get, init) writes both accessors; a property block keeps them in one definition, with value crossing the boundary.
• private: fields are reachable only from the type's own methods; access is enforced on the constructor and destructor calls the compiler makes too.
• static fields are shared and defined once; static methods have no this.
• const methods may not change the object, and a const object may call only const methods (TKO0935).
• new/delete run constructor and destructor; placement new constructs in existing memory, and you call the destructor yourself. new[]/delete[] run them for every element.

Exercises

1. Write an IntStack that keeps its items in memory from reallocate, with a constructor, destructor, copy constructor and operator let. Show that a copied stack and an assigned stack are both independent of the original. Route both copy paths through one private helper. {solution="ex12-01"}
2. Write a Temperature that stores kelvin and exposes Celsius and Fahrenheit as read-write properties, with the absolute-zero check in exactly one place. {solution="ex12-02"}
3. Give Account a private constructor and a static function Create( … ) as Account that refuses a negative opening balance. What can callers no longer do, and why is that the point?
4. Remove the self-assignment check from copying.tiko's operator let and run c = c. What happens, and which line of let is responsible?
5. Temp converts silently from float64. Write a procedure taking a Temp, call it with a plain number, and decide whether that call should have compiled. How would you redesign Temp if not?

Operator Overloading

Chapter 10 left a gap: two Points could be copied but not compared, because the compiler would not guess what equality meant. Chapter 12 gave types methods, so a vector could have an Add method — and code that reads a.Add( b.Scale( 2 ) ) where a mathematician would write a + 2b.

An operator overload is a procedure that the compiler calls when it sees an operator applied to your type. It is ordinary code with an unusual name, and it costs exactly what the same code costs as a method. What it buys is notation: a type that behaves like a number can be written like one.

The chapter ends with the other side of that — what not to overload — because notation that lies about what it does is worse than a method name that tells the truth.

A value type that reads like a number

'' vec2.tiko -- a value type that reads like a number.

type Vec2
    x as float64
    y as float64
    declare operator +=( byref o as Vec2 )       '' members: they
    declare operator cast( ) as ustring          '' change or convert
end type                                         '' this object

'' Binary operators are global: neither operand is "this".
operator + ( byref a as Vec2, byref b as Vec2 ) as Vec2
    return type( a.x + b.x, a.y + b.y )
end operator

operator - ( byref a as Vec2, byref b as Vec2 ) as Vec2
    return type( a.x - b.x, a.y - b.y )
end operator

operator - ( byref a as Vec2 ) as Vec2               '' unary minus
    return type( -a.x, -a.y )
end operator

'' Scaling needs both orders, or 2 * v will not compile.
operator * ( byref a as Vec2, byval k as float64 ) as Vec2
    return type( a.x * k, a.y * k )
end operator

operator * ( byval k as float64, byref a as Vec2 ) as Vec2
    return a * k
end operator

operator = ( byref a as Vec2, byref b as Vec2 ) as boolean
    return a.x = b.x andalso a.y = b.y
end operator

operator <> ( byref a as Vec2, byref b as Vec2 ) as boolean
    return not ( a = b )
end operator

operator abs ( byref a as Vec2 ) as float64          '' the length
    return sqr( a.x * a.x + a.y * a.y )
end operator

operator Vec2.+=( byref o as Vec2 )
    x += o.x
    y += o.y
end operator

operator Vec2.cast( ) as ustring
    return "(" & x & ", " & y & ")"
end operator

dim as Vec2 a = ( 1, 2 ), b = ( 3, 4 )

print "a + b     = "; a + b
print "b - a     = "; b - a
print "-a        = "; -a
print "2 * a     = "; 2 * a
print "a + b * 2 = "; a + b * 2; "   (* still binds first)"
print "abs( b )  ="; abs( b )
print "a = b: "; a = b; ",  a <> b: "; a <> b

a += b
print "a += b  -> "; a

a + b     = (4, 6)
b - a     = (2, 2)
-a        = (-1, -2)
2 * a     = (2, 4)
a + b * 2 = (7, 10)   (* still binds first)
abs( b )  = 5
a = b: false,  a <> b: true
a += b  -> (4, 6)

An operator is defined like a function whose name is the operator: operator + ( a, b ) as Vec2. From then on, a + b on two Vec2s is a call to it.

A few things in that listing are deliberate:

• Scaling needs both orders. a * 2 and 2 * a are two different operators — the left operand's type differs — so the second is defined in terms of the first. Leave one out and the other order does not compile.
• Unary minus is its own operator, with one parameter.
• = and <> are separate. Defining one does not define the other, and they must agree, so <> is written as not ( a = b ).
• abs is overloadable, and for a vector the natural meaning is the length.
• Precedence does not change. a + b * 2 still multiplies first, because the compiler parses the expression before it decides which procedures to call.

operator cast( ) as ustring is what makes print a + b work: print takes the vector's conversion to text. It is also what makes "x" & v work, an assignment to a ustring, and the string functions — ucase( v ), left( v, 2 ) — treat the vector as its text.

Member or global

Some operators are declared inside the type and some outside it, and the compiler enforces which:

'' wrongplace.tiko -- a binary operator declared as a member.
type Vec2
    x as float64
    y as float64
    declare operator +( byref o as Vec2 ) as Vec2   '' refused
end type

wrongplace.tiko(5,13): error TKO0790: operator '+' cannot be a member of a type
     |
   5 |     declare operator +( byref o as Vec2 ) as Vec2   '' refused
     |             ^

The rule follows from what each operator acts on:

@ is the opposite case from +: as a global it is refused with TKO0697: operator '@' must be a member of a type.

A global operator is not a method, so it sees only the public parts of the type. That is usually right — an operator built from the type's public interface cannot break its invariants — and when it is not, the operator can call a method that does the work.

Conversions in both directions

'' fixedpoint.tiko -- exact decimals, stored as whole thousandths.
''
'' 0.1 + 0.2 is not 0.3 in floating point. Stored as an integer count
'' of thousandths, it is exactly 300.

type Fixed
    raw as int64                              '' thousandths
    declare operator let( byval v as float64 )
    declare operator cast( ) as float64
    declare operator cast( ) as string
end type

'' let from another type: assigning a float64 rounds it once, here.
operator Fixed.let( byval v as float64 )
    raw = cint( v * 1000 )
end operator

operator Fixed.cast( ) as float64
    return raw / 1000
end operator

operator Fixed.cast( ) as string
    dim as int64 whole = raw \ 1000
    dim as int64 part = abs( raw ) mod 1000
    dim as ustring sign = iif( raw < 0 andalso whole = 0, "-", "" )
    return sign & whole & "." & right( "00" & part, 3 )
end operator

operator + ( byref a as Fixed, byref b as Fixed ) as Fixed
    dim as Fixed r
    r.raw = a.raw + b.raw
    return r
end operator

operator * ( byref a as Fixed, byref b as Fixed ) as Fixed
    dim as Fixed r
    r.raw = ( a.raw * b.raw ) \ 1000
    return r
end operator

operator < ( byref a as Fixed, byref b as Fixed ) as boolean
    return a.raw < b.raw
end operator

operator = ( byref a as Fixed, byref b as Fixed ) as boolean
    return a.raw = b.raw
end operator

dim as Fixed p, q, sum, want
p = 0.1
q = 0.2
want = 0.3
sum = p + q
print "fixed:   0.1 + 0.2 = "; sum; "   equal to 0.3: "; sum = want

dim as float64 fp = 0.1 + 0.2
print "float64: 0.1 + 0.2 ="; fp; "   equal to 0.3: "; cbool( fp = 0.3 )

dim as Fixed price, qty
price = 19.99
qty = 3
print "19.99 * 3 = "; price * qty
print "as a float64:"; cast( float64, price * qty )
dim as Fixed twenty
twenty = 20
print "19.99 < 20: "; price < twenty

fixed:   0.1 + 0.2 = 0.300   equal to 0.3: true
float64: 0.1 + 0.2 = 0.3   equal to 0.3: false
19.99 * 3 = 59.970
as a float64: 59.97
19.99 < 20: true

Fixed stores an amount as a whole number of thousandths, so 0.1 + 0.2 is exactly 300 thousandths and compares equal to 0.3. The float64 line beside it compares unequal — although print shows it as 0.3, because print rounds to fifteen significant digits. The difference is real and invisible; that is the whole case for a fixed-point type.

Two conversions make it convenient:

• operator let( byval v as float64 ) lets a float64 be assigned to a Fixed. The rounding happens once, there, and nowhere else.
• operator cast( ) as float64 lets a Fixed be used where a float64 is wanted. cast( float64, price * qty ) calls it explicitly; passing a Fixed to a procedure that takes a float64 calls it implicitly.

A type may have several cast operators — Fixed has two — distinguished by the type they produce.

The built-in maths functions take it too. sqr, abs, int and the rest accept a type with a numeric cast, convert, and work on the number:

'' castmath.tiko -- a built-in maths function given a type with a cast.
''
'' Fixed converts itself to float64, so sqr( f ) is sqr( 2.5 ):
'' 1.58113883008419.

type Fixed
    raw as int64
    declare operator cast( ) as float64
end type

operator Fixed.cast( ) as float64
    return raw / 1000
end operator

dim as Fixed f
f.raw = 2500
dim as float64 r = sqr( f )
print "sqr( f ) ="; r

sqr( f ) = 1.58113883008419

A type with no numeric cast — and no operator sqr of its own — is refused: TKO0928.

Reaching into an object

'' indexing.tiko -- [] , let from another type, and smart handles.

type Grid
    declare constructor( byval w as int32, byval h as int32 )
    declare destructor( )
    declare operator []( byval i as int32 ) byref as int32
    declare operator let( byval fill as int32 )
    declare operator @( ) as int32 ptr
    w as int32
    h as int32
  private:
    cells as int32 ptr
end type

constructor Grid( byval w as int32, byval h as int32 )
    this.w = w
    this.h = h
    cells = callocate( w * h, sizeof( int32 ) )
end constructor

destructor Grid( )
    deallocate( cells )
end destructor

'' Returning byref is what lets g[4] appear on the LEFT of =.
operator Grid.[]( byval i as int32 ) byref as int32
    return cells[i]
end operator

'' let from an int32: "g = 7" fills every cell.
operator Grid.let( byval fill as int32 )
    for i as int32 = 0 to w * h - 1 : cells[i] = fill : next
end operator

'' @g gives the first cell, which is what a C function would want.
operator Grid.@( ) as int32 ptr
    return cells
end operator

dim as Grid g = Grid( 3, 2 )
g = 7
g[4] = 42
print "g[0], g[4], g[5] ="; g[0]; g[4]; g[5]
print "via @g:"; ( @g )[4]

'' A handle that forwards -> and * to the Grid it refers to. These
'' two are global operators, taking the handle as their operand.
type GridHandle
    target as Grid ptr
end type

operator -> ( byref hnd as GridHandle ) byref as Grid
    return *hnd.target
end operator

operator * ( byref hnd as GridHandle ) byref as Grid
    return *hnd.target
end operator

dim as GridHandle hd
hd.target = @g        '' careful: this is Grid's own @, so a cell!
print "hd.target is the cells, not the Grid: "; _
      cbool( hd.target = cast( Grid ptr, @g ) )
hd.target = varptr( g )      '' varptr is never overloaded
print "through ->: w ="; hd->w; ",  through *: h ="; ( *hd ).h

g[0], g[4], g[5] = 7 42 7
via @g: 42
hd.target is the cells, not the Grid: true
through ->: w = 3,  through *: h = 2

operator [] returning byref is what lets g[4] appear on the left of an assignment: the caller receives the element itself, not a copy of it. A [] that returns by value gives a read-only index.

operator let( byval fill as int32 ) is let from a different type: g = 7 fills every cell. let from the type's own type is the copy assignment Chapter 12 covered; let from anything else is a conversion into the object.

GridHandle forwards -> and * to the Grid it holds, which is the shape of every "smart" handle — a type that behaves like a pointer while doing something extra, such as counting references, which Chapter 21's SharedPtr does.

The middle of the listing is a warning about @. Because Grid overloads @ to return its cells, @g no longer means the address of g — and assigning it to a Grid ptr hands the handle the cells instead of the grid. The compiler noticed:

indexing.tiko(61,1): warning TKO0830: suspicious pointer assignment: a 'grid ptr' from a 'int32 ptr'

varptr( g ) is never overloaded, and always means the address of g, which is why the listing uses it for the real assignment.

A type that counts

'' rangecounter.tiko -- a type that a for loop can count with.
''
'' for e as T = first to last [step s] calls five operators on T:
'' for (once, at the start), next (should the loop go on?) and step
'' (advance) -- each in a form with and without the step value.

type Every
    declare constructor( byval startAt as int32, byval stepBy as int32 )
    declare operator for( )
    declare operator for( byref stp as Every )
    declare operator step( )
    declare operator step( byref stp as Every )
    declare operator next( byref last as Every ) as integer
    declare operator next( byref last as Every, _
                           byref stp as Every ) as integer
    declare operator cast( ) as string
    value as int32
    by_ as int32
end type

constructor Every( byval startAt as int32, byval stepBy as int32 )
    value = startAt
    by_ = stepBy
end constructor

operator Every.for( )                   '' nothing to prepare
end operator

operator Every.for( byref stp as Every )
end operator

operator Every.step( )                  '' no step clause: own stride
    value += by_
end operator

operator Every.step( byref stp as Every )   '' step clause given
    value += stp.value
end operator

operator Every.next( byref last as Every ) as integer
    return value <= last.value
end operator

operator Every.next( byref last as Every, _
                     byref stp as Every ) as integer
    return value <= last.value
end operator

operator Every.cast( ) as string
    return str( value )
end operator

print "every third from 1 to 12:";
for e as Every = Every( 1, 3 ) to Every( 12, 0 )
    print " "; e;
next
print

print "0 to 20, step 5:";
for e as Every = Every( 0, 1 ) to Every( 20, 0 ) step Every( 5, 0 )
    print " "; e;
next
print

every third from 1 to 12: 1 4 7 10
0 to 20, step 5: 0 5 10 15 20

for e as T = first to last step s works for any type that implements the counting protocol:

Each comes in two forms, with and without the step value, matching whether the for has a step clause. Nothing in the protocol assumes a direction or even a number — next decides when to stop — so a counter can walk dates, a range of keys or a countdown, as Exercise 2 does.

For walking the elements of a container rather than counting, Chapter 20's for each and its iterator protocol are the better tool.

Taking over allocation

'' allocators.tiko -- a type that controls where its objects live.

type Tracked
    declare operator new( byval size as uinteger ) as any ptr
    declare operator delete( byval p as any ptr )
    n as int32
end type

dim shared as int32 live      '' a count every Tracked shares

'' new and delete are static members: there is no object yet in new,
'' and none left in delete. They allocate; they do not construct.
operator Tracked.new( byval size as uinteger ) as any ptr
    live += 1
    print "  allocating "; size; " bytes"
    return allocate( size )
end operator

operator Tracked.delete( byval p as any ptr )
    live -= 1
    print "  releasing"
    deallocate( p )
end operator

dim as Tracked ptr a = new Tracked
dim as Tracked ptr b = new Tracked
print "live objects:"; live
delete a
delete b
print "live objects:"; live

  allocating 4 bytes
  allocating 4 bytes
live objects: 2
  releasing
  releasing
live objects: 0

operator new and operator delete replace the allocation step for one type. They do not construct or destroy — the constructor and destructor still run, before and after — they only decide where the bytes come from and go back to. That makes them the place for a memory pool, an allocation counter as here, or a check that every object was freed.

They are implicitly static: in new the object does not exist yet, and in delete it no longer does, so there is no this — and neither this nor a bare field may be used in either (TKO0937).

What not to overload

An operator is a promise about meaning that the reader will hold you to without reading the code. Break it and every expression using the type becomes a puzzle.

• Keep the meaning. + combines, - takes away, = is equality and nothing else. A + that appends to a log or a * that means "repeat" makes correct-looking code wrong.
• Keep the algebra where it applies. If a + b exists, a - b, -a and a += b should exist and agree with it. If = exists, <> must be its negation.
• Keep the cost obvious. a + b reads as cheap. If it allocates, copies a megabyte or touches a disk, give it a name that says so.
• Do not overload @ unless the type genuinely is a handle to something else — the listing above shows what it does to every @ in the program.
• Do not overload and, or and not for logic. The built-in andalso and orelse short-circuit; an overloaded and cannot, and a reader cannot tell which they are looking at.
• When the operation has no single obvious meaning, use a method. Dot( a, b ) and Cross( a, b ) are better than picking one of them to be *.

What it costs

• An operator is a procedure call with the same cost as the method it replaces, and short ones are inlined. Notation is free.
• Returning a type by value copies it. For a small value type like Vec2 that is the right design; for a type that owns memory, each a + b constructs a temporary and runs its copy constructor.
• Conversions run silently. An implicit cast or let in a loop is a call per iteration that nothing in the source shows.

Summary

• operator + and its kind are procedures the compiler calls for your type. Precedence and associativity never change.
• Binary and unary arithmetic, comparisons, -> and * are global; compound assignment, let, cast, [], @, the for protocol and new/delete are members — the compiler enforces it (TKO0790, TKO0697).
• Define both orders of a mixed-type operator, and define <> alongside =.
• cast( ) as ustring makes a type printable, and usable with & and the string functions.
• let from another type and cast to another type are conversions in and out; the built-in maths functions use a numeric cast.
• [] returning byref gives an assignable index.
• Overloading @ changes the meaning of every @; varptr is never overloaded.
• for, step and next make a type a for counter, in either direction.
• operator new/delete choose where memory comes from; construction and destruction still happen around them.
• Overload only where the meaning is the obvious one, and keep the algebra and the cost honest.

Exercises

1. Write Money, stored in whole cents, with +, -, multiplication by a count, <, = and a cast to text like $15.30. Add a thousand ten-cent amounts and check the total is exactly $100. Why is there no Money * Money? {solution="ex13-01"}
2. Write a Countdown type that a for loop counts down with, printing liftoff at zero. What happens when the start is below the end? {solution="ex13-02"}
3. Add operator * to Vec2 as a dot product, then try to add a second * for the cross product. What stops you, and what does that tell you about overloading * here at all?
4. Give Fixed an operator /. What should 1 / 3 give, and where exactly does the rounding happen?
5. Remove operator <> from vec2.tiko and keep the a <> b line. What does the compiler say, and would you rather it had derived <> from = for you?

Inheritance and Polymorphism

A drawing program has circles, rectangles and squares, and somewhere in it a loop that has to ask each one for its area. Written with the tools of Chapter 12, every shape is its own type, and the loop needs a select case on some "kind" field to know which code to run. The next shape means editing that loop, and every other loop like it.

Inheritance lets one type start as another and add to it. Polymorphism lets a single call reach the right code for whatever object is really there, so the loop asks "what is your area?" and never needs to know what it is talking to.

tiko gives you two ways to build them, and this chapter is in two parts. Part 1 is the classic model, the one C++ and FreeBASIC share: a base type with virtual methods, extended by every shape, and the three places it goes wrong: an object deleted through the wrong pointer, an object passed by value, and a relationship that was never really "is a". Part 2 is interfaces: a contract that any type can take on, as many as it needs, without spending its one base on it. The two mix freely, and the chapter ends with how to choose.

Part 1: The classic model

A derived type starts as its base and adds to it, and a method the base marks virtual can be replaced by each derived type. Everything in this part works the same whether the type is declared with type … extends or, as Part 2 shows, with class.

Extending a type

'' extends.tiko -- a derived type starts as its base, and adds to it.

type Vehicle
    declare constructor( byval wheels as int32 = 4 )
    declare function Describe( ) as ustring
    wheels as int32
    speed as int32
end type

constructor Vehicle( byval wheels as int32 )
    this.wheels = wheels
end constructor

function Vehicle.Describe( ) as ustring
    return wheels & " wheels at " & speed & " km/h"
end function

type Bicycle extends Vehicle
    declare constructor( byval gears as int32 )
    declare function Describe( ) as ustring
    gears as int32
end type

constructor Bicycle( byval gears as int32 )
    base( 2 )                     '' build the Vehicle part first
    this.gears = gears
end constructor

function Bicycle.Describe( ) as ustring
    return base.Describe( ) & ", " & gears & " gears"
end function

dim as Bicycle bike = Bicycle( 21 )
bike.speed = 25                   '' a field it inherited
print bike.Describe( )

dim as Vehicle car
print car.Describe( )

print "sizes:"; sizeof( Vehicle ); sizeof( Bicycle )
print "offsets:"; offsetof( Bicycle, wheels ); _
    offsetof( Bicycle, speed ); offsetof( Bicycle, gears )

2 wheels at 25 km/h, 21 gears
4 wheels at 0 km/h
sizes: 8 12
offsets: 0 4 8

type Bicycle extends Vehicle makes a Bicycle a Vehicle with more fields. It has wheels and speed without declaring them, and the offsets show where they live: the base's fields come first, exactly as they are laid out in a Vehicle, and gears follows them. That layout is the point. A Bicycle starts with a complete Vehicle, so anything that works on a Vehicle can be handed one.

base( 2 ) runs the base's constructor first. The Vehicle part of a Bicycle is built before the Bicycle constructor's own body runs, and base( … ) chooses which Vehicle constructor builds it and with what. Written without a base( … ) call, the base's default constructor runs instead.

base.Describe( ) calls the base's version of a method from inside the derived one. Bicycle's Describe has the same name as Vehicle's and hides it: on a Bicycle, Describe means Bicycle.Describe. Reusing the base's text through base. means the two descriptions cannot drift apart.

A base without a default constructor

Vehicle's constructor takes wheels = 4 as a default, so it can be called with nothing at all. That default is load-bearing:

'' nodefault.tiko -- a base with no default constructor leaves the
'' derived type with no way to copy itself.

type Vehicle
    declare constructor( byval wheels as int32 )
    wheels as int32
end type

constructor Vehicle( byval wheels as int32 )
    this.wheels = wheels
end constructor

type Bicycle extends Vehicle
    declare constructor( )
    gears as int32
end type

constructor Bicycle( )
    base( 2 )
end constructor

dim as Bicycle bike
print bike.wheels

nodefault.tiko(13,1): error TKO0926: 'bicycle' needs a copy constructor: its base has no constructor that takes no argument, so none can be made for it
     |
  13 | type Bicycle extends Vehicle
     | ^

A derived type gets a copy constructor for free, and that free one works by building a fresh base part with the base's default constructor, then copying every field across. With no default constructor in the base, there is nothing to build the fresh part with, so the derived type must write its own copy constructor. fbc refuses the same program with its error 188.

Give the base a constructor that takes no arguments, or default its parameters as extends.tiko does, and every type derived from it can be copied without writing anything.

base( … ) passes whatever the base's constructors take, chosen the way any overloaded call is. Here it hands a label up, and the base's default value for it, "unnamed", is what keeps Sensor copyable:

'' basestring.tiko -- passing text up to the base's constructor.

type Labelled
    declare constructor( byref label as ustring = "unnamed" )
    label as ustring
end type

constructor Labelled( byref label as ustring )
    this.label = label
end constructor

type Sensor extends Labelled
    declare constructor( byref label as ustring, _
                         byval reading as float64 )
    reading as float64
end type

constructor Sensor( byref label as ustring, byval reading as float64 )
    base( label )
    this.reading = reading
end constructor

dim as Sensor s = Sensor( "boiler", 81.5 )
print s.label; " reads"; s.reading

boiler reads 81.5

Calling through the base

A Bicycle can be handed to anything that takes a Vehicle. What happens to its methods when it is?

'' hiding.tiko -- without virtual, the call is chosen by the type the
'' compiler sees, not by the object that is really there.

type Vehicle
    declare function Kind( ) as ustring
    wheels as int32
end type

function Vehicle.Kind( ) as ustring
    return "vehicle"
end function

type Bicycle extends Vehicle
    declare function Kind( ) as ustring
end type

function Bicycle.Kind( ) as ustring
    return "bicycle"
end function

sub Show( byref v as Vehicle )
    print "  Show sees a "; v.Kind( )
end sub

dim as Bicycle bike
print "directly: "; bike.Kind( )
Show( bike )

directly: bicycle
  Show sees a vehicle

Show declares its parameter as a Vehicle, so inside Show the compiler knows only that it has a Vehicle, and v.Kind( ) is compiled as a call to Vehicle.Kind. Which object arrives at run time does not enter into it. Hiding a method changes what the name means on the derived type; it changes nothing for code that sees the object as its base.

That is the right behaviour for most methods, and it costs nothing. But it is exactly wrong for the drawing program's loop, which sees every shape as "a shape" and needs each one's own answer.

Virtual methods

'' shapes.tiko -- one loop over different shapes, each one answering
'' for itself.

type Shape extends Object
    declare abstract function Area( ) as float64
    declare virtual function Kind( ) as ustring
    declare function Describe( ) as ustring
end type

function Shape.Kind( ) as ustring
    return "shape"
end function

'' Not virtual, yet the calls inside it are.
function Shape.Describe( ) as ustring
    return Kind( ) & ", area " & Area( )
end function

type Circle extends Shape
    declare constructor( byval r as float64 )
    declare function Area( ) as float64 override
    declare function Kind( ) as ustring override
    r as float64
end type

constructor Circle( byval r as float64 )
    this.r = r
end constructor

function Circle.Area( ) as float64
    return 3.14159 * r * r
end function

function Circle.Kind( ) as ustring
    return "circle"
end function

type Rect extends Shape
    declare constructor( byval w as float64 = 1, _
                         byval h as float64 = 1 )
    declare function Area( ) as float64 override
    '' virtual AND override: a type derived from Rect may override
    '' it again.
    declare virtual function Kind( ) as ustring override
    w as float64
    h as float64
end type

constructor Rect( byval w as float64, byval h as float64 )
    this.w = w : this.h = h
end constructor

function Rect.Area( ) as float64
    return w * h
end function

function Rect.Kind( ) as ustring
    return "rectangle"
end function

type Square extends Rect
    declare constructor( byval side as float64 )
    declare function Kind( ) as ustring override
end type

constructor Square( byval side as float64 )
    base( side, side )
end constructor

function Square.Kind( ) as ustring
    return "square"
end function

dim as Circle c = Circle( 1 )
dim as Rect r = Rect( 2, 3 )
dim as Square s = Square( 2 )
dim as Shape ptr all( 0 to 2 ) = { @c, @r, @s }

for i as int32 = lbound( all ) to ubound( all )
    print all( i )->Describe( )
next

circle, area 3.14159
rectangle, area 6
square, area 4

Four words do the work.

• extends Object. Object is the built-in root type. Extending it gives every object of the type one hidden field, a pointer to the type's table of virtual methods, and that pointer is how a call finds the right code at run time.
• virtual marks a method that derived types may replace. A call to it looks the method up in the object's table, so it reaches the version belonging to the object that is really there.
• override marks a method that replaces a virtual one. The compiler checks that there is something to replace: override on a method the base did not make virtual is TKO0621.
• abstract declares a virtual method with no body at all. Shape cannot say what its area is, so it says only that every shape has one.

Shape.Describe is not virtual, yet its output is different for every shape, because the two calls inside it are. That is a common and useful shape: the base writes the procedure once, and the derived types supply the parts that differ.

An override is not itself virtual. Circle.Kind replaces Shape.Kind, but a type derived from Circle could not override it again. Rect declares its Kind as virtual … override, which both replaces Shape's and opens it to Rect's own descendants. That is what lets Square override it in turn. Square keeps Rect's Area, because a square's area is a rectangle's.

Leave virtual off, and the next level down cannot override:

'' notvirtual.tiko -- an override is not itself virtual. A type
'' derived from Dog that writes Speak again hides it; it does not
'' override it.

type Animal extends Object
    declare virtual function Speak( ) as ustring
end type

function Animal.Speak( ) as ustring
    return "..."
end function

type Dog extends Animal
    declare function Speak( ) as ustring override
end type

function Dog.Speak( ) as ustring
    return "woof"
end function

type Puppy extends Dog
    declare function Speak( ) as ustring     '' hides Dog's
end type

function Puppy.Speak( ) as ustring
    return "yip"
end function

sub Listen( byref a as Animal )
    print "  through Animal: "; a.Speak( )
end sub

dim as Puppy pup
print "directly: "; pup.Speak( )
Listen( pup )

directly: yip
  through Animal: woof

Dog.Speak is an override but not virtual, so Puppy's Speak hides it without replacing it. Called on a Puppy directly it answers yip; called through an Animal it reaches Dog's. Writing override on Puppy.Speak would not help: there is no open slot for it to fill, and the compiler says so (TKO0621). The fix is one word in Dog, virtual … override, exactly as Rect does in shapes.tiko.

C++ programmers expect the opposite, because there an override stays virtual all the way down. tiko follows fbc, where it does not, so code moves between the two compilers unchanged.

Abstract types

'' abstract.tiko -- a type with an unwritten method cannot be created.

type Shape extends Object
    declare abstract function Area( ) as float64
end type

dim as Shape s
print s.Area( )

abstract.tiko(7,14): error TKO0620: 's' has an abstract type, which cannot be instantiated
     |
   7 | dim as Shape s
     |              ^

A type with an abstract method is incomplete on purpose, so it cannot be created: there would be nothing to run when someone asked for its area. A derived type that fills in every abstract method can be created, and one that leaves any of them unwritten is abstract too, with the same error. What can exist is a Shape ptr or a byref Shape parameter, pointing at a complete type.

Seeing the table

tiko --dump-vtable prints each type's table of virtual methods. For shapes.tiko:

object vtable 0 slot(s)
shape vtable 2 slot(s)
  slot 0 <abstract>
  slot 1 _ZN5SHAPE4KINDEv
circle vtable 2 slot(s)
  slot 0 _ZN6CIRCLE4AREAEv
  slot 1 _ZN6CIRCLE4KINDEv
rect vtable 2 slot(s)
  slot 0 _ZN4RECT4AREAEv
  slot 1 _ZN4RECT4KINDEv
square vtable 2 slot(s)
  slot 0 _ZN4RECT4AREAEv
  slot 1 _ZN6SQUARE4KINDEv
vtables: 5

Every type in the family has the same two slots in the same order, Area then Kind. A derived type copies its base's table and replaces the entries it overrides, so a call through a Shape always looks in slot 0 for Area, whichever table it finds. Shape's slot 0 is <abstract>, which is why a Shape cannot exist; Square's slot 0 still names Rect.Area, which it inherited. The names are mangled the way C++ mangles them: _ZN6SQUARE4KINDEv is SQUARE::KIND().

The dump is the place to look when a call reaches the wrong method. It shows what the compiler built, and comparing it with what you meant is usually enough to find the mistake.

Virtual destructors

An object created with new and deleted through a pointer to its base is destroyed by whichever destructor that pointer's type names:

'' vdtor.tiko -- delete through a base pointer runs the destructor the
'' pointer's type names, unless that destructor is virtual.

type Plain
    declare destructor( )
    id as int32
end type

destructor Plain( )
    print "  ~Plain"
end destructor

type PlainFile extends Plain
    declare destructor( )
    handle as int32
end type

destructor PlainFile( )
    print "  ~PlainFile: the file is closed"
end destructor

type Careful extends Object
    declare virtual destructor( )
end type

destructor Careful( )
    print "  ~Careful"
end destructor

type CarefulFile extends Careful
    declare destructor( )
    handle as int32
end type

destructor CarefulFile( )
    print "  ~CarefulFile: the file is closed"
end destructor

print "plain destructor:"
dim as Plain ptr p = new PlainFile
delete p

print "virtual destructor:"
dim as Careful ptr q = new CarefulFile
delete q

plain destructor:
  ~Plain
virtual destructor:
  ~CarefulFile: the file is closed
  ~Careful

The rule from the previous section applies to destructors as well, and this is where it costs most. A destructor is virtual only if it says so, so a middle level that declares a plain one closes the chain below it:

'' vdtormiddle.tiko -- 'virtual' is not inherited by a destructor that
'' does not say it. Each level that will be deleted through a base
'' pointer declares its destructor virtual.

type Resource extends Object
    declare virtual destructor( )
end type
destructor Resource( )
    print "  ~Resource"
end destructor

type File extends Resource
    declare destructor( )              '' not virtual
    handle as int32
end type
destructor File( )
    print "  ~File"
end destructor

type LogFile extends File
    declare destructor( )
    lines as int32
end type
destructor LogFile( )
    print "  ~LogFile: flushed"
end destructor

type SafeFile extends Resource
    declare virtual destructor( )      '' virtual again
    handle as int32
end type
destructor SafeFile( )
    print "  ~SafeFile"
end destructor

type SafeLog extends SafeFile
    declare destructor( )
    lines as int32
end type
destructor SafeLog( )
    print "  ~SafeLog: flushed"
end destructor

print "middle destructor not virtual:"
dim as Resource ptr r = new LogFile
delete r

print "middle destructor virtual:"
r = new SafeLog
delete r

middle destructor not virtual:
  ~File
  ~Resource
middle destructor virtual:
  ~SafeLog: flushed
  ~SafeFile
  ~Resource

LogFile's destructor never ran, although Resource's is virtual and LogFile declares one of its own. File declared a destructor without virtual, and from there down nothing overrides. SafeFile says virtual again, and its descendant is destroyed properly. The same happens when the middle type declares no destructor at all: the one the compiler writes for it is not virtual. Every level that may be deleted through a base pointer declares virtual destructor.

Asking what an object really is

Sometimes the code holding an Animal needs something only a Dog has:

'' isdown.tiko -- asking what an object really is, and treating it as
'' that type once the answer is yes.

type Animal extends Object
    declare virtual function Speak( ) as ustring
end type

function Animal.Speak( ) as ustring
    return "..."
end function

type Dog extends Animal
    declare virtual function Speak( ) as ustring override
    tricks as int32
end type

function Dog.Speak( ) as ustring
    return "woof"
end function

type Puppy extends Dog
    declare function Speak( ) as ustring override
end type

function Puppy.Speak( ) as ustring
    return "yip"
end function

sub Inspect( byref a as Animal )
    print a.Speak( );
    if a is Dog then
        '' a really is a Dog (or something derived from one)
        dim as Dog ptr d = cast( Dog ptr, @a )
        print ", knows"; d->tricks; " tricks";
    end if
    if a is Puppy then print ", and is a puppy";
    print
end sub

dim as Animal cat
dim as Dog rex
rex.tricks = 3
dim as Puppy pup
pup.tricks = 1

Inspect( cat )
Inspect( rex )
Inspect( pup )

...
woof, knows 3 tricks
yip, knows 1 tricks, and is a puppy

a is Dog is true when the object is a Dog or anything derived from one, which is why the puppy passes both tests. Once it is true, cast( Dog ptr, @a ) treats the same object as a Dog. The cast itself checks nothing, so always guard it with is.

is needs the type to extend Object, because the answer is read from the hidden table pointer, and it only accepts a type derived from the variable's own type: asking whether an Animal is an Animal is refused (TKO0798), and asking it of a type with no table is TKO0799. fbc refuses both too.

Use is sparingly. A chain of if a is Dog … elseif a is Cat … is the select case this chapter set out to remove, wearing different clothes. If the question is "what should this animal do", the answer is a virtual method.

Passing by value

'' slicing.tiko -- a derived object passed by value arrives as a copy of
'' its base part, and only that.

type Animal extends Object
    declare virtual function Speak( ) as ustring
    legs as int32
end type

function Animal.Speak( ) as ustring
    return "..."
end function

type Dog extends Animal
    declare function Speak( ) as ustring override
    tricks as int32
end type

function Dog.Speak( ) as ustring
    return "woof"
end function

sub ByReference( byref a as Animal )
    print "byref:  "; a.Speak( ); ", a Dog? "; ( a is Dog )
end sub

sub ByValue( byval a as Animal )
    print "byval:  "; a.Speak( ); ", a Dog? "; ( a is Dog )
end sub

dim as Dog rex
rex.tricks = 3
ByReference( rex )
ByValue( rex )

dim as Animal copy = rex
print "copied: "; copy.Speak( )

'' The hidden field: one pointer, to the type's table of methods.
print "sizes:"; sizeof( Object ); sizeof( Animal ); sizeof( Dog )

byref:  woof, a Dog? -1
byval:  ..., a Dog?  0
copied: ...
sizes: 8 16 24

The last line of the output is the hidden field from the previous section made visible. Object is eight bytes, the table pointer and nothing else. Animal adds legs and rounds up to 16, and Dog adds tricks and rounds to 24.

Composition, and abstract types as contracts

Inheritance says "is a". A Bicycle is a Vehicle, and a Square is a Rect. When the relationship is "has a" or "uses a", inheritance is the wrong tool, even when it would save typing: a Circle that extends a Point to borrow its x and y can be passed to every procedure that expects a point, and none of them meant a circle. Give the Circle a Point field instead.

'' pluginhost.tiko -- an abstract type as the contract, and a host that
'' is built from parts rather than derived from one.

'' The contract: anything that can transform a line of text.
type Plugin extends Object
    declare abstract function Name( ) as ustring
    declare abstract function Apply( byref text as ustring ) as ustring
    declare virtual destructor( )
end type

destructor Plugin( )
end destructor

type Shout extends Plugin
    declare function Name( ) as ustring override
    declare function Apply( byref text as ustring ) as ustring override
end type

function Shout.Name( ) as ustring
    return "shout"
end function

function Shout.Apply( byref text as ustring ) as ustring
    return ucase( text ) & "!"
end function

type Reverse extends Plugin
    declare function Name( ) as ustring override
    declare function Apply( byref text as ustring ) as ustring override
end type

function Reverse.Name( ) as ustring
    return "reverse"
end function

function Reverse.Apply( byref text as ustring ) as ustring
    dim as ustring result
    for i as int32 = len( text ) to 1 step -1
        result &= mid( text, i, 1 )
    next
    return result
end function

'' The host HAS plugins; it is not one. It owns them, so it deletes
'' them -- through Plugin ptr, which is why the destructor is virtual.
type Host
    declare destructor( )
    declare sub Add( byval p as Plugin ptr )
    declare function Run( byref text as ustring ) as ustring
    plugins( any ) as Plugin ptr
end type

destructor Host( )
    for i as int32 = lbound( plugins ) to ubound( plugins )
        delete plugins( i )
    next
end destructor

sub Host.Add( byval p as Plugin ptr )
    redim preserve plugins( 0 to ubound( plugins ) + 1 )
    plugins( ubound( plugins ) ) = p
end sub

function Host.Run( byref text as ustring ) as ustring
    dim as ustring t = text
    for i as int32 = lbound( plugins ) to ubound( plugins )
        t = plugins( i )->Apply( t )
        print "  after "; plugins( i )->Name( ); ": "; t
    next
    return t
end function

scope
    dim as Host h
    h.Add( new Reverse )
    h.Add( new Shout )
    print h.Run( "plugins" )
end scope

  after reverse: snigulp
  after shout: SNIGULP!
SNIGULP!

Plugin is all contract: two abstract methods and a virtual destructor, with no data. Any type that extends it and fills in Name and Apply is a plugin, and Host can run it without knowing what it is. Written this way, the contract is an abstract type.

Host is the other half. It is not a plugin and does not extend one. It has plugins: it owns them, runs them in order, and deletes them in its destructor. The deletes go through Plugin ptr, which is why Plugin's destructor is virtual. Adding a third plugin means writing one new type; Host does not change.

A type extends exactly one base, and that is where this model runs out. A Plugin that must also be something else, a second contract on the same type, a small value that should follow the contract without carrying a table pointer: none of them fits in one chain. Part 2 is the answer.

Part 2: Interfaces

Part 1's contracts are base types, and a type has one base. A Drawing that is both a shape and a document, a Point that should be printable without carrying a table pointer, a library type that already extends something else: none of them fits. An interface is a contract that is not a base. Any type can take on as many as it needs, and keep its own base if it has one.

A contract any type can take on

'' ifaceshapes.tiko -- the shapes loop again, with an interface as the
'' contract and classes that implement it.

interface IShape
    function Area( ) as float64
    function Kind( ) as ustring
end interface

class Circle
    implements IShape
    declare constructor( byval r as float64 )
    declare function Area( ) as float64 implements IShape.Area
    declare function Kind( ) as ustring implements IShape.Kind
    r as float64
end class

constructor Circle( byval r as float64 )
    this.r = r
end constructor

function Circle.Area( ) as float64
    return 3.14159 * r * r
end function

function Circle.Kind( ) as ustring
    return "circle"
end function

class Rect
    implements IShape
    declare constructor( byval w as float64, byval h as float64 )
    declare function Area( ) as float64 implements IShape.Area
    declare function Kind( ) as ustring implements IShape.Kind
    w as float64
    h as float64
end class

constructor Rect( byval w as float64, byval h as float64 )
    this.w = w : this.h = h
end constructor

function Rect.Area( ) as float64
    return w * h
end function

function Rect.Kind( ) as ustring
    return "rectangle"
end function

'' Written once, against the contract only.
sub Describe( byval s as IShape )
    print s.Kind( ); ", area"; s.Area( )
end sub

dim as Circle c = Circle( 1 )
dim as Rect r = Rect( 2, 3 )
dim as IShape shapes( 0 to 1 )
shapes( 0 ) = c          '' an object converts to an interface it
shapes( 1 ) = @r         '' implements, and so does a pointer to one
for i as int32 = 0 to 1
    Describe( shapes( i ) )
next
print "an IShape value is"; sizeof( IShape ); " bytes"

circle, area 3.14159
rectangle, area 6
an IShape value is 16 bytes

interface IShape lists what a shape can do and nothing else: no fields, no code, no constructor. Every member is public and abstract, so there is nothing to write on one but its signature. An interface is never created by itself.

class Circle is a type that always extends Object. It is the same as type Circle extends Object, written so that the object model is the default. Its body starts with the contracts it takes on, implements IShape, and then each method says which member of the contract it provides: declare function Area( ) as float64 implements IShape.Area. The mapping is explicit. A public method called Area does not become IShape.Area until it says so, and a member the class forgets is an error that names it:

'' missing.tiko -- a member of the contract with no method mapped
'' to it.

interface IShape
    function Area( ) as float64
    function Kind( ) as ustring
end interface

class Circle
    implements IShape
    declare function Area( ) as float64 implements IShape.Area
    r as float64
end class

function Circle.Area( ) as float64
    return 3.14159 * r * r
end function

missing.tiko(10,16): error TKO1000: 'circle' does not implement 'ishape.kind'; add 'implements ishape.kind' to one of its methods
     |
  10 |     implements IShape
     |                ^
missing.tiko(6,5): note: 'kind' is required by interface 'ishape', declared here
     |
   6 |     function Kind( ) as ustring
     |     ^

An interface value is a reference. dim as IShape shapes( 0 to 1 ) holds two of them, and each is the address of an object together with that object's table for IShape: sixteen bytes, as the last line says. A Circle converts to IShape implicitly, and so does a pointer to one. Members are called with ., never ->, because the value is not a pointer you can follow. Describe is written against IShape alone, and runs unchanged for every type that implements it.

Two contracts at once

'' twocontracts.tiko -- one type, two contracts, and a member name the
'' two contracts both use for different things.

interface IShape
    function Area( ) as float64
    function Name( ) as ustring           '' what the shape is called
end interface

interface IDocument
    function Name( ) as ustring           '' the file it is saved in
    sub Save( )
    sub Close( )
end interface

interface IResource
    sub Close( )
end interface

class Drawing
    implements IShape, IDocument, IResource
    declare function Area( ) as float64 implements IShape.Area
    declare function Title( ) as ustring implements IShape.Name
    declare function FileName( ) as ustring implements IDocument.Name
    declare sub Close( ) implements IDocument.Close, IResource.Close
    w as float64 = 4
    h as float64 = 2
private:
    declare sub WriteOut( ) implements IDocument.Save
end class

function Drawing.Area( ) as float64
    return w * h
end function

function Drawing.Title( ) as ustring
    return "floor plan"
end function

function Drawing.FileName( ) as ustring
    return "plan.txt"
end function

sub Drawing.Close( )
    print "closed"
end sub

sub Drawing.WriteOut( )
    print "saving "; FileName( )
end sub

dim as Drawing d
dim as IShape s = d
dim as IDocument doc = d
dim as IResource res = d
print s.Name( ); " / "; doc.Name( ); ", area"; s.Area( )
doc.Save( )                   '' reaches the private WriteOut
res.Close( )

floor plan / plan.txt, area 8
saving plan.txt
closed

Drawing takes on three interfaces on one implements line. Part 1 could not express any of this:

• Two contracts both have a Name, and they mean different things. The shape's name is its title, the document's is a file name. They are different members, so Drawing maps each to its own method: Title implements IShape.Name, FileName implements IDocument.Name. A method's name does not have to match the member's.
• One method can provide several members. Close implements IDocument.Close and IResource.Close at once, because closing the drawing means the same thing to both.
• A private method can implement a member. WriteOut is under private:, so code outside Drawing cannot call it directly (TKO0625), yet doc.Save( ) reaches it. Saving is something you do to a document, not to a drawing that happens to be one.

Small values

A type that implements an interface does not need a table pointer of its own. The table travels in the interface value instead, so a plain type can implement one:

'' ifacevalue.tiko -- a plain type implements an interface and stays
'' the size of its fields.

interface IShow
    function Show( ) as ustring
end interface

type Point
    implements IShow
    x as int32
    y as int32
    declare function Show( ) as ustring implements IShow.Show
end type

function Point.Show( ) as ustring
    return "(" & x & ", " & y & ")"
end function

'' The same contract written as an abstract base type.
type Showable extends Object
    declare abstract function Show( ) as ustring
end type

type OldPoint extends Showable
    x as int32
    y as int32
    declare function Show( ) as ustring override
end type

function OldPoint.Show( ) as ustring
    return "(" & x & ", " & y & ")"
end function

dim as Point p = ( 3, 4 )
dim as IShow v = p
print v.Show( )
print "Point:"; sizeof( Point ); " bytes"
print "OldPoint:"; sizeof( OldPoint ); " bytes"

(3, 4)
Point: 8 bytes
OldPoint: 16 bytes

Point is eight bytes, its two fields. OldPoint follows the same contract written as an abstract base type, and every one of them pays eight bytes more for its table pointer. An array of a million points is 8 MB one way and 16 MB the other. The interface's sixteen bytes are paid only where an interface value is actually held.

Asking an interface what it holds

'' ifacequery.tiko -- asking what is behind an interface value.

interface IShow
    function Show( ) as ustring
end interface

interface ISize
    function Size( ) as int32
end interface

class Note
    implements IShow, ISize
    declare function Show( ) as ustring implements IShow.Show
    declare function Size( ) as int32 implements ISize.Size
    text as ustring = "buy milk"
end class

function Note.Show( ) as ustring
    return text
end function

function Note.Size( ) as int32
    return len( text )
end function

class Label
    implements IShow
    declare function Show( ) as ustring implements IShow.Show
end class

function Label.Show( ) as ustring
    return "label"
end function

sub Inspect( byval v as IShow )
    print v.Show( ); ":"
    '' A cross cast: does the same object ALSO implement ISize?
    dim as ISize sz = trycast( ISize, v )
    if sz <> nullptr then print "  size"; sz.Size( )
    '' A downcast: is it a Note?
    if v is Note then
        dim as Note ptr np = cast( Note ptr, v )
        print "  a note saying "; np->text
    end if
    print "  is ISize: "; cbool( v is ISize )
end sub

dim as Note n
dim as Label l
Inspect( n )
Inspect( l )

buy milk:
  size 8
  a note saying buy milk
  is ISize: true
label:
  is ISize: false

is works with an interface on either side. v is Note asks whether the object behind v is a Note, or derives from one; v is ISize asks whether it implements ISize. Neither needs the object's type to be related to IShow.

trycast answers the same questions with a value. trycast( ISize, v ) is a cross cast: the same object, seen through another interface it implements, or an empty interface value, equal to nullptr, when it implements no such thing. trycast( Note ptr, v ) is the checked downcast. cast( Note ptr, v ) is the unchecked one, which is why the listing asks v is Note first: a wrong cast is undefined behaviour, exactly as it is in Part 1.

Who owns the object

An interface value refers to an object. It does not own it, copy it or keep it alive. The compiler refuses the one case it can see, a temporary that would be gone by the next statement:

'' temporary.tiko -- an interface value refers to an object, and a
'' temporary would be gone before the reference was used.

interface IShow
    function Show( ) as ustring
end interface

type Tag
    implements IShow
    text as ustring
    declare function Show( ) as ustring implements IShow.Show
end type

function Tag.Show( ) as ustring
    return text
end function

function MakeTag( ) as Tag
    dim as Tag t
    t.text = "new"
    return t
end function

dim as IShow v = MakeTag( )
print v.Show( )

temporary.tiko(24,14): error TKO1012: an interface reference cannot refer to a temporary; store the object in a variable first
     |
  24 | dim as IShow v = MakeTag( )
     |              ^

delete through an interface is refused too (TKO1013): the interface does not know how the object was made, or whether it was made with new at all.

Classes and interfaces together

Interfaces do not replace class inheritance. They sit beside it, and a call through an interface honours every override below the class that implements it:

'' ifaceclass.tiko -- a class hierarchy behind an interface: the call
'' through the interface reaches the most derived override.

interface ISpeaker
    sub Speak( )
end interface

class Animal
    implements ISpeaker
    declare virtual sub Speak( ) implements ISpeaker.Speak
end class

sub Animal.Speak( )
    print "..."
end sub

class Dog
    inherits Animal
    declare virtual sub Speak( ) override
end class

sub Dog.Speak( )
    print "woof"
end sub

class Puppy
    inherits Dog
    declare sub Speak( ) override
end class

sub Puppy.Speak( )
    print "yip"
end sub

dim as Animal a
dim as Dog d
dim as Puppy p
dim as ISpeaker speakers( 0 to 2 )
speakers( 0 ) = a
speakers( 1 ) = d
speakers( 2 ) = p
for i as int32 = 0 to 2
    speakers( i ).Speak( )
next

...
woof
yip

class Dog names its base with an inherits line, the class's spelling of extends, at the top of its body. Animal implements ISpeaker.Speak with a virtual method, so an ISpeaker that refers to a Dog or a Puppy reaches their overrides. The rule from Part 1 still holds: Dog writes virtual … override so that Puppy can override it in turn.

Use this only when you want a base class that supplies code its subclasses replace. The default style is the one in the previous sections: each class implements the interfaces it needs, with no base class and no virtual anywhere, and shared code goes in a procedure that takes the interface.

Generic interfaces and constraints

An interface can take type parameters. Each instance, IComparable( of Money ) or IComparable( of Date ), is an interface of its own. And a generic procedure or type can require its type argument to implement one, with as in its parameter list:

'' ifacegeneric.tiko -- a generic interface, and a generic procedure
'' that requires one of its type argument.

interface IComparable( of T )
    function CompareTo( byref other as T ) as int32
end interface

class Money
    implements IComparable( of Money )
    declare constructor( byval cents as int32 = 0 )
    declare function CompareTo( byref other as Money ) as int32 _
        implements IComparable( of Money ).CompareTo
    cents as int32
end class

constructor Money( byval cents as int32 )
    this.cents = cents
end constructor

function Money.CompareTo( byref other as Money ) as int32
    return sgn( cents - other.cents )
end function

'' T must implement IComparable( of T ); the body calls T's own
'' CompareTo directly.
function Largest( of T as IComparable( of T ) )( a( ) as T ) as T
    dim as int32 best = lbound( a )
    for i as int32 = lbound( a ) + 1 to ubound( a )
        if a( i ).CompareTo( a( best ) ) > 0 then best = i
    next
    return a( best )
end function

dim as Money prices( 0 to 2 ) = _
    { Money( 250 ), Money( 1999 ), Money( 75 ) }
print "largest:"; Largest( prices( ) ).cents

largest: 1999

of T as IComparable( of T ) is a constraint: whatever T is, it must implement IComparable of itself. Several are written in braces, of K as { IComparable( of K ), IShow }. The constraint is checked once, where Largest is used, and a type argument that falls short is refused there:

'' constraint.tiko -- a type argument that does not implement what the
'' parameter requires is refused where the generic is used.

interface IComparable( of T )
    function CompareTo( byref other as T ) as int32
end interface

function Largest( of T as IComparable( of T ) )( a( ) as T ) as T
    return a( lbound( a ) )
end function

dim as int32 counts( 0 to 2 ) = { 3, 1, 2 }
print Largest( counts( ) )

constraint.tiko(13,14): error TKO1022: type argument 'int32' does not implement 'icomparable( of int32 )', which 'largest( of t )' requires
     |
  13 | print Largest( counts( ) )
     |              ^

Without the constraint, Largest( counts( ) ) would fail inside the body, at a( i ).CompareTo, with an error about a member that int32 does not have, and the reader would have to work out why from there. A constraint moves the error to the call that caused it.

A constraint does not change how the body runs. Largest is still compiled once for each type it is used with, and inside it a( i ).CompareTo( … ) is an ordinary call to Money.CompareTo: no interface value and no table. Chapter 17 covers generics in full.

Choosing between them

The two mix freely. A class can inherit a base and implement interfaces, a type can extend a class, and a classic base can implement an interface with virtual methods that its descendants override.

What it costs

• Every object of a type that extends Object carries one pointer, eight bytes, plus whatever padding it causes. A million small objects pay eight megabytes for it.
• A virtual call is an indirect call: load the table pointer, load the slot, call through it. That is a few instructions, but it prevents the call from being inlined, which matters inside a hot loop over small methods.
• A non-virtual method costs nothing extra, inherited or not. Make methods virtual because a derived type needs to replace them, not by default.
• is walks the chain of base types, so it costs a little more the deeper the hierarchy.
• Abstract types and override are compile-time only. They cost nothing at run time.
• An interface value is sixteen bytes, the object's address and its table for that interface, and a call through it is one indirect call, the same as a virtual call. A class whose implementing method is virtual adds a short hop through its own table, so an override is reached.
• Implementing an interface adds nothing to an object. A plain type stays the size of its fields; the table is in the interface value.
• trycast and is with an interface search a small table at run time. Converting an object to an interface, or an interface to one it inherits, does not.
• Mappings and constraints are compile-time only. A constrained generic calls T's methods directly.

Summary

• type B extends A starts B as a complete A, base fields first. base( … ) chooses the base constructor, which runs before the derived body; base.Method( ) calls the base's version.
• A base with no default constructor forces every derived type to write a copy constructor (TKO0926). Default the base constructor's parameters instead.
• Without virtual, a call is chosen by the type the compiler sees. A method of the same name on a derived type hides the base's; it does not replace it for code that sees the base.
• extends Object adds a table pointer. virtual methods are looked up in it at run time; override replaces one (TKO0621 if there is nothing to replace); abstract declares one with no body. An override is not virtual unless you also write virtual, so a type derived from it hides the method instead of overriding it.
• A type with an abstract method cannot be created (TKO0620). --dump-vtable shows the table the compiler built.
• Give a base that will be deleted through a pointer a virtual destructor, or the derived destructor never runs, and repeat virtual at every level below it: a plain destructor in the middle stops the chain.
• a is T is true for T and anything derived from it; guard every cast( T ptr, @a ) with it. Prefer a virtual method to a chain of is tests.
• Passing by value slices an object to its base part. Pass objects with virtual methods byref.
• Use inheritance for "is a" and composition for "has a". A type extends one base, so an abstract type as a contract uses that base up.
• An interface lists methods and properties with no data or code. A type or class takes it on with an implements line and maps each member with implements IFoo.Member, explicitly; a missing one is TKO1000. class is a type that always extends Object, with its base on an inherits line.
• A type may implement any number of interfaces. Members of the same name in two interfaces are different members; a method's name need not match its member's, one method may provide several, and a private method may provide one.
• An interface value is a 16-byte reference. It converts from any implementing object or pointer, is called with ., and does not own its object: temporaries are refused (TKO1012), delete through it is refused (TKO1013), and the object must outlive it.
• is and trycast take interfaces on either side; trycast answers null instead of guessing.
• A plain type can implement an interface without a table pointer of its own.
• Generic interfaces are interfaces per instance. A constraint, of T as IFoo, is checked where the generic is used (TKO1022) and costs nothing in the body.

Exercises

1. Add a Triangle with a base and a height to the shapes program, without changing Shape, Circle or the loop. What did you have to write, and what did you not? {solution="ex14-01"}
2. Write a Counter plugin whose Apply appends the length of the text and counts its calls, and whose destructor prints the count. Create it with new, use it through a Plugin ptr, and delete it through that pointer. Which line makes the destructor run? {solution="ex14-02"}
3. Remove virtual from vdtor.tiko's Careful destructor. Predict the output, then run it.
4. Change ByValue in slicing.tiko to take its parameter byref. Which lines of output change, and why?
5. A Stack that extends Array, or a Stack that has an Array field: argue for one. What can a caller do to the first that a stack should not allow?
6. Add a Triangle to ifaceshapes.tiko. Then give Rect a second interface, IScalable, with sub Scale( byval k as float64 ), and scale a rectangle through it. Check with is that a triangle is not scalable. Which existing code did you change? {solution="ex14-03"}
7. Rewrite pluginhost.tiko with an IPlugin interface. Who deletes the plugins now, and why can the host no longer do it through the interface?

Modules and Packages

Sooner or later a program uses two libraries written by people who never met, and both of them define Parse. With #include there is nothing to be done about it. A header is text pasted into the file that includes it, so both Parses land in the same scope and the program does not compile. That is not the only cost. A header's helpers are visible to everyone who includes it. Its code is compiled again by every file that includes it. And when two libraries each need setting up before main runs, the order they are set up in depends on how the program happened to be linked.

A module is tiko's answer to all four. It is a directory of source files that compiles as one unit, shows the world only what it marks public, keeps its names behind its own name, is built once and cached, and is initialised in a fixed order: after everything it uses, before anything that uses it. A package is a set of modules shipped together, with a small tiko.toml saying what it is called and what it depends on.

This chapter builds a module, then several, then a package. Everything in it is a tiko extension: #include still works exactly as Chapter 16 describes, and a program that never writes #import compiles as if modules did not exist.

What #include cannot do

Here are two small header libraries. csv.ti counts the fields in a line; ini.ti finds where a value starts. Each is reasonable on its own:

'' csv.ti -- a header library: its code is pasted into every includer.
function Parse( byval s as ustring ) as int32
    '' the number of fields in a comma-separated line
    var n = 1
    for i as int32 = 1 to len( s )
        if mid( s, i, 1 ) = "," then n += 1
    next
    return n
end function

'' ini.ti -- another header library, by another author.
function Parse( byval s as ustring ) as int32
    '' where the value starts in a "key=value" line
    return instr( s, "=" ) + 1
end function

A program that wants both cannot have them:

#include once "headers/csv.ti"
#include once "headers/ini.ti"

print Parse( "a,b,c" )

headers/ini.ti(2,1): error TKO0600: 'parse' is already declared in this scope
     |
   2 | function Parse( byval s as ustring ) as int32
     | ^
headers/csv.ti(2,1): note: declared here
     |
   2 | function Parse( byval s as ustring ) as int32
     | ^

Neither author did anything wrong. The only fix available to them is to wrap their library in a namespace, and every library author has to remember to do it. The problem is the mechanism: #include puts every name a library declares into the scope of whoever includes it.

A module is a directory

hello.tiko
greet/
    greet.tiko

#module "greet"

public function Hello( byval who as ustring ) as ustring
    return "Hello, " & Capitalise_( who ) & "!"
end function

function Capitalise_( byval s as ustring ) as ustring
    return ucase( left( s, 1 ) ) & mid( s, 2 )
end function

#import "./greet"

print greet.Hello( "world" )

Hello, World!

Three rules are at work.

1. Every file of a module begins with #module "name", and the name is the directory's name. Comments may come first; any other directive is TKO0360. A name that does not match its directory is TKO0361.
2. #import comes before any code. The path is quoted, uses / on every platform, and ./greet is relative to the file that imports it. An #import after any statement — even after a #define or a #cmdline — is TKO0362.
3. The module's names are reached through the module's name. The import binds greet, the last segment of the path, and the program calls greet.Hello.

Because a module's name becomes a name in the program, no segment of an import path may be a keyword: #import "./sub/inner" is TKO0364, 'sub' is a keyword. Name modules for what they hold — text, report — not after statements.

There is no list of files on the command line and no build script. tiko hello.tiko reads the #import, finds greet/, compiles it, and links it in.

Who can see what

Hello is marked public; Capitalise_ is not. Reaching for it from outside is refused, and the message names the rule:

#import "./greet"

print greet.Capitalise_( "world" )        '' refused

notpublic.tiko(3,13): error TKO0860: 'capitalise_' is not public in module 'main.greet'
     |
   3 | print greet.Capitalise_( "world" )        '' refused
     |             ^

main.greet is the module's chain: its package's name followed by its path. A program with no tiko.toml is the package main. The chain is how tiko tells two modules called greet apart.

A module has three levels of visibility, and the default is the middle one:

That default is the opposite of a classic program's, where a procedure is visible everywhere unless it says private. It is the right way round for a library: a helper you forgot to mark never becomes part of your API, so you are free to change it later.

The difference between the last two is what each one means to the linker. Measured on the object tiko -c writes for a module named gen with one procedure at each level:

Every name is spelled from the chain, so two packages' Parse procedures can never collide in the linker either. A module-private procedure is still an external symbol, because code compiled outside the module sometimes has to call it: a generic's body, and an exported macro's expansion, are compiled in the importer. A private one is local to the module's object and cannot be reached that way at all. The section on generics shows the difference.

A public procedure may not name a type that is not public. An importer should never be handed a value of a type it cannot name, so public function Make( ) as Secret, where Secret has no public, is refused: TKO0868: 'secret' is module-private but appears in the public declaration of 'make'. Every type that appears in a public signature is public too.

A module with several files

A module may be as many files as it needs, and all of them form one module. They see each other's non-private names freely, in any order:

#module "report"

public function Row( byval label as ustring, _
                       byval value as int32 ) as ustring
    return PadRight_( label ) & Format_( value )
end function

#module "report"

'' Both helpers are module-private: report.tiko calls them, and
'' nothing outside the module can.
function PadRight_( byval s as ustring ) as ustring
    return left( s & space( LABEL_WIDTH ), LABEL_WIDTH )
end function

function Format_( byval n as int32 ) as ustring
    return right( space( 6 ) & str( n ), 6 )
end function

const LABEL_WIDTH = 10

#import "./report"

print report.Row( "apples", 12 )
print report.Row( "pears", 3075 )

apples        12
pears       3075

report.tiko calls PadRight_ and Format_, which are declared in another file. PadRight_ uses LABEL_WIDTH, which is declared below it. None of that needs a declare line, and there is no internal header gluing the files together, because a module is compiled as a single unit and its top-level declarations are collected before any of them is checked.

Inside a procedure body nothing changes: a local variable is still declared before it is used. And code outside any module keeps declare-before-use for everything, as Chapter 7 described.

Macros do not cross between the files of a module. A #define in report.tiko is not visible in format.tiko; each file starts from the same clean slate. A macro several files need belongs in a small header that each of them includes.

Imports do not cross between the files either. An import belongs to the file that writes it, so each file says where its own names come from. A sibling file that writes util. without an #import "../util" of its own gets TKO0743: 'util' is not declared, with a note pointing at the file that did import it: 'util' is imported by another file of this module; an import belongs to the file that writes it.

Importing

#import has two optional parts, and the imported names can be brought into scope with using:

#module "shapes"

public const PI_ = 3.14159265358979

public type Circle
    r as float64
    declare function Area( ) as float64
end type

function Circle.Area( ) as float64
    return PI_ * r * r
end function

public function UnitCircle( ) as Circle
    return type( 1.0 )
end function

#import "./shapes" as sh
#import "./greet"
using greet

dim c as sh.Circle
c.r = 2
print c.Area( )

var u = sh.UnitCircle( )
print u.r, sh.PI_

print Hello( "using" )

 12.56637061435916
 1             3.14159265358979
Hello, Using!

• as sh renames the module for this program. Use it when two imports end in the same name, or when a name is long. The module is still shapes; only the spelling here changes.
• Types are reached the same way as procedures: dim c as sh.Circle. A value of a module's type behaves exactly as one of your own: c.Area( ) needs no qualification, because a method belongs to its type.
• using greet makes greet's public names usable bare, as using does for a namespace.

When two modules you are using both supply a name, a bare use of it is refused rather than guessed:

#import "./tag"
#import "./label"
using tag
using label

print Kind( )                             '' refused

ambiguous.tiko(6,7): error TKO0919: 'kind' is ambiguous: more than one USING in effect declares it; qualify it
     |
   6 | print Kind( )                             '' refused
     |       ^

Write tag.Kind( ) or label.Kind( ) and move on. That is the qualified name doing its job: the question "which Kind?" has an answer written in the source.

Passing an import on

An import is private to the module that writes it. A module can pass one on to its own importers by adding public to the #import:

#module "kit"
#import "../greet" public       '' re-exported: importers see kit.greet
#import "../shapes"             '' kit's own business

public function Banner( ) as ustring
    return greet.Hello( "kit" ) & " pi=" & left( str( shapes.PI_ ), 4 )
end function

#import "./kit"

print kit.Banner( )
print kit.greet.Hello( "again" )

Hello, Kit! pi=3.14
Hello, Again!

kit re-exports greet, so an importer of kit reaches it as kit.greet. shapes is imported without public, so it stays kit's business:

#import "./kit"

print kit.shapes.PI_                       '' refused

noleak.tiko(3,11): error TKO0745: the namespace 'kit' has no member named 'shapes'
     |
   3 | print kit.shapes.PI_                       '' refused
     |           ^

That is what makes a module's dependencies its own to change. Swapping shapes for something else inside kit cannot break a program that imports kit, because no such program could have used it.

Packages

So far every module has sat beside the program that uses it. Once modules are meant for more than one program, they become a package: a directory tree of modules with a tiko.toml at its root.

acme/
    tiko.toml
    config/config.tiko
    text/pad/pad.tiko
app/
    tiko.toml
    main.tiko

# The package acme: two modules, and one setting with a default.
[package]
name = "acme"
version = "0.3.0"

[defines]
ACME_COLUMNS = "8"

#module "config"

'' ACME_COLUMNS comes from the package's tiko.toml, or from the
'' program that depends on acme -- never from the program's code.
public function Columns( ) as int32
    return ACME_COLUMNS
end function

#module "pad"
#import "acme/config"

public function Cell( byval s as ustring ) as ustring
    return "|" & left( s & space( config.Columns( ) ), _
                       config.Columns( ) )
end function

pad imports acme/config by its full path. Inside a package, a path that starts with the package's own name means "this package". The module pad's chain is acme.text.pad, and the program still calls it pad: the last segment.

The program says where to find acme:

[package]
name = "app"

[dependencies]
acme = { path = "../acme" }

#import "acme/text/pad"

print pad.Cell( "tiko" ); pad.Cell( "modules" ); "|"

|tiko    |modules |

A tiko.toml is optional — the programs before this section had none — and you add one for four things:

1. A stable name. Without one, a package is called main. With one, the name is written in the code and goes wherever the code goes.
2. A root. The directory holding tiko.toml is where the package starts. A relative import may not climb above it (TKO0142).
3. Dependencies that travel with the project. acme = { path = "../acme" } is in the repository, not in anyone's environment, so a fresh checkout builds with a bare tiko main.tiko.
4. Settings for a library. That is [defines], and it needs a section of its own.

The file is a small subset of TOML: tables, strings, integers, booleans, inline tables and arrays of strings. Anything it does not recognise is TKO0144, naming the file and line. version is recorded for a future package manager and not otherwise used.

Where an import is looked for

For #import "acme/text/pad", tiko tries these in order and takes the first that has it:

Nothing found is TKO0140, and the message lists every place it looked. std is special: it is always the toolchain's, so no project can replace the standard library by accident. And one build may not find two different directories both claiming the same package name (TKO0149).

Settings that cross the boundary, and settings that do not

acme's config module reads ACME_COLUMNS, which no source file defines. It comes from the package's [defines], and a program that depends on acme can override it in its own manifest:

[package]
name = "tuned"

[dependencies]
acme = { path = "../acme" }

[dependencies.acme.defines]
ACME_COLUMNS = "4"

|tiko|modu|

The same main.tiko now prints four-column cells. That is the only way in. A module never sees the macros of the program that imports it — not its #defines, and not its -d options either. This is deliberate, and it is what lets a module be compiled once and cached: if a program's macros could reach into a module, every program would need its own copy of it.

#module "cfg"

public function Mode( ) as ustring
#ifdef VERBOSE
    return "verbose"
#else
    return "quiet"
#endif
end function

#import "./cfg"
#cmdline "-d VERBOSE"

#ifdef VERBOSE
print "main: VERBOSE is defined"
#endif
print "cfg:  "; cfg.Mode( )

main: VERBOSE is defined
cfg:  quiet

Initialisation

A module may have top-level statements, shared variables with constructors, and constructor and destructor subs. tiko runs them in one order, the same on every target and whatever order the objects are linked in:

A module is initialised after every module it imports, and before any code that imports it. Shutdown is the exact reverse.

#module "logger"

type Sink
    lines as int32
    declare constructor( )
    declare destructor( )
end type

constructor Sink( )
    print "  logger: sink opened"
end constructor

destructor Sink( )
    print "  logger: sink closed after"; lines; " line(s)"
end destructor

dim shared sink_ as Sink
print "  logger: top-level code"

public sub Info( byval s as ustring )
    sink_.lines += 1
    print "  log: "; s
end sub

#module "store"
#import "../logger"

logger.Info( "store opening" )

sub Close_( ) destructor
    logger.Info( "store closing" )
end sub

public function Count( ) as int32
    return 3
end function

#import "./store"

sub Early( ) constructor 101
    print "constructor 101"
end sub

print "main: store has"; store.Count( ); " items"

constructor 101
  logger: sink opened
  logger: top-level code
  log: store opening
main: store has 3 items
  log: store closing
  logger: sink closed after 2 line(s)

Read the output top to bottom. logger is set up completely — its shared Sink constructed, then its top-level code — before store's top-level code calls logger.Info. store is set up before main runs. At the end, store's destructor runs while logger is still there to log it, and only then is the Sink destroyed. Each module is initialised once, however many modules import it.

That is the guarantee classic code cannot make. Without modules, two libraries' setup code runs in the order the linker happened to place it, and that order differs between Windows and Linux.

Numbered constructors keep their meaning. constructor 101 asks for a slot by number, and those slots run before any module is initialised — which is why Early printed first. A number must be from 101 to 65535 (TKO0422). Use one only for something that genuinely must precede everything, and prefer the import order, which says why one thing comes before another.

Macros and generics across modules

Exporting a macro

A module's macros are its own unless it exports them by name, after defining them:

#module "check"

#macro Ensure( cond )
    if ( cond ) = 0 then Fail_( #cond, __FILE__, __LINE__ )
#endmacro
#export Ensure

#define MAX_ITEMS 10
#export MAX_ITEMS

sub Fail_( byval what as ustring, byval f as ustring, _
           byval ln as int32 )
    print "check failed: "; what; " ("; f; " line"; ln; ")"
end sub

#import "./check"

'' A Fail_ of this program's own, which the macro must not call.
sub Fail_( byval what as ustring, byval f as ustring, _
           byval ln as int32 )
    print "wrong Fail_"
end sub

var items = 12
check.Ensure( items <= check.MAX_ITEMS )
check.Ensure( items > 0 )
print "done"

check failed: items <= 10 (macros.tiko line 10)
done

• An exported macro is used qualified, as check.Ensure( … ). A bare Ensure( … ) is not a macro at all, so importing a module can never capture a name of yours.
• Its body means what it meant in the module. Fail_ in Ensure's body is check's Fail_, even though the program cannot name it and has a Fail_ of its own. Names the module does not declare — items, the macro's argument — are left as the caller wrote them.
• __FILE__ and __LINE__ expand where the macro is used, as they always do. And #cond shows the argument after expansion, so check.MAX_ITEMS appears as 10.
• The module's own headers keep their macros too. When a public declaration names a type from one of the module's headers, your program reads that header to get the type, but the header's macros stop there. A module that wants a header shared whole, macros and all, says public #include. Headers from outside the package, such as windows.bi, are read as if you had included them yourself.

Exporting a generic

#module "coll"

public type Stack( of T )
    declare sub Push( byval v as T )
    declare function Pop( ) as T
    declare function Count( ) as int32
private:
    items( any ) as T
end type

sub Stack( of T ).Push( byval v as T )
    redim preserve items( ubound( items ) + 1 )
    items( ubound( items ) ) = v
    Counted_( )
end sub

function Stack( of T ).Pop( ) as T
    var v = items( ubound( items ) )
    if ubound( items ) = 0 then
        erase items
    else
        redim preserve items( ubound( items ) - 1 )
    end if
    return v
end function

function Stack( of T ).Count( ) as int32
    return ubound( items ) + 1
end function

'' Module-private, yet reached from the importer: Push's body is
'' instantiated there, and calls it.
sub Counted_( )
    pushes += 1
end sub

public dim shared pushes as int32

#import "./coll"

dim nums as coll.Stack( of int32 )
nums.Push( 1 )
nums.Push( 2 )
print nums.Pop( ); nums.Count( )

dim names as coll.Stack( of ustring )
names.Push( "ada" )
print names.Pop( ); names.Count( )

print "pushes:"; coll.pushes

 2 1
ada 0
pushes: 3

A generic is instantiated by the program that uses it, so its body travels with the module's interface and is compiled in the importer. That is why Counted_ is module-private rather than private: Push's body calls it from the program's own code, and a module-private name is an external symbol that code can reach. Make the helper private and the instantiation fails with TKO0743: 'counted_' is not declared, because a private name does not exist outside the module's object.

Building

tiko main.tiko does everything: it finds the imports, builds each module once, and links the result. Four options show or change what it did.

--print-graph lists the modules a program imports, in the order they are built, and stops. Each line is a chain, the directory it came from, and what it imports:

> tiko --print-graph main.tiko
acme.config    ../acme/config
acme.text.pad  ../acme/text/pad  [acme.config]

-v reports, per module, whether the build cache had it. Three builds of the two programs above:

module acme.config: miss (new)
module acme.text.pad: miss (new)

module acme.config: hit
module acme.text.pad: hit

module acme.config: miss (define ACME_COLUMNS=4)
module acme.text.pad: miss (define ACME_COLUMNS=4)

The cache is one directory shared by every project of yours: %LOCALAPPDATA%\tiko\modcache on Windows, ~/.cache/tiko/modcache on Linux (or under $XDG_CACHE_HOME), and wherever TIKO_MODCACHE names if you set it. Nothing is written into the project, so there is nothing to add to .gitignore, and a module two projects build the same way is compiled once for both. A build with different settings is a different entry, as the third build shows. The key is the module's content, not file dates, so touching a file changes nothing.

What a change rebuilds, measured on acme:

If you ever doubt the cache, tiko --clean-module-cache empties it, and --no-module-cache builds without reading or writing it. Nothing is lost but time.

--emit-tki prints a module's interface, the declarations an importer is compiled against, as ordinary tiko source:

> tiko --emit-tki thermo
'' tiko-interface 1
'' module main.thermo
namespace __tko_mod_main_thermo
declare function Round1_( byval x as float64 ) as float64
public declare function ToFahrenheit( byval c as float64 ) as float64
end namespace

(The real output carries a #line before each declaration, pointing back at the source.) The module-private Round1_ is in it, without public: that is how a generic or macro body compiled in the importer can reach it, while the importer's own code cannot.

Objects, libraries and DLLs

The other output kinds work on a program that imports:

• tiko -c -m main main.tiko -o out/main.o compiles to objects, writing each module beside the output as its chain — out/main.store.o, out/main.logger.o. Linking out/main.o alone later finds the modules' objects through a record inside it.
• tiko -lib puts every module's object into the archive, so the library is complete on its own.
• tiko -dll links the modules in and exports only what the root marks export. A module's names never are.

__TKO_MODULE__ is defined while a module is compiled and not otherwise, and __TIKO_COMPILER__ is defined by tiko always: the test for "am I being compiled by tiko", for code that must also build elsewhere.

The standard library as a package

The standard library is the package std, shipped in the toolchain's pkg/ directory. Its modules are strings, fs, containers, chrono, tasks, outcome, numeric, money, logs, tui and memory, and the library chapters (Part IV) use them throughout:

#import "std/strings"
#import "std/fs"

print strings.Replace( "a-b-c", "-", "+" )
print strings.StartsWith( "tiko-lang", "tiko" )
print strings.Reverse( "tiko" )

print fs.PathStem( "notes.txt" ); " "; fs.PathExtension( "notes.txt" )
print fs.PathFileName( "logs/today.txt" )

a+b+c
true
okit
notes .txt
today.txt

These are ordinary modules, built the way this chapter describes: compiled once, cached, and initialised in import order. Everything in the chapter applies to them:

• using strings makes Replace a bare name, as for any module. Two usings that supply the same name -- strings.Contains and containers.Contains -- are TKO0919 when the name is used bare; qualify it.
• Their macros are exported, and used qualified: logs.LogInfo( "started" ).
• A statement may need one. spawn and select channel lower to std/tasks, so a file that uses them imports it (TKO0423 otherwise), and builds with -mt.

Converting a header library into a module

Most existing libraries are headers. Moving one to a module is mostly mechanical, and the result is easier to use and cheaper to build. Here is a header library, the classic way:

'' thermo.ti -- a header library, the classic way.
#pragma once

namespace Thermo

private function Round1_( byval x as float64 ) as float64
    return int( x * 10 + 0.5 ) / 10
end function

private function ToFahrenheit( byval c as float64 ) as float64
    return Round1_( c * 9 / 5 + 32 )
end function

end namespace

#include once "classic/thermo.ti"

print Thermo.ToFahrenheit( 21.5 )

And the same library as a module, with a program using it:

'' thermo -- the same library, as a module.
#module "thermo"

function Round1_( byval x as float64 ) as float64
    return int( x * 10 + 0.5 ) / 10
end function

public function ToFahrenheit( byval c as float64 ) as float64
    return Round1_( c * 9 / 5 + 32 )
end function

#import "./thermo"

print thermo.ToFahrenheit( 21.5 )

Both programs print the same line:

 70.7

The recipe, file by file:

Two steps need judgement. Headers that include each other cannot become two modules, because imports may not form a cycle (TKO0141, which names the cycle); merge them into one module, or move what they share into a third. And a system header your public signatures mention — a Windows handle type, say — stays an #include in the module. The module's interface records that it needs the header, and including it again in the importer is harmless: it is the same file, read once.

What it costs

• A module is compiled once and cached, so the first build of a program that imports a large module is slow and every later build is not. A header library is compiled again by every file that includes it, every time.
• Changing a public declaration rebuilds every module that imports it, directly or through others. Changing a body rebuilds one module. Keep public interfaces stable and the rebuild stays small.
• Calls across a module boundary are ordinary calls. A module is not a separate binary at run time; the linker joins everything into one executable, as before.
• Generics are instantiated in the importer, so each program that uses coll.Stack( of int32 ) compiles that instance itself, and a program that instantiates many pays in compile time and binary size exactly as Chapter 17 described.
• The cache costs disk space and is never cleaned on its own. --clean-module-cache empties it.

Summary

• #include pastes a library into its includer: two libraries' names collide (TKO0600), helpers are visible to all, and the code is compiled again by every includer.
• A module is a directory; every file begins #module "name", matching the directory (TKO0360, TKO0361). #import "./greet" comes before any code (TKO0362) and binds greet.
• Module-private is the default. public exports; private keeps a name inside the module's object. Names are spelled from the module's chain, so they never collide in the linker either. Reaching a module-private name is TKO0860.
• Inside a module, declaration order does not matter, across files or within one. Macros do not cross between its files.
• #import … as x renames; using makes public names bare, and a name two usings supply is TKO0919. #import … public re-exports; otherwise imports never leak (TKO0745).
• A package is a directory tree with a tiko.toml: a name, a root, path dependencies, and [defines] a program overrides through [dependencies.NAME.defines].
• A module never sees its importer's macros or -d options, which is what makes it cacheable, and which silently takes a module's #else branch if you expect otherwise.
• Modules initialise after what they import and finalise in reverse, once each, the same on every target. Numbered constructors run first.
• #export NAME shares a macro, used qualified and hygienic. An exported generic is instantiated in the importer, so its helpers must be module-private, not private.
• --print-graph, -v, --no-module-cache, --clean-module-cache and --emit-tki show and control the build; -c, -lib and -dll work on programs that import.
• std is a package in the toolchain, and its modules are ordinary modules.
• Converting a header library: add #module, drop the guard and the namespace, mark the API public, unmark the helpers, and turn configuration macros into [defines].

Exercises

1. Write a module temp with public CToF and FToC and a module-private rounding helper, and a program that prints a table from −10 to 30 degrees in steps of 10, converting each value there and back. {solution="ex15-01"}
2. Write a module counter whose top-level code prints that it is initialising and whose destructor sub prints that it is done, with a public shared hits. Import it from two modules, left and right, that each add one to hits, and import all three from a program. Predict how often counter initialises, then run it. {solution="ex15-02"}
3. Move report's LABEL_WIDTH into a third file of the module. What did you have to change elsewhere, and why?
4. Make acme's column count a parameter of pad.Cell instead of a [defines] setting. What does the program gain, and what does the package lose?
5. Take a header library of your own and convert it with the recipe. Which step needed a decision rather than an edit?

Namespaces, Headers and the Preprocessor

A program outgrows one file long before it outgrows one author. The geometry goes in one file, the measuring in another, the program that uses them in a third, and each has to know enough about the others to call them. Chapter 15 builds that out of modules, which is what new code should do. This chapter is the machinery underneath: files compiled one at a time and joined by the linker, headers that tell each file what the others provide, namespaces that keep their names apart, and the preprocessor, which rewrites the source before the compiler reads a word of it.

You need all of it anyway. Every binding to a C library is a header. Every program that must behave differently on Windows and Linux says so with the preprocessor. And a module is itself built from these parts, so the rules here are the ones that hold underneath Chapter 15.

A program in several files

Here is a small program in three source files. The part they share goes in a header, a .ti file that declares what exists without defining it:

'' geometry.ti -- what the three files of the program share.
#pragma once

type Point
    x as float64
    y as float64
end type

'' Defined in shapes.tiko.
declare function Distance( byval a as Point, _
                           byval b as Point ) as float64
declare function Perimeter( pts() as Point ) as float64

'' Defined in measure.tiko, read everywhere.
extern measured as int32

declare promises a procedure that some other file defines; extern promises a variable. The two source files keep those promises. shapes.tiko defines the procedures, and measure.tiko owns the counter:

'' shapes.tiko -- distances and perimeters.
#include once "geometry.ti"

private function Square( byval v as float64 ) as float64
    return v * v
end function

function Distance( byval a as Point, byval b as Point ) as float64
    measured += 1
    return sqr( Square( a.x - b.x ) + Square( a.y - b.y ) )
end function

function Perimeter( pts() as Point ) as float64
    dim total as float64 = 0
    for i as int32 = lbound( pts ) to ubound( pts )
        var j = iif( i = ubound( pts ), lbound( pts ), i + 1 )
        total += Distance( pts( i ), pts( j ) )
    next
    return total
end function

'' measure.tiko -- owns the counter that geometry.ti declares.
#include once "geometry.ti"

dim shared measured as int32

The main file includes the same header and uses all of it:

'' geometry.tiko -- the main module.
#include once "geometry.ti"

dim box( 0 to 3 ) as Point = { ( 0, 0 ), ( 4, 0 ), ( 4, 3 ), ( 0, 3 ) }

print "diagonal: "; Distance( box( 0 ), box( 2 ) )
print "perimeter:"; Perimeter( box() )
print "distances measured:"; measured

diagonal:  5
perimeter: 14
distances measured: 5

Build it by naming every source file; the first one named is the program:

tiko geometry.tiko shapes.tiko measure.tiko -o geometry

Each .tiko file is compiled on its own, as if the others did not exist. That is why each one includes the header: a file knows only what it has been told. The linker then joins the compiled files and connects every declare and extern to the one definition behind it. dim shared measured in measure.tiko is that definition; extern measured in the header lets the other two files use it.

#include pastes a file's text in place. #include once pastes it only the first time, and #pragma once at the top of a header says the same thing from the inside, so the header is safe however many times it is reached. geometry.ti says both ways, which costs nothing.

Where #include looks

'' platform.tiko -- one source, two platforms.
#include once "include/platform.ti"

print "platform: "; PLATFORM
print "pointers:"; POINTER_BITS; " bits"
print "a path:   logs"; PATH_SEP; "today.txt"
#ifdef __FB_MSVC__
print "target:   windows-x86_64"
#endif

'' platform.ti -- what this program needs to know about where it runs.
#pragma once
#include once "separator.ti"

#if defined( __FB_WIN32__ )
    const PLATFORM = "Windows"
#elseif defined( __FB_LINUX__ )
    const PLATFORM = "Linux"
#elseif defined( __FB_DARWIN__ )
    const PLATFORM = "macOS"
#else
    #error "platform.ti: this platform is not supported"
#endif

#ifdef __FB_64BIT__
    const POINTER_BITS = 64
#else
    const POINTER_BITS = 32
#endif

'' separator.ti -- found beside platform.ti, which includes it.
#pragma once

#ifdef __FB_WIN32__
    const PATH_SEP = "\"
#else
    const PATH_SEP = "/"
#endif

platform.tiko names include/platform.ti, a path relative to itself. platform.ti then includes separator.ti with no directory at all, and finds it beside platform.ti: a name is looked for first beside the file that includes it, not beside the program's first file. That is what lets a header bring its own helpers along without knowing where it will be used from.

When the file is not there, tiko tries the directory the compiler was run in, then each directory given with -i, in the order they were given, and finally its own inc/, which holds the C bindings and the classic headers that ship with the toolchain. The first place that has the file wins. So a header of your own in the working directory, named like one of the toolchain's, is the one your files get when they include that name, in place of the toolchain's. Give your headers names of their own, or include them by a path relative to the file that needs them.

Headers or modules?

Chapter 15 opens with the two problems headers cannot solve: every name a header declares lands in the scope of whoever includes it, and its code is compiled again by every file that does. For your own code, a module is the better tool. Headers remain the right one for a binding to a C library, for macros meant to be shared between files, and for existing code that is already written as headers.

Linkage

Two files compiled on their own can both define a name, and neither compile notices. The linker does:

'' clash.tiko -- two files, each with a sub called Init.
sub Init( )
    print "report ready"
end sub

Init( )

'' clashother.tiko -- the other file.
sub Init( )
    print "export ready"
end sub

lld-link: error: duplicate symbol: INIT
clang: error: linker command failed with exit code 1 (use -v to see invocation)
tiko-lang: error TKO0116: link failed (exit 1)

(The linker's two >>> defined at lines, which name the temporary object files, are left out.)

A procedure is public by default: its name is visible to the linker, so every other file can call it, and two files cannot both have one. private keeps a procedure inside its own file. Make one of the two Inits private and the program links. The file that owns it reaches it through a public procedure:

'' privatefixother.tiko -- its Init is its own business.
private sub Init( )
    print "export ready"
end sub

sub StartExport( )
    Init( )
end sub

'' privatefix.tiko -- two subs called Init, and the program links.
declare sub StartExport( )

sub Init( )
    print "report ready"
end sub

Init( )
StartExport( )

report ready
export ready

Make helpers private as a habit. A public helper is a name another file can collide with, and an entry point nobody meant to offer.

Private by default, for a region

Marking every helper costs a word on each one. #pragma push( private, true ) turns the default around until the matching #pragma pop( private ). Inside the region a procedure is private unless it says public:

'' pragmaother.tiko -- one public entry point over private helpers.
#pragma push( private, true )

sub Validate( byval n as int32 )
    print "validating"; n
end sub

sub Store( byval n as int32 )
    print "storing"; n
end sub

'' Inside the region, but it says so: visible to the program.
public sub Flush( )
    print "flushing"
end sub

#pragma pop( private )

sub Save( byval n as int32 )
    Validate( n )
    Store( n )
end sub

'' pragma.tiko -- Save and Flush are all this file can reach.
declare sub Save( byval n as int32 )
declare sub Flush( )

Save( 42 )
Flush( )

validating 42
storing 42
flushing

pragma.tiko can reach Save, which follows the region, and Flush, which says public inside it. Declaring Validate there and calling it would fail at the link: no file offers a public Validate.

Shared variables

You have seen one way to share a variable: dim shared in exactly one file and extern in the header. The other is common. Every file that declares the same common shared variable gets the same storage, and no file has to be the owner:

'' tally.tiko -- both files declare the same common variable.
common shared hits as int32
declare sub Hit( )

Hit( )
Hit( )
Hit( )
print "hits:"; hits

'' tallyother.tiko -- the same declaration, the same storage.
common shared hits as int32

sub Hit( )
    hits += 1
end sub

hits: 3

Prefer extern. With extern, one file says where the variable lives and the header says what it is, so a mismatch is caught when the files are compiled. common makes every file restate the declaration, and nothing checks that they agree: declared int32 in one file and float64 in the other, the program still links, and counts wrong.

Startup and shutdown

A program sometimes has work to do before its first statement, and after its last. A sub marked constructor runs before the main program; one marked destructor runs after it. A number after the keyword is a priority:

'' startup.tiko -- the main module, with the log's setup and teardown.
private sub OpenLog( ) constructor 101
    print "101: open the log"
end sub

private sub CloseLog( ) destructor 101
    print "101: close the log"
end sub

print "main runs"

'' startupconfig.tiko -- the configuration, which needs the log.
private sub LoadConfig( ) constructor 102
    print "102: load the config"
end sub

private sub SaveConfig( ) destructor 102
    print "102: save the config"
end sub

'' Top-level code in a file that is not the main module.
print "startupconfig.tiko: top-level code"

101: open the log
102: load the config
startupconfig.tiko: top-level code
main runs
102: save the config
101: close the log

Constructors run lowest priority first, and destructors run in the opposite order, so whatever started first is shut down last: the configuration is saved before the log it writes to is closed. Give anything that depends on something else a higher number than the thing it depends on.

Look at where startupconfig.tiko's own top-level code ran: before the main program, after the constructors. Only one file is the main module, whose top-level code is the program. Every other file's top-level code runs as part of startup. The main module is the first file named, unless -m names another:

'' whichmain.tiko -- compiled first, but not the main module.
#cmdline "-m whichmainother"

print "whichmain.tiko: top-level code"

'' whichmainother.tiko -- named by -m, so its code is the program.
print "whichmainother.tiko: the main program"

whichmain.tiko: top-level code
whichmainother.tiko: the main program

#cmdline "-m whichmainother" puts the option in the source, so the listing builds the same way whoever compiles it.

Namespaces

A private procedure is hidden from other files, but that does not help when two parts of the same program both want a function called Area. A namespace gives names a family name:

namespace Geo
    const ORIGIN = 0

    function Area( byval w as float64, byval h as float64 ) as float64
        return w * h
    end function

    namespace Units
        const CM_PER_M = 100
    end namespace
end namespace

'' A namespace can be reopened, here or in another file.
namespace Geo
    function Perimeter( byval w as float64, _
                        byval h as float64 ) as float64
        return 2 * ( w + h )
    end function
end namespace

print "area, perimeter and centimetres per metre:"
print Geo.Area( 2, 3 ), Geo.Perimeter( 2, 3 )
print Geo.Units.CM_PER_M

sub Report( )
    using Geo
    print Area( 4, 5 ), Units.CM_PER_M
end sub

Report( )

area, perimeter and centimetres per metre:
 6             10
 100
 20            100

• Outside the namespace, a member is written qualified: Geo.Area, Geo.Units.CM_PER_M.
• Namespaces nest, and the path is written out in full.
• A namespace can be reopened. The second namespace Geo adds Perimeter to the same namespace, and another file can do the same.
• using Geo makes Geo's members usable without the prefix, from the using to the end of the block that contains it: here, the sub Report. Written at the top of a file, it lasts to the end of the file.

How a name is found

using makes a namespace's names available. It does not make them preferred:

namespace Defaults
    const LIMIT = 20
    function Label( ) as ustring
        return "from Defaults"
    end function
end namespace

const LIMIT = 10
function Label( ) as ustring
    return "from the module"
end function

sub Show( )
    using Defaults
    print LIMIT, Label( )
end sub

print "inside Show, under using Defaults:"
Show( )
print "qualified:"
print Defaults.LIMIT, Defaults.Label( )

inside Show, under using Defaults:
 10           from the module
qualified:
 20           from Defaults

Inside Show, LIMIT and Label are both declared twice: once in Defaults, which using brings in, and once at module level. A name declared on the way out — in the procedure, then in the enclosing scopes, then the module — beats every using. The namespace's members are reached only where nothing closer has the name. To mean the namespace's, qualify it.

When two usings both supply a name and nothing closer declares it, tiko does not pick one:

namespace Report
    function Title( ) as ustring
        return "Report"
    end function
end namespace

namespace Archive
    function Title( ) as ustring
        return "Archive"
    end function
end namespace

using Report, Archive
print Title( )

ambiguous.tiko(14,7): error TKO0919: 'title' is ambiguous: more than one USING in effect declares it; qualify it
     |
  14 | print Title( )
     |       ^

Macros

The preprocessor reads the source before the compiler does, and macros are its main tool. A macro is a name for a piece of text, and the preprocessor replaces every use of the name with the text:

#define LIMIT 100
#define SQUARE( x ) ( ( x ) * ( x ) )
#define AS_TEXT( x ) #x
#define PASTE( prefix, n ) prefix##n

#macro SWAP_VALUES( a, b )
    scope
        var t_ = a
        a = b
        b = t_
    end scope
#endmacro

dim as int32 width1 = 3, width2 = 8
print "limit:"; LIMIT
print "square of 1 + 2:"; SQUARE( 1 + 2 )
print "as text: "; AS_TEXT( width1 + width2 )
print "pasted:"; PASTE( width, 2 )
SWAP_VALUES( width1, width2 )
print "after the swap:"; width1; width2

limit: 100
square of 1 + 2: 9
as text: width1 + width2
pasted: 8
after the swap: 8 3

• #define LIMIT 100 is an object-like macro: LIMIT becomes 100 wherever it appears.
• #define SQUARE( x ) ... takes arguments. The text of each argument is substituted for the parameter, unevaluated.
• #x turns an argument into a string literal, so AS_TEXT( width1 + width2 ) is the text width1 + width2, not the sum.
• prefix##n pastes two pieces into one token: PASTE( width, 2 ) becomes the identifier width2.
• #macro … #endmacro defines a macro that spans several lines, which is how a macro expands to whole statements. SWAP_VALUES wraps its temporary in a scope so that two uses in one procedure do not both declare t_.

A macro cannot take the name of a reserved word: #define print 1 is TKO0371, "'print' cannot be defined as a macro: it is a reserved word".

#define BAD_SQUARE( x ) x * x
#define SQUARE( x ) ( ( x ) * ( x ) )

print "BAD_SQUARE( 1 + 2 ) ="; BAD_SQUARE( 1 + 2 )
print "SQUARE( 1 + 2 )     ="; SQUARE( 1 + 2 )
print "100 / SQUARE( 5 )   ="; 100 / SQUARE( 5 )

BAD_SQUARE( 1 + 2 ) = 5
SQUARE( 1 + 2 )     = 9
100 / SQUARE( 5 )   = 4

What a function cannot do is generate code. This macro writes an enum and a function that names its values, from one list:

'' One list of names, written once: the enum and its spelling.
#macro DEFINE_ENUM( name, a, b, c )
    enum name
        a
        b
        c
    end enum

    function name##_Name( byval v as name ) as ustring
        select case v
        case a : return #a
        case b : return #b
        case c : return #c
        end select
        return "?"
    end function
#endmacro

DEFINE_ENUM( Colour, RED, GREEN, BLUE )
DEFINE_ENUM( Size, SMALL, MEDIUM, LARGE )

print Colour_Name( GREEN ); " "; Size_Name( LARGE )
for c as Colour = RED to BLUE
    print c; " "; Colour_Name( c )
next

GREEN LARGE
 0 RED
 1 GREEN
 2 BLUE

Each value is written once, so the names can never disagree with the enum. name##_Name builds the function's name from the enum's, and #a turns each value into its own spelling.

The built-in macros

A family of built-in macros works on macro arguments themselves:

#define SIZES 8, 16, 32, 64

print "count:"; __FB_ARG_COUNT__( SIZES )
print "third:"; __FB_ARG_EXTRACT__( 2, SIZES )
#define PAIR key = value
print "left of =: "; __FB_QUOTE__( __FB_ARG_LEFTOF__( PAIR, = ) )
print "right of =: "; __FB_QUOTE__( __FB_ARG_RIGHTOF__( PAIR, = ) )
print "evaluated:"; __FB_EVAL__( 6 * 7 )

dim as int32 total32 = 32
print "joined:"; __FB_JOIN__( total, 32 )
__FB_UNQUOTE__( "print ""unquoted into code""" )

'' A name no other use of the macro can collide with.
#macro TIMES_TWO( v )
    __FB_UNIQUEID_PUSH__( tmp )
    var __FB_UNIQUEID__( tmp ) = v
    print "twice"; v; " is"; __FB_UNIQUEID__( tmp ) * 2
    __FB_UNIQUEID_POP__( tmp )
#endmacro
TIMES_TWO( 5 )
TIMES_TWO( 9 )

count: 4
third: 32
left of =: key
right of =: value
evaluated: 42
joined: 32
unquoted into code
twice 5 is 10
twice 9 is 18

The last row is what makes a macro safe to use twice. TIMES_TWO needs a temporary, and a fixed name would be declared twice the second time the macro is used. __FB_UNIQUEID__( tmp ) names a fresh one for each expansion.

Conditional compilation

#if, #ifdef, #elseif, #else and #endif choose which lines are compiled at all. A line in a branch not taken is never compiled, so it can call things that do not exist on that platform. platform.ti, above, is the usual use: one source, a different constant on each platform. The platform symbols tiko defines:

You define your own on the command line: -d NAME=value is #define NAME value before the first line of every file.

'' TRACE is 0 to turn tracing off -- or so the author thought.
#define TRACE 0

#ifdef TRACE
print "#ifdef TRACE: tracing is on"
#endif

#if TRACE
print "#if TRACE: tracing is on"
#else
print "#if TRACE: tracing is off"
#endif

#ifdef TRACE: tracing is on
#if TRACE: tracing is off

Compile-time checks

The preprocessor can also ask questions about the program, and stop the build when the answer is wrong:

'' Questions answered while compiling, not while running.
dim as int32 count = 0
dim as int64 total = 0

#if defined( __FB_64BIT__ ) andalso defined( __TIKO_VERSION__ )
print "a 64-bit build, by tiko"
#endif

#if typeof( count ) = typeof( int32 )
print "count is an int32"
#endif
#if typeof( total ) <> typeof( count )
print "total is not the same type as count"
#endif

'' The build stops here if the record's layout ever changes.
type Header
    magic as uint32
    size as uint32
end type
#assert sizeof( Header ) = 8
print "Header is"; sizeof( Header ); " bytes"

a 64-bit build, by tiko
count is an int32
total is not the same type as count
Header is 8 bytes

• defined( NAME ) is true when NAME has been declared — as a macro, or as anything else above the line — and it combines with andalso, orelse and not.
• #if typeof( a ) = typeof( b ) compares two types exactly: int32 is not int64, and a pointer is not what it points to.
• #assert stops the build when its condition is false. Header is a record written to files; if a field is ever added, the build stops at the #assert, before a file in the old layout is misread.

#error stops the build with a message of your own:

'' This program needs a configuration chosen on the command line.
#ifndef CONFIG
    #error "build with -d CONFIG=1 (development) or 2 (release)"
#else
    print "configuration"; CONFIG
#endif

unsupported.tiko(3,5): error TKO0309: #error: "build with -d CONFIG=1 (development) or 2 (release)"
     |
   3 |     #error "build with -d CONFIG=1 (development) or 2 (release)"
     |     ^

Built with -d CONFIG=1, the same file prints configuration 1. The print sits in the #else branch for a reason: #error reports, and the compiler carries on reading the rest of the file for more errors, so a print that used CONFIG after the #endif would add a second error, about a name that is not declared. Keep the code that needs the setting in the branch that has it.

#print is the gentle form. #print the build is starting reports its text as a note, TKO0315, and the build goes on.

Source information and pragmas

The compiler knows where every line came from, and tells you:

sub Report( )
    print "in "; __FUNCTION__; ", line"; __LINE__
end sub

print "line:"; __LINE__
Report( )
print "an ISO date has 10 characters:"; len( __DATE_ISO__ )
#line 1000 "generated.tiko"
print "now:"; __LINE__; " of "; __FILE__
#ifdef __TIKO_VERSION__
print "compiled by tiko"
#endif

line: 5
in REPORT, line 2
an ISO date has 10 characters: 10
now: 1001 of generated.tiko
compiled by tiko

• __LINE__ is the line it is written on, and __FILE__ the file's name.
• __FUNCTION__ is the enclosing procedure's name, in upper case.
• __DATE_ISO__ is the date of the build as yyyy-mm-dd, which is why the listing prints only its length: the date changes every day, and an expected output cannot.
• #line 1000 "generated.tiko" tells the compiler that the next line is line 1000 of generated.tiko. A program that writes tiko source for another program uses it, so errors point at the file a person edits rather than the generated one.

Diagnostics use the same information, so after a #line they point where it says.

Pragmas are instructions to the compiler. You have met #pragma once and #pragma push( private, true ). #pragma reserve keeps a name from being declared at all, which lets a library claim a name now and fill it in later:

'' Keep a name free for a later version of the library.
#pragma reserve Checksum

function Checksum( byval s as ustring ) as uint32
    return len( s )
end function

reserved.tiko(4,1): error TKO0769: 'checksum' is reserved by '#pragma reserve' and cannot be declared
     |
   4 | function Checksum( byval s as ustring ) as uint32
     | ^

#lang "fb" names the language a file is written in. It is the only one tiko accepts, so you never need to write it; any other is TKO0317.

What it costs

• A header is compiled again by every file that includes it. That is nothing for a few declarations and a great deal for a large C binding included by fifty files. Modules exist to pay that cost once (Chapter 15).
• Linking is where separately compiled files meet, so a clash or a missing definition is found last, after every file has compiled, and reported by the linker in the linker's terms.
• A macro has no type and no scope. It is checked only after it is expanded, so a mistake in a macro is reported at every use, in terms of the expanded text.
• A branch of conditional compilation is compiled only on the platform that takes it. Code for Linux that is never built on Linux is code nobody has compiled. Build every target you claim to support.
• Namespaces cost nothing when the program runs. A namespace changes the name the linker sees, not the code.

Summary

• A program can be several files, each compiled on its own and joined by the linker. A header (.ti) declares what the files share: declare for procedures, extern for variables. Exactly one file defines each.
• #include once and #pragma once make a header safe to reach more than once. A header is looked for first beside the file that includes it, then in the working directory, then in each -i directory in turn, then in tiko's inc/.
• Procedures are public by default. private keeps one in its file, and #pragma push( private, true ) … #pragma pop( private ) makes private the default for a region.
• dim shared in one file with extern in the header shares a variable; common shared makes every declaring file share one, unchecked.
• constructor and destructor subs run before and after the main program, lowest priority first and last respectively. Other files' top-level code runs during startup; -m chooses the main module.
• A namespace qualifies names. It nests and can be reopened; using makes its members available to the end of the enclosing block, but any closer declaration of the name wins, and two usings supplying the same name is TKO0919.
• Macros substitute text: parenthesise parameters and bodies, use # to quote, ## to paste, and #macro for whole statements. Prefer a function unless the macro generates code.
• #if tests a value and #ifdef tests a definition. Platform symbols include __FB_WIN32__, __FB_LINUX__, __FB_64BIT__ and __FB_MSVC__; -d defines your own.
• defined( ), #if typeof, #assert and #error check the program while it compiles. #error reports and the compiler reads on.
• __FILE__, __LINE__, __FUNCTION__ and __DATE_ISO__ describe the source; #line renumbers it. #pragma reserve claims a name.

Exercises

1. Write CUBE( x ) so that CUBE( 1 + 1 ) is 8 and 100 \ CUBE( 2 ) is 12. Then call it with a function that prints each time it runs. How many times does it run, and why? {solution="ex16-01"}
2. Write a #macro DEFINE_COUNTER( name ) that defines a function Next<name> returning 1, 2, 3 … on successive calls, so that DEFINE_COUNTER( Order ) gives NextOrder( ). Define two counters and show they count separately. {solution="ex16-02"}
3. Write a program that prints the name of the environment variable that holds the temporary directory: TEMP on Windows, TMPDIR on Linux. Make it refuse to build anywhere else. {solution="ex16-03"}
4. Put a function Area in a namespace Disc (one radius) and another in a namespace Rect (width and height). Call the first without its prefix from a sub, and the second qualified. {solution="ex16-04"}
5. Take clash.tiko and clashother.tiko and make them link with #pragma push( private, true ) instead of the word private. Which file did you change, and why that one?

Generics

A Stack of integers and a Stack of names are the same code twice. Before generics you had three ways to avoid writing it twice, and all of them cost something: a macro, which the compiler checks only after it has been pasted somewhere; a container of any ptr, which forgets what it holds and makes you remember; or copy and paste, which works until the two copies drift.

A generic is the fourth way. You write the code once with a placeholder for the type, and the compiler produces a real, separately checked version for each type you ask for. The code inside a generic is ordinary tiko — the same parser, the same type checking, the same errors — and what comes out is as fast and as strongly typed as the version you would have written by hand.

A type with a parameter

of T after the type's name declares a type parameter, and T is then usable anywhere a type can appear:

'' box.tiko -- a generic type, and one type per argument list.
type Box( of T )
    as T value
end type

dim b as Box( of int32 )
dim s as Box( of ustring )
dim n as Box( of Box( of int32 ) )    '' nesting is fine

b.value = 42
s.value = "hello"
n.value.value = 7

print b.value
print s.value
print n.value.value

 42
hello
 7

Box( of int32 ) and Box( of ustring ) are two distinct types, as unrelated as any two types you wrote by hand. One will not assign to the other and one will not pass where the other is expected. Nesting is ordinary too: Box( of Box( of int32 ) ) is a box holding a box.

Each distinct argument list produces one instantiation, and writing Box( of int32 ) in two different files produces the same type in both — they link.

A procedure with a parameter

Procedures take type parameters the same way, and you usually do not have to state them:

'' minmax.tiko -- a generic procedure, inferred and given explicitly.
function Larger( of T )( byval a as T, byval b as T ) as T
    if a > b then return a
    return b
end function

print Larger( 3, 9 )                    '' T inferred from the arguments
print Larger( of float64 )( 1.5, 2.5 )  '' or stated
print Larger( "apple", "pear" )

 9
 2.5
pear

The first call says nothing about T; the compiler infers int32 from the arguments. The second states it, Larger( of float64 )( … ), which you need when the arguments would infer something else — here, two literals that would otherwise have been integers.

Inference reads the arguments, and nothing else. It cannot work backwards from what you do with the result, so a generic whose type parameter appears only in the return type must always be given its argument explicitly.

Members, and bodies written out of line

A generic type can declare the full range of members — methods, properties, constructors, operators — and their bodies can live away from the declaration, which is how you keep a long type readable. The body repeats the ( of T ):

'' stack.tiko -- a generic type with members, defined out of line.
type Stack( of T )
    public:
        declare sub Push( byval item as T )
        declare function Pop( ) as T
        declare function IsEmpty( ) as boolean
        declare property Count( ) as int32
    private:
        as T items( any )
        as int32 count_
end type

sub Stack( of T ).Push( byval item as T )
    if this.count_ > ubound( this.items ) then
        redim preserve this.items( this.count_ * 2 + 3 )
    end if
    this.items( this.count_ ) = item
    this.count_ += 1
end sub

function Stack( of T ).Pop( ) as T
    this.count_ -= 1
    return this.items( this.count_ )
end function

function Stack( of T ).IsEmpty( ) as boolean
    return this.count_ = 0
end function

property Stack( of T ).Count( ) as int32
    return this.count_
end property

dim words as Stack( of ustring )
words.Push( "first" )
words.Push( "second" )
print "count: " & words.Count
print words.Pop( )
print words.Pop( )
print "empty: " & words.IsEmpty( )

count: 2
second
first
empty: true

Two details in that listing are worth naming. redim preserve this.items( … ) grows the storage, so T must be a type an array can hold — which every type is. And declaration order does not matter: the member bodies come after the type, and a generic may be used before its bodies are written, because the body is not compiled until something asks for an instantiation.

More than one parameter is the obvious extension:

'' pair.tiko -- two type parameters, and an out-of-line member.
type Pair( of K, V )
    declare function Describe( ) as ustring
    as K key
    as V value
end type

function Pair( of K, V ).Describe( ) as ustring
    return "(" & this.key & ", " & this.value & ")"
end function

dim score as Pair( of ustring, int32 )
score.key = "gold"
score.value = 3
print score.Describe( )

dim ratio as Pair( of ustring, float64 )
ratio.key = "pi"
ratio.value = 3.14
print ratio.Describe( )

(gold, 3)
(pi, 3.14)

Inheritance and virtual methods

An instantiation is an ordinary type, so it can extend another type, and it can override virtual methods. Dispatch through a base pointer reaches the instantiation's override:

type Base_ extends Object
    declare virtual function Name( ) as ustring
end type
function Base_.Name( ) as ustring
    return "base"
end function

type Holder( of T ) extends Base_
    declare virtual function Name( ) as ustring
    as T v
end type
function Holder( of T ).Name( ) as ustring
    return "holder"
end function

dim h as Holder( of int32 )
dim p as Base_ ptr = @h
print p->Name( )          '' virtual dispatch into an instantiation

holder

This is what makes a generic usable inside an object-oriented design rather than beside one: a Holder( of int32 ) is a Base_, and code that knows only about Base_ can hold one.

A type parameter promises nothing

This is the part of generics that surprises people coming from languages with constraints, and it shapes how you design with them.

of T does not say what T can do. It is a blank. The compiler therefore cannot check the body of a generic when it reads it — it has no idea whether T has a > operator — and instead checks the body once per instantiation, against the real type. A demand the type cannot meet is an error at that point:

'' unconstrained.tiko -- a type parameter promises nothing, so the
'' body's demands are checked only once a type goes in. Fails to build.
type Ranked( of T )
    declare function Best( byval a as T, byval b as T ) as T
    as T dummy
end type

function Ranked( of T ).Best( byval a as T, byval b as T ) as T
    if a > b then return a        '' needs '>' on T
    return b
end function

type Opaque_
    as int32 x
end type

dim r as Ranked( of Opaque_ )     '' Opaque_ has no '>'

That program does not build:

unconstrained.tiko(9,10): error TKO0720: no operator > compares this type
     |
   9 |     if a > b then return a        '' needs '>' on T
     |          ^
unconstrained.tiko(17,10): note: in instantiation of 'ranked( of opaque_ )'
     |
  17 | dim r as Ranked( of Opaque_ )     '' Opaque_ has no '>'
     |          ^

The upside of the same rule is that a generic costs nothing to declare. Stack( of T ) works for every type that can be copied, and you never had to enumerate what those are.

Making your own type fit a generic's demands

The standard containers state their demands in their documentation rather than in the language. Map( of K, V ) needs two things of a key: that K = K compiles, and that the key can hash itself. You supply both:

#import "std/containers"
using containers

'' The hash contract: extend Hashable and override its HashOf( ).
type Point_ extends Hashable
    as int32 x, y
    declare constructor( byval x as int32 = 0, byval y as int32 = 0 )
    declare function HashOf( ) as ulongint override
end type

constructor Point_( byval x as int32, byval y as int32 )
    this.x = x : this.y = y
end constructor

function Point_.HashOf( ) as ulongint
    return containers.HashOf( x ) * 31 + containers.HashOf( y )
end function

operator = ( byref a as Point_, byref b as Point_ ) as boolean
    return ( a.x = b.x ) andalso ( a.y = b.y )
end operator

dim seen as Map( of Point_, ustring )
dim as Point_ a = Point_( 1, 2 ), b = Point_( 1, 2 )
seen[ a ] = "first"
print "keys: " & seen.Count & "   lookup: " & seen[ b ]

keys: 1   lookup: first

The key extends Hashable and overrides its HashOf( ), because the container's code lives in the std/containers module: a free HashOf overload of your own would be invisible to it, and a module's namespace cannot be re-opened to add one. The built-in types are hashable already, and containers.HashOf( x ) is how the override hashes its fields.

Filling a container from a list

A container starts empty, and filling it used to take one statement per element. A brace list says the same thing at the declaration, the way it does for an array:

'' fill.tiko -- a brace list fills a container: one call per element,
'' in order, after the container is constructed.
#import "std/containers"

'' Your own generic takes part by having an Add, a PushBack or a Push.
type Bag( of T )
    declare sub Add( byval item as T )
    as T items( any )
end type

sub Bag( of T ).Add( byval item as T )
    redim preserve this.items( ubound( this.items ) + 1 )
    this.items( ubound( this.items ) ) = item
end sub

dim primes as containers.Array( of int32 ) = { 2, 3, 5, 7 }
dim queue as containers.LinkedList( of ustring ) = { "a", "b", "c" }
dim ages as containers.Map( of ustring, int32 ) = _
    { { "Ada", 36 }, { "Grace", 85 } }
dim bag as Bag( of float64 ) = { 0.5, 1.5 }

print "primes:"; primes.Count( ); ", last"; primes[3]
for each s as ustring in queue
    print s;
next
print
print "Grace is"; ages["Grace"]
print "bag holds"; ubound( bag.items ) + 1; ", last"; bag.items( 1 )

primes: 4, last 7
abc
Grace is 85
bag holds 2, last 1.5

Each element becomes one call, in order, after the container is constructed. primes gets four calls to Push, queue three to PushBack, and ages two to Add. An element that is itself a brace list supplies the arguments of one call, which is how { "Ada", 36 } becomes ages.Add( "Ada", 36 ). They are ordinary calls, so the ordinary conversions apply: "Ada" is a string literal and arrives as the ustring key the map wants.

The demand is the same kind this chapter has been describing: a shape, not a declaration. A brace list fills a type that needs constructing and has a member called Add, PushBack or Push, looked for in that order. Nothing says "I am a collection". Bag( of T ) takes part because it has an Add, and because its dynamic array is a field that needs constructing.

A type that needs constructing but has none of the three methods is refused:

'' fillnone.tiko -- a type that needs constructing but has no Add,
'' PushBack or Push cannot take a brace list.

type Counter
    declare constructor( )
    as int32 n
end type

constructor Counter( )
    this.n = 100
end constructor

dim c as Counter = { 1, 2 }         '' refused
print c.n

fillnone.tiko(13,20): error TKO0843: 'c' cannot take an initializer list: its type has no Add, PushBack or Push method
     |
  13 | dim c as Counter = { 1, 2 }         '' refused
     |                    ^

Overloading, and when a second generic is unreachable

Two generic procedures may share a name when they take different numbers of arguments, and the call picks the right one — the usual overloading rules apply to the instantiations.

Two limits are worth knowing before you lean on it:

• A second generic with a different number of type parameters does not resolve. Declaring both Make( of T )( a as T ) and Make( of K, V )( k as K, v as V ) compiles, but the two-argument call fails with TKO0632: no overload of 'make<8>' matches these arguments.
• An unconstrained of T parameter matches everything, so an overload that differs only in parameter type is unreachable. Given Widen( of T )( a as T ) and Widen( of T )( a as T ptr ), a pointer argument still selects the first — T binds to the pointer type. Distinguish overloads by arity, or give them different names.

Instantiation has a depth limit

A generic that instantiates itself, directly or through another generic, would otherwise recurse until memory ran out. The compiler stops at 64 levels:

type Wrap( of T )
    as T v
end type
'' each instantiation asks for another, one level deeper
type Deep( of T )
    as Wrap( of Deep( of Wrap( of T ) ) ) w
end type
dim d as Deep( of int32 )
print sizeof( d )

toodeep.tiko(6,26): error TKO0829: generic instantiation of 'wrap' is too deep
(the limit is 64; -maxinstdepth raises it)

Almost always this means a mistake — a type whose definition mentions itself without a pointer or an indirection to stop the recursion. When the depth is genuinely needed, -maxinstdepth raises the ceiling.

What it costs

Each distinct argument list produces its own code, so Array( of int32 ) and Array( of ustring ) are two separate compiled copies. That is the deal generics make: you pay in binary size for code that runs with no indirection, no boxing and no run-time type checks. A handful of instantiations is nothing; a generic instantiated over forty types in a large program is worth noticing.

Compared with the alternatives, a generic is checked where an any ptr container is not, and it produces real diagnostics where a macro produces diagnostics about the expansion.

Summary

• type Box( of T ) and function Larger( of T )( … ) declare type parameters; each distinct argument list is a separate, fully checked type or procedure.
• Type arguments are inferred from a call's arguments, and stated explicitly when they are not — inference never works backwards from the return type.
• Members may be defined out of line, repeating the ( of T ). Declaration order does not matter.
• An instantiation is an ordinary type: it can extend a base and override virtual methods.
• A type parameter promises nothing. The body is checked per instantiation; the error lands in the body, and a note: in instantiation of names the instantiation that caused it.
• A container states its demands in its docs: Map's key needs =, and a HashOf( ) override from extending containers.Hashable.
• A brace list fills a container: one Add, PushBack or Push per element, after construction, and { k, v } supplies one call's arguments. Any type that needs constructing and has one of those methods takes part.
• Instantiation stops at 64 levels deep; -maxinstdepth raises it.

Exercises

1. Write Queue( of T ) — first in, first out — with Enqueue, Dequeue and a Count property, and use it with two different element types. {solution="ex17-01"}
2. Write Swap_( of T )( byref a as T, byref b as T ). Then try calling it with an int32 and a float64 in the same call, and read the error: what does it tell you about how inference decides T? {solution="ex17-02"}
3. Make a Version_ type with major and minor fields usable as a Map key, so that two separately built values of the same version find the same entry. {solution="ex17-03"}
4. Take the Ranked( of T ) listing above and make it compile by giving Opaque_ the operator it needs. Which line did you have to change — the generic, or the type you put in it?

Errors: Optional, Result and Runtime Checks

Every program meets things that go wrong: a file that is not there, a line that does not parse, a divisor that turns out to be zero. The question is never whether failure happens but where the program finds out about it, and whether the code that finds out is the code that can do something about it.

tiko gives you two kinds of answer. Values that say they might have failed — Optional and Result from the standard library — put the possibility of failure into a function's type, so the caller cannot use the answer without first deciding what to do when there is none. Runtime errors — the error statement, on error goto, and the checks -exx adds — stop the normal flow and jump to a handler. The first kind is for failures a caller is expected to handle. The second is for the ones it is not.

This chapter covers both, then the checks the compiler can add for you and the assertions you can add yourself.

A value that means "nothing"

The oldest way to report failure is a value that cannot otherwise happen:

'' The coldest reading, or -1 when there are none.
function Coldest( temps() as int32 ) as int32
    if ubound( temps ) < lbound( temps ) then return -1
    var lowest = temps( lbound( temps ) )
    for i as int32 = lbound( temps ) + 1 to ubound( temps )
        if temps( i ) < lowest then lowest = temps( i )
    next
    return lowest
end function

dim none() as int32
dim winter( 0 to 2 ) as int32 = { 3, -1, 2 }

print "no readings:"; Coldest( none() )
print "a cold day: "; Coldest( winter() )

no readings:-1
a cold day: -1

Optional

Optional( of T ) is a T that might not be there. It lives in the standard library's outcome module:

#import "std/outcome"

'' Where v is in a, if it is anywhere.
function FindIndex( a() as int32, byval v as int32 ) _
    as outcome.Optional( of int32 )
    for i as int32 = lbound( a ) to ubound( a )
        if a( i ) = v then return outcome.Some( i )
    next
    return outcome.None( of int32 )( )
end function

dim xs( 0 to 3 ) as int32 = { 4, 8, 15, 16 }

var found = FindIndex( xs(), 15 )
if found.HasValue( ) then print "15 is at"; found.Value( )

var missing = FindIndex( xs(), 99 )
print "99 is at "; str( missing.ValueOr( -1 ) ); ", which means nowhere"

dim nothing as outcome.Optional( of ustring )
print "a new Optional has a value: "; nothing.HasValue( )

15 is at 2
99 is at -1, which means nowhere
a new Optional has a value: false

FindIndex can now say "15 is at index 2" and "99 is not here" without inventing an index that means nothing. The caller has to ask: there is no way to get an int32 out of the result without choosing between HasValue, Value and ValueOr.

A few details matter in practice:

• A new Optional is empty. dim nothing as outcome.Optional( of ustring ) holds no value, and there is no separate "uninitialised" state to get wrong.
• Some infers T from its argument, and a literal is an integer. outcome.Some( 42 ) is an Optional( of integer ), which does not convert to an Optional( of int32 ). Write outcome.Some( of int32 )( 42 ) when the type has to be exact. None has no argument to infer from, so its type is always written.
• Value( ) on an empty Optional is a bug at the call site, not a recoverable condition: it raises runtime error 1, "illegal function call". Ask HasValue first, or use ValueOr. The section on runtime errors shows the error being caught.

Result

Optional says that there is no answer. Often the caller also needs to know why, and that is Result( of T, E ): a T on success, or an E that explains the failure.

#import "std/outcome"

type DigitResult as outcome.Result( of int32, ustring )

function ParseDigit( byval s as ustring ) as DigitResult
    if ( len( s ) = 1 ) andalso ( s >= "0" ) andalso ( s <= "9" ) then
        return outcome.Ok( of int32, ustring )( asc( s ) - 48 )
    end if
    var why = "'" & s & "' is not a digit"
    return outcome.Fail( of int32, ustring )( why )
end function

dim inputs( 0 to 2 ) as ustring = { "7", "x", "42" }
for each s as ustring in inputs
    var r = ParseDigit( s )
    if r.IsOk( ) then
        print s; " ->"; r.Value( )
    else
        print s; " -> "; r.Failure( )
    end if
next

dim blank as DigitResult
print "a new Result succeeded: "; blank.IsOk( )

7 -> 7
x -> 'x' is not a digit
42 -> '42' is not a digit
a new Result succeeded: false

Ok and Fail each mention only one of T and E in their parameters, so the compiler cannot work out the other: both type arguments are always written. That is why the result type is usually named once with a type … as, here DigitResult, and the long spelling appears only where a value is built.

A new Result is a failure. dim blank as DigitResult reports IsOk( ) false, and its Failure( ) is a default-constructed E. That is the safe default — code that forgets to set a Result cannot accidentally claim success — and it has a consequence for your own error types: E needs a constructor that takes no arguments, because every Result builds one.

Passing a failure on

A function that calls another fallible function usually wants to stop at the first failure and pass it up, adding what it knows:

#import "std/outcome"

type Quotient as outcome.Result( of float64, ustring )

function Divide( byval a as float64, byval b as float64 ) as Quotient
    if b = 0 then
        return outcome.Fail( of float64, ustring )( "division by zero" )
    end if
    return outcome.Ok( of float64, ustring )( a / b )
end function

'' The average of a total over a count, which can fail the same way.
function Average( byval total as float64, byval n as int32 ) as Quotient
    var q = Divide( total, n )
    if q.IsOk( ) = false then
        var why = "no average: " & q.Failure( )
        return outcome.Fail( of float64, ustring )( why )
    end if
    return q
end function

print Average( 10, 4 ).ValueOr( 0 )
var bad = Average( 10, 0 )
print bad.IsOk( ); " "; bad.Failure( )

 2.5
false no average: division by zero

Average asks IsOk( ), and on failure builds its own failure that wraps the reason it was given. On success it returns Divide's result unchanged, because it is already the right type. Each layer either handles a failure or passes it on, and either way it says so in its code.

An error type of your own

A ustring is enough to tell a person what went wrong. A program that wants to react to the failure — retry, skip the line, ask again — needs something it can branch on. The standard library's fs.FsError is the model: a code, the operation and path involved, and a message. A configuration parser built the same way:

#import "std/outcome"

'' WHY A LINE WAS REFUSED, modelled on fs.FsError: a code to branch
'' on, the line it happened on, and a message for people.
enum ConfigCode
    CFG_OK
    CFG_NO_EQUALS
    CFG_EMPTY_KEY
    CFG_NOT_A_NUMBER
end enum

type ConfigError
    code as ConfigCode
    lineNo as int32
    declare constructor( )
    declare constructor( byval c as ConfigCode, byval n as int32 )
    declare function Message( ) as ustring
end type

constructor ConfigError( )
end constructor

constructor ConfigError( byval c as ConfigCode, byval n as int32 )
    code = c : lineNo = n
end constructor

function ConfigError.Message( ) as ustring
    dim as ustring what
    select case code
    case CFG_NO_EQUALS    : what = "expected key=value"
    case CFG_EMPTY_KEY    : what = "the key is empty"
    case CFG_NOT_A_NUMBER : what = "the value is not a number"
    end select
    return "line " & str( lineNo ) & ": " & what
end function

type Setting
    key as ustring
    value as int32
end type

type SettingResult as outcome.Result( of Setting, ConfigError )

function Refuse( byval c as ConfigCode, byval n as int32 ) _
    as SettingResult
    var e = ConfigError( c, n )
    return outcome.Fail( of Setting, ConfigError )( e )
end function

function ParseLine( byval text as ustring, byval n as int32 ) _
    as SettingResult
    var eq = instr( text, "=" )
    if eq = 0 then return Refuse( CFG_NO_EQUALS, n )
    dim s as Setting
    s.key = trim( left( text, eq - 1 ) )
    if len( s.key ) = 0 then return Refuse( CFG_EMPTY_KEY, n )
    var v = trim( mid( text, eq + 1 ) )
    if ( len( v ) = 0 ) orelse ( str( valint( v ) ) <> v ) then
        return Refuse( CFG_NOT_A_NUMBER, n )
    end if
    s.value = valint( v )
    return outcome.Ok( of Setting, ConfigError )( s )
end function

'' Every line, or the first failure: the error travels up unchanged.
function ParseConfig( lines() as ustring, settings() as Setting ) _
    as outcome.Result( of int32, ConfigError )
    for i as int32 = lbound( lines ) to ubound( lines )
        var r = ParseLine( lines( i ), i + 1 )
        if r.IsOk( ) = false then
            return outcome.Fail( of int32, ConfigError )( r.Failure( ) )
        end if
        redim preserve settings( ubound( settings ) + 1 )
        settings( ubound( settings ) ) = r.Value( )
    next
    return outcome.Ok( of int32, ConfigError )( ubound( settings ) + 1 )
end function

sub Load( lines() as ustring )
    dim settings() as Setting
    var r = ParseConfig( lines(), settings() )
    if r.IsOk( ) then
        print "loaded"; r.Value( ); " settings:";
        for each s as Setting in settings
            print " "; s.key; "="; str( s.value );
        next
        print
    else
        print "refused, "; r.Failure( ).Message( )
    end if
end sub

dim good( 0 to 2 ) as ustring = _
    { "width = 80", "height=24", "tabs = 4" }
dim bad( 0 to 2 ) as ustring = _
    { "width = 80", "height=tall", "tabs = 4" }
Load( good() )
Load( bad() )

loaded 3 settings: width=80 height=24 tabs=4
refused, line 2: the value is not a number

ConfigError carries a ConfigCode for programs and a Message( ) for people, and it has the constructor with no arguments that Result needs. ParseLine fails in three distinct ways, and ParseConfig stops at the first failing line and passes its error up unchanged. The caller sees exactly which line failed and why, and nothing below it had to print anything.

The standard library's own errors work the same way. Reading a file that is not there:

#import "std/fs"

var r = fs.ReadAllText( "no-such-file.txt" )
if r.IsOk( ) = false then
    var e = r.Failure( )
    if e.Code( ) = fs.FS_NOT_FOUND then print "not found"
    print e.Operation( ); " "; e.Path( ); ": "; e.Message( )
end if

not found
ReadAllText no-such-file.txt: no such file or directory

ReadAllText returns a Result whose failure is an fs.FsError. Code( ) is what the program branches on — fs.FS_NOT_FOUND here, the same on Windows and Linux — and the text is for the person reading the output.

Statements that return an error code

Some of the runtime's oldest statements have a function form that returns an error number instead of raising a runtime error. open and close are the ones you will meet most:

dim as int32 f = freefile

'' The function form of OPEN returns an error code; it does not raise.
print "missing:"; open( "no-such-file.txt" for input as #f )
print "present:"; open( "notes.txt" for input as #f )
print "close:"; close( #f )
print "again:"; close( #f )

missing: 2
present: 0
close: 0
again: 1

0 means success. Any other value is a runtime error number: 2 is "file not found", and closing a file number that is no longer open is 1, "illegal function call".

Written as a statement — open "notes.txt" for input as #f — the same failure is silent unless the program was compiled with -e, and then it stops the program with runtime error 2. A missing file that nothing reports is the worst outcome of the three, which is the reason to use the function form whenever a missing file is an expected event rather than a bug.

Runtime errors

A runtime error is a numbered failure that stops the normal flow of the program. The runtime raises them itself — a file that cannot be opened, an Optional with no value — and your code can raise one with error n. If nothing handles it, the program prints the error and ends with that number as its exit code. To handle one, install a handler with on error goto:

#cmdline "-exx"

function Ratio( byval a as int32, byval b as int32 ) as int32
    if b = 0 then error 11            '' 11: division by zero
    return a \ b
end function

sub Report( byval a as int32, byval b as int32 )
    on local error goto failed
    dim as int32 r = Ratio( a, b )
    print a; " /"; b; " ="; r
    exit sub
failed:
    '' Copy the error out first: the next runtime call may clear it.
    dim as int32 code = err, where_ = erl
    print "error"; code; " at line"; where_; " in "; *erfn( )
end sub

Report( 12, 4 )
Report( 1, 0 )

 12 / 4 = 3
error 11 at line 4 in RATIO

on local error goto failed installs a handler for the rest of Report. When Ratio raises error 11, control jumps to failed:, where four functions describe what happened:

erfn and ermn need the error-location information -exx compiles in; without it erfn( ) is empty. And read err before doing anything else. The next call into the runtime may clear it: print "error"; err printed 0 in testing, because printing the text came first. Copying it to a variable on the handler's first line is the reliable form.

An empty Optional raises in exactly this way:

#import "std/outcome"

sub ShowValue( byval o as outcome.Optional( of int32 ) )
    on local error goto empty
    dim as int32 v = o.Value( )
    print "value:"; v
    exit sub
empty:
    dim as int32 code = err
    print "no value: runtime error"; code
end sub

ShowValue( outcome.Some( of int32 )( 42 ) )
ShowValue( outcome.None( of int32 )( ) )

value: 42
no value: runtime error 1

What the jump skips

type Connection
    handle as int32
    declare constructor( )
    declare destructor( )
end type

constructor Connection( )
    print "  connection opened"
end constructor

destructor Connection( )
    print "  connection closed"
end destructor

sub Transfer( )
    dim c as Connection
    defer print "  transfer finished"
    error 5
end sub

on error goto failed
Transfer( )
end

failed:
dim as int32 code = err
print "handler: error"; code

  connection opened
handler: error 5

Runtime checking

Some mistakes can only be caught while the program runs: an array index out of range, a null pointer followed. The compiler adds checks for them when asked, and three options control how many:

Chapters 8 and 11 showed what -exx catches. A program can ask how it was compiled through __FB_ERR__, a bit mask:

#cmdline "-exx"

'' Which runtime checks this program was compiled with.
print "__FB_ERR__ ="; __FB_ERR__
print "error checking:  "; iif( __FB_ERR__ and 1, "on", "off" )
print "array bounds:    "; iif( __FB_ERR__ and 8, "on", "off" )
print "null pointers:   "; iif( __FB_ERR__ and 16, "on", "off" )

__FB_ERR__ = 1823
error checking:  on
array bounds:    on
null pointers:   on

The values are 0 with no option, 513 with -e, 515 with -ex and 1823 with -exx. Bit 1 is error checking, 8 array bounds and 16 null pointers; the rest are documented with the compiler's options.

Each check also has an option of its own, for a build that wants one check and not the rest: -earray (8), -enullptr (16), -elocation, -eassert, -edebug, -edebuginfo, -eunwind and -earraydims, each setting only its own bit. -g sets 32, 64 and 128 — assertions, debug information and __FB_DEBUG__ — so __FB_ERR__ is 224 with -g alone and 2047 with -g -exx.

Checks cost speed. -exx adds a comparison to every array access and a test to every pointer dereference. Use it while developing and testing, and decide deliberately whether a release build keeps it.

Assertions

An assertion states something that must be true, and stops the program if it is not. tiko has two kinds.

#assert is checked by the compiler, against constants:

const SLOTS = 12

#assert SLOTS mod 4 = 0          '' holds
#assert SLOTS <= 8               '' refused

print "built"

staticassert.tiko(4,1): error TKO0604: #assert condition failed
     |
   4 | #assert SLOTS <= 8               '' refused
     | ^

It costs nothing at run time and catches a configuration mistake before the program exists.

assert is checked while the program runs, and only in a build with -g. Without -g it compiles to nothing at all, so it can stay in code that ships:

#cmdline "-g"

function Average( byval total as int32, byval n as int32 ) as int32
    assert( n > 0 )
    return total \ n
end function

dim as int32 a = Average( 10, 0 )

assertfail.tiko(4): assertion failed at AVERAGE: n > 0

The failure names the file, the line, the procedure and the condition as written, and the program exits with status 1. assertwarn does the same and carries on.

An assertion is for a mistake in the program — a caller passing 0 where the documentation says "never 0". A condition the program should expect, like a missing file, is a Result.

What it costs

• An Optional( of T ) is a T plus a flag, and the T is default-constructed even when there is no value. A Result( of T, E ) holds both a T and an E. For small types this is nothing; for a large one, return it through a parameter or a pointer instead.
• Checking a Result is a branch at every call. That is the point: the cost is paid where the decision is made.
• on error goto costs nothing until an error happens, and then skips destructors on its way out of a procedure, which is a cost of a different kind.
• -exx slows every array access and pointer dereference. assert costs nothing without -g, and a comparison with it.

Summary

• A sentinel return value is ambiguous whenever it can also be a real answer.
• Optional( of T ) holds a value or nothing: HasValue, Value (raises when empty), ValueOr. Some( v ) and None( of T )( ) build one; a new one is empty.
• Result( of T, E ) holds a value or a reason: IsOk, Value, Failure, ValueOr. Ok and Fail need both type arguments, so name the type with type … as. A new Result is a failure, and E needs a default constructor.
• Pass failures up by returning them; give the error a code to branch on, as fs.FsError does.
• open( … ) and close( … ) used as functions return an error number instead of raising.
• error n raises a runtime error; on [local] error goto handles it; err, erl, erfn, ermn describe it. Copy err first.
• An error that leaves a procedure for an outer handler skips that procedure's destructors and defers. tiko has no resume.
• -e, -ex and -exx add runtime checks; __FB_ERR__ says which were used.
• #assert checks constants at compile time (TKO0604); assert checks at run time under -g.

Exercises

1. Write ToInt( s ) returning an Optional( of int32 ) that accepts digits with an optional leading minus and refuses anything else. Use it to total the valid entries of a list and count the refused ones. {solution="ex18-01"}
2. Write Divide and Root, each returning a Result( of float64, ustring ), and a RootOfRatio( a, b ) that stops at the first failure and passes it on. Run it over a success and one failure of each kind. {solution="ex18-02"}
3. Give ConfigError a CFG_DUPLICATE code, and make ParseConfig refuse a key that appears twice. Which function has to know about the other lines, and why?
4. Move the handler in escape.tiko into Transfer. What changes in the output, and why?
5. Compile runtimeerrors.tiko with -e instead of -exx. Which part of the handler's output changes?

Testing, Debugging and Tooling

A program that compiles is a program the compiler could read. Whether it does what you meant is a separate question, and the sooner you find out the cheaper the answer is. A failed test costs a minute. A wrong number found in a report a month later costs the month.

This chapter is about the tools that shorten that distance. Tests check behaviour you have decided on, every time you build. Tracing and a debugger show you what a program did when a test, or a user, finds it doing something else. And the compiler's own options — its warnings, its diagnostics file and its ways of showing you the code it generated — are tools too.

Unit tests with fbcunit

tiko ships a unit-test library, fbcunit. A test is a block of code that checks something with an assertion; tests are grouped into suites; one call runs them all and prints a summary. No module fronts the library, so it is reached with #include:

'' A stack of int32, and the tests that pin its behaviour down.
#include once "fbcunit.bi"

type Stack
    declare sub Push( byval v as int32 )
    declare function Pop( ) as int32
    declare function Count( ) as int32
private:
    dim items( any ) as int32
end type

sub Stack.Push( byval v as int32 )
    redim preserve items( ubound( items ) + 1 )
    items( ubound( items ) ) = v
end sub

function Stack.Pop( ) as int32
    var top = items( ubound( items ) )
    if ubound( items ) = 0 then
        erase items
    else
        redim preserve items( ubound( items ) - 1 )
    end if
    return top
end function

function Stack.Count( ) as int32
    return ubound( items ) + 1
end function

SUITE( lifo )
    TEST( starts_empty )
        dim s as Stack
        CU_ASSERT_EQUAL( s.Count( ), 0 )
    END_TEST

    TEST( pops_in_reverse )
        dim s as Stack
        s.Push( 1 ) : s.Push( 2 ) : s.Push( 3 )
        CU_ASSERT_EQUAL( s.Pop( ), 3 )
        CU_ASSERT_EQUAL( s.Pop( ), 2 )
        CU_ASSERT_EQUAL( s.Count( ), 1 )
    END_TEST

    TEST( counts_pushes )
        dim s as Stack
        s.Push( 7 )
        CU_ASSERT_EQUAL( s.Count( ), 2 )   '' wrong on purpose
    END_TEST
END_SUITE

end iif( fbcu.run_tests( ), 0, 1 )

      firsttests.tiko(48) : error : TESTS.LIFO.COUNTS_PUSHES CU_ASSERT_EQUAL(s.Count( ),2)

SUMMARY

 Asserts    Passed    Failed  Suite                                      Tests
--------  --------  --------  --------------------------------------  --------
       5         4         1  lifo                                           3
--------  --------  --------  --------------------------------------  --------
       5         4         1  Total                                          3

SUITE( lifo ) … END_SUITE groups the tests; each TEST( name ) … END_TEST is one test with a fresh Stack of its own. fbcu.run_tests( ) runs every test, prints each failed assertion with its file, line and the assertion as written, then the summary, and returns true only if everything passed. The last line turns that into the program's exit code — 1 here, because the third test is wrong on purpose — which is what a build script or a CI job checks.

The assertions you will use most:

A failed assertion does not stop its test: the test carries on, and every failed assertion is listed before the summary. That is why one TEST can hold several related checks, as pops_in_reverse does.

Tests in a multi-module project

In a real project the code under test lives in a module, and the tests are a separate program that imports it. Here is the stack as Chapter 15's kind of module, stacks/stacks.tiko:

#module "stacks"

'' A last-in, first-out stack of int32.
public type Stack
    declare sub Push( byval v as int32 )
    declare function Pop( ) as int32
    declare function Peek( ) as int32
    declare function Count( ) as int32
private:
    dim items( any ) as int32
end type

sub Stack.Push( byval v as int32 )
    redim preserve items( ubound( items ) + 1 )
    items( ubound( items ) ) = v
end sub

function Stack.Pop( ) as int32
    var top = items( ubound( items ) )
    if ubound( items ) = 0 then
        erase items
    else
        redim preserve items( ubound( items ) - 1 )
    end if
    return top
end function

function Stack.Peek( ) as int32
    return items( ubound( items ) )
end function

function Stack.Count( ) as int32
    return ubound( items ) + 1
end function

and its tests:

'' The stacks module's tests: a program of their own.
#import "./stacks"
#include once "fbcunit.bi"

SUITE( lifo )
    TEST( starts_empty )
        dim s as stacks.Stack
        CU_ASSERT_EQUAL( s.Count( ), 0 )
    END_TEST

    TEST( pops_in_reverse )
        dim s as stacks.Stack
        s.Push( 1 ) : s.Push( 2 ) : s.Push( 3 )
        CU_ASSERT_EQUAL( s.Pop( ), 3 )
        CU_ASSERT_EQUAL( s.Pop( ), 2 )
        CU_ASSERT_EQUAL( s.Count( ), 1 )
    END_TEST
END_SUITE

SUITE( peeking )
    TEST( leaves_the_top )
        dim s as stacks.Stack
        s.Push( 7 )
        CU_ASSERT_EQUAL( s.Peek( ), 7 )
        CU_ASSERT_EQUAL( s.Count( ), 1 )
    END_TEST
END_SUITE

end iif( fbcu.run_tests( ), 0, 1 )

SUMMARY

 Asserts    Passed    Failed  Suite                                      Tests
--------  --------  --------  --------------------------------------  --------
       4         4         0  lifo                                           2
       2         2         0  peeking                                        1
--------  --------  --------  --------------------------------------  --------
       6         6         0  Total                                          3

The test program sees the module exactly as any other client does: stacks.Stack and its public members, and nothing private. That is a feature. Tests written against the public surface keep passing while you rearrange the inside, and they fail when you change what callers depend on — which is the change you wanted to hear about.

SUMMARY

 Asserts    Passed    Failed  Suite                                      Tests
--------  --------  --------  --------------------------------------  --------
       2         2         0  peeking                                        1
       4         4         0  lifo                                           2
--------  --------  --------  --------------------------------------  --------
       6         6         0  Total                                          3

Tracing

Sometimes the quickest way to see what a program did is to have it tell you. Three predefined names make that useful: __FILE__ is the source file being compiled, __LINE__ the line, and __FUNCTION__ the procedure — in capitals, and __FB_MAINPROC__ at module level. Wrapped in a macro, they report where the macro was used, not where it was defined:

'' trace.ti -- TRACE( msg ) reports where it was called from, on
'' stderr. Without TRACING defined it compiles to nothing.
#pragma once

#ifdef TRACING
    #define TRACE( msg ) _
        TraceAt( __FILE__, __LINE__, __FUNCTION__, msg )
#else
    #define TRACE( msg )
#endif

sub TraceAt( byval file as ustring, byval lineNo as int32, _
             byval proc as ustring, byval msg as ustring )
    var f = freefile( )
    open err for output as #f
    print #f, file & "(" & lineNo & ") " & proc & ": " & msg
    close #f
end sub

#define TRACING
#include once "trace.ti"

function Mean( values( ) as float64 ) as float64
    var n = ubound( values ) - lbound( values ) + 1
    TRACE( "count " & n )
    dim total as float64
    for i as int32 = lbound( values ) to ubound( values )
        total += values( i )
    next
    TRACE( "total " & total )
    return total / n
end function

dim readings( 0 to 2 ) as float64 = { 3.5, 4.0, 5.5 }
var m = Mean( readings( ) )
TRACE( "mean " & m )

tracing.tiko(6) MEAN: count 3
tracing.tiko(11) MEAN: total 13
tracing.tiko(17) __FB_MAINPROC__: mean 4.333333333333333

Two decisions in trace.ti are worth copying:

• Trace output goes to stderr. open err opens the standard error stream as a file number. The program's real output stays on stdout, so program > result.txt still captures exactly the result, and the trace stays on the screen or goes to a file of its own with 2> trace.txt.
• Tracing is switched off by the preprocessor, not by an if. Remove the #define TRACING line and TRACE( … ) expands to nothing: the calls are gone from the generated code, and so is the cost of building their messages. The TraceAt procedure itself is still compiled; it is never called.

Debugging with -g

Tracing answers questions you thought to ask before you ran the program. A debugger answers the ones you think of while it is stopped. Compile with -g and tiko writes DWARF debug information into the executable: line tables, procedures, and every variable with its type. Any DWARF debugger can then stop the program and show you its state. The one that comes with the LLVM install tiko already uses is lldb.

'' A small program to stop inside: build it with -g.
type Reading
    sensor as ustring
    value as float64
end type

function Total( rs( ) as Reading ) as float64
    dim sum as float64
    for i as int32 = lbound( rs ) to ubound( rs )
        sum += rs( i ).value
    next
    return sum
end function

dim rs( 1 to 3 ) as Reading
rs( 1 ).sensor = "north" : rs( 1 ).value = 3.5
rs( 2 ).sensor = "south" : rs( 2 ).value = 4.0
rs( 3 ).sensor = "east"  : rs( 3 ).value = 5.5
dim label as ustring = "total"
print label; Total( rs( ) )

total 13

Build it with tiko -g readings.tiko, then run it under lldb, stopping at line 10:

(lldb) breakpoint set -f readings.tiko -l 10
(lldb) run
* thread #1, stop reason = breakpoint 1.1
    frame #0: readings.exe`Total(rs=0x0000005c4975f7e0) at readings.tiko:10
   9   	    for i as int32 = lbound( rs ) to ubound( rs )
-> 10  	        sum += rs( i ).value
   11  	    next
(lldb) frame variable sum i
(double) sum = 0
(long) i = 1
(lldb) up
frame #1: readings.exe`main at readings.tiko:20
-> 20  	print label; Total( rs( ) )
(lldb) frame variable label
(ustring) label = (data = u"total", len = 5, size = 16)
(lldb) frame variable rs
(Reading(1 to 3)) rs = {
  [0] = {
    sensor = (data = u"north", len = 5, size = 18)
    value = 3.5
  }
  [1] = {
    sensor = (data = u"south", len = 5, size = 18)
    value = 4
  }
  [2] = {
    sensor = (data = u"east", len = 4, size = 18)
    value = 5.5
  }
}

The breakpoint is set by file and line, the stop shows the source around it, and up moves to the caller. Variables keep their types: a record shows its fields, a ustring its text and length, and the array its declared bounds, Reading(1 to 3). lldb spells some types the way C would — an int32 is a long and a float64 a double — because that is the vocabulary it has.

Two kinds of value read less well. An array parameter, such as rs inside Total, is passed as a descriptor, and lldb shows the descriptor's fields — the data pointer, the element size, and eight dimension slots of which only the first dimensions mean anything. And a ustring shows its bookkeeping beside its text.

(lldb) p rs[1]
(Reading) {
  sensor = (data = u"south", len = 5, size = 18)
  value = 4
}

The same session needs no commands in tkEdit (Chapter 2). Put the cursor on line 10, press F9 for a breakpoint and F6 to build with -g and run. The program stops in Total, and the Locals tab of the output pane lists rs, sum and i. There rs is (1 to 3) and opens to rs(1), rs(2) and rs(3), each element named by the subscript the program uses: tkEdit reads the array descriptor, so the zero-based pitfall above does not apply. A string shows its text, u"north", without the bookkeeping, and a watch takes the program's spelling, rs(2).sensor.

Warnings and the diagnostics workflow

An error stops the build. A warning does not: the program is built, and it runs. That is exactly what makes warnings worth your attention.