Level 1: What you already know, just Scoped

In this section we will cover the concepts that are basically the same between a scripting language like Python and Scopes.

Printing Output

What language is complete without a print statement!

print "Hello World!"

The print statement takes a variable number of arguments, just like in Python:

# print with 2 arguments
print "Hello" "World!"

Here the basic string type is used. We will describe strings in more detail later.

We also show the use of comments in the last example

Assignment, Binding, Values & Variables

In Scopes there is an explicit distinction between variables and values. Variables are names bound to a place or slot of explicit memory that can then be mutated later, whereas values are immutable once created and cannot be modified in place.

Because Scopes is a semi-functional Scheme-like language values are more common than variables, so we describe them first in terms of the let syntax:

let name = "Bob"

Simple enough. As we will see once we learn about expressions though that the right hand side can be much more complex than in most languages.

To assign the result of an expression you just need to syntactically scope it:

let sum = (1 + 2 + 3)

There is an infix version of the let-assignment as well:

sum := 1 + 2 + 3

For one we can do multiple assignments in the same let block:

let
    name = "Bob"
    password = "password"
    height = 6

You can also do a multiple assignment like so:

let a b c = 1 2 3
print "a:" a "b:" b "c:" c

Note that if you reassign a value to the same name this is not actually mutating the first value.

let count = 0
let count = (count + 1)

The storage location (stack, heap, and data segment) of the value generated from an expression is determined by the kind of expression.

You can explicitly choose stack or data-segment by using the local or global mechanisms.

For those unaware the “stack” is a data structure used in most programming languages that provides a fast and convenient way to store data in nested scopes that can easily be managed and torn down without manual memory managemeng. For example when a function goes out of scope all of the locally defined memory values can be freed.

The heap is all the memory that is not in the stack and can essentially be accessed in any scope as long as the layout of data is known. This memory must be manually managed since it lacks a mechanism like scoping rules to know when data is no longer needed.

local count = 0
print count
count = count + 1
print count

Note that the assignment for already instantiated variables is infix.

In scopes there is also another location values can stored in called the data segment which is an actual location in the output binary. Therefore it is not located on either the stack or the heap. However, because of this you must be careful not to put too large of data because your final executable binary will have to carry it around with it.

Global values can be defined as such.

NOTE: Currently there is a bug in scopes that causes the print command to show some gobbledy-gook like PureCast$halafakes instead of the normal representation of the value. The following snippet shows how to rectify this with the deref function which won’t be explained until Level 2.

global count = 0
count = count + 1
print count
print (deref count)

You can unbind names by using unlet:

let
    a = 3
    b = 4

unlet a
# print a
print b

—

There is also the bind syntax you can use to bind symbols. It is primarily used in combination with other macros which we will get to much later but it is simple enough on its own to mention here:

(bind a) 3

print a

Operators

In most Lisp/Schemes there are no “operators” in the sense of infix notations (i.e. arg1 operator arg2) and only a uniform prefix notation (i.e. operator arg1 arg2). Thus unifying operators and functions.

For many things an infix operator is preferable because it mirrors the common notations such as in mathematics.

In Scopes you can use both! And you can also overload operators although we won’t talk about that until Level 2.

We will take the + and * operators to demonstrate the point (arithmetic and numbers described in detail below, but nothing here should surprise you).

We start with an understandable arithmetic expression which is also valid Scopes code. There is an order of operations but we write this unambiguously for now:

(0 * 3) + (1 * (4 * 5))

The equivalent prefix code is:

(+ (* 0 3) (* 1 (* 4 5)))

One advantage of prefix code is that for infix expressions like:

1 + 2 + 3 + 4

You can write them more simply in a summation type notation:

(+ 1 2 3 4)

Functions in prefix notation are also more flexible when you get to higher order functional programming.

Primitives & Simple Expressions

Overview

Overview:

Type	Type Symbols	Example
Boolean	`bool`	`true`, `false`
Integer	`i32` (default), `i8`, `i16`, `i64`	`1`, `-4`, `+7`, `3:i8`, `0x20`, `0b01101001:i8`
Unsigned Integer	`u8`, `u16`, `u32`, `u64`	`3:u64`
Floating Point Number	`f32` (default), `f64` (double precision)	`1.`, `1.0`, `3.456:f64`, `nan`, `inf`, `1e12`
Empty Signifier	`Nothing`	`none`
Null Pointer	`NullType`	`null`
Fixed-lengthString	`string`	`"hello"`
Lists	`List`	`'()`, `'("a" true 1)`, `'[0 1]`, `{3; 1 }`

Note that you can determine the type of a value with the typeof function:

typeof true
typeof 1
typeof 1.0
typeof none
typeof null

let hello_type = (typeof "hello")
print "string type:" hello_type

typeof 'something
typeof '()

Here we can see use of the 3 different primitive syntax units in SLN notation and how they map to the primitive values in the Scopes language.

typeof, true, null, none, and 'something are all symbols
1 and 1.0 are numbers
"hello" is a string
'() is a list

Special Values

There are a few special values defined in the core language:

none: Which signifies emptiness and uses no storage.
null: Which is similar to a null-pointer and is used for pointer comparisons.

Some interesting identities:

print (typeof none)
print (typeof null)

let a = ()
print (typeof a)

Booleans

The two primitive boolean values are given by the two symbols true and false.

We have simple boolean logical operators in and, or, and not:

true and false
true or true
not true

Similar to Python other values can be interpreted implicitly as booleans, however unlike in Python not all of them can do this.

not 0
not 1

Note that we also have the bitwise versions of these:

# bitwise and
true & false

# bitwise or
true | false

How these are used differently than and and or will be explained in higher levels.

Primitive Strings

Strings are anything surrounded by double-quotes ". As we have already seen:

print "a string"
typeof "stringzz"

Multiline strings can be given with quadruple-double-quotes and then continued using indentation adjusted up to the column after the quadruple-double-quotes:

""""a multiline string
    That is continued here
    Thats 4 (four) double-"quotes"

Note you don’t need to escape the double-quotes in the block since multiline blocks are considered “raw”, unlike single line strings where double-quotes need to be escaped:

print "The man said \"hello\""

Note that unlike languages like Python single quotes (') cannot be used for delimiting strings like double-quotes (").

let multiline = """"a multiline string
                    That is continued here
                    Thats 4 (four) double-quotes

print multiline

For instance this will raise an error:

let multiline = """"a multiline string
    That is continued here
    Thats 4 (four) double-quotes

To join strings together you can use the .. operator:

let header = ("Hello" .. " There:")

print (header .. " Bob")

Or like the `+=` etc. you can concatenate and assign in a single statement:

local msg = ""

msg ..= "Dear Scopes,\n"
msg ..= "    Your charm is irresistable!\n\n"
msg ..= "    Love,\n"
msg ..= "    Rust"

print msg

You can get the number of characters in a string with the countof function:

let alphabet = "abcdefghijklmnopqrstuvwxyz"

print (countof alphabet)

You can retrieve a particular character like this:

let alphabet = "abcdefghijklmnopqrstuvwxyz"

alphabet @ 4
(@ alphabet 4)

There are also some slice routines:

slice: Get characters from start to end
lslice: Get characters to the left of an index
rslice: Get characters to the right of an index

let alphabet = "abcdefghijklmnopqrstuvwxyz"

print (slice alphabet 0 3)
print (lslice alphabet 3)

print (slice alphabet 3 (countof alphabet))
print (rslice alphabet 3)

One final note is that being a low-level language we will have much more to talk about with regards to strings in Level 2 for concerns regarding interop with C and memory allocation etc.

Integers & Unsigned Integers

While integers are familiar to Python programmers the family of different types around them is unfamiliar. This is because Python provides an idealized view of what an integer is. In lower level languages like C/C++ and Scopes the underlying byte representation of values is a first class concept, in order to be able to tightly control memory usage for performant applications.

Additionally there is the concept of a signed and unsigned integer. Using an unsigned integer frees up a bit that would normall be taken up with information of the sign (i.e. positive or negative).

Signed integers are useful for arithmetic and numerical calculations and unsigned integers are useful as indices and other identifiers that you wouldn’t perform arithmetic on.

Signed integers have type symbols of the form i<num_bits> and unsigned integers of the form u<num_bits>.

Where num_bits can be: 8, 16, 32, or 64.

For visual completeness:

Num Bits	Signed	Unsigned
8	`i8`	`u8`
16	`i16`	`u16`
32	`i32`	`u32`
64	`i64`	`u64`

Numbers from SLN without a . are parsed as i32 by default.

assert ((typeof 13) == i32)

The literal syntax for getting any type of number is the numerical value syntax (e.g. 3) followed by :<type_symbol>.

So that for the number 8 as an i8 number you can write:

print 8:i8

Floating Point Numbers

Floating point numbers (“floats”) are similar to integers in syntax.

Num Bits	Symbol
32 (single precision)	`f32`
64 (double precision)	`f64`

Floats can be gotten from literals by adding a decimal notation or the explicit annotation:

# integer
typeof 1

# floats
typeof 1.
typeof 1.0
typeof 1:f32
typeof 1:f64

f32 is the default for unannotated literals.

You can also use scientific notation equivalent to 1*10^n:

3e4
typeof 3e4

3e-4
typeof 3e4

There are 3 special values for floating point numbers:

+inf or inf: positive “infinity”
-inf: negative “infinity”
nan or -nan: not a number

That have special relationships (sorry went a little crazy with all of the combinations):

2. + inf
2. * inf
2. / inf
inf / 2.

2. // inf
# be careful...
inf // 2.

2. + nan
2. * nan
2. / nan
nan / 2.
2. // nan
# be careful...
nan // 2.


inf + inf
inf * inf
inf / inf
inf // inf
-inf + inf
-inf * inf
-inf / inf
inf / -inf
-inf // inf

The operators are described later in the arithmetic section but should be obvious.

Symbols

Defining Symbols

A full description of symbols will have to wait until level 3 as this is Scheme territory. However we introduce them here since they are a primitive.

Symbols are just everything that is not a number, string, or list (or comment).

Symbols are what you assign values to:

let my_symbol = 0

Here we are using a symbol syntax compatible with most other programming languages (in Python this is called “snake case”). However unlike other languages symbols have much more freedom in what their syntax is. As long as a symbol can’t be parsed as a number, string, list, or comment it will be interpreted as a symbol. Also any of the brackets or separator symbols are not allowed in symbols (i.e. #;()[]{}, from the SLN definition).

Additionally the Scopes language adds some extra restrictions you will notice for the ' and ` characters. We will see in a few places where ' (sugar-quote) gets used in this level, but it will be much later that we encounter ` (spice-quote).

That means all of the following are valid:

let =a-Symbol+for_you~ = 0

let @begin = "itemize"

let * = 4
let two+two = 4
let 2+2 = 4

let yes^you^can = "but should you?"

let valid? = false
let !!important!! = "you are under arrest"

However these will produce errors:

let 'hello = 0
let hell'o = 0
let hello' = 0

let `hello = 0

However the following are fine:

let hel`lo` = 0

The reason for this is is so that Scopes can distinguish between the value a symbol is bound to (like a variable name) and the structure of the symbol itself (i.e. the characters in the symbol).

“Quoting” & Symbols

This is our first encounter with a concept in the Lisp/Scheme world called “quoting”.

Lets bind a value to a symbol first:

let message = "Hello"

We should already understand that printing message will print the string we assigned/bound to it:

print message

However if we quote the message symbol we get what looks like a string “message”:

print 'message

In some sense it really is a “string” in that it is a sequence of UTF-8 characters (with some restrictions).

We can even get the string of the symbol as a real string:

'message as string

Its kind of like in English where you put quotes around a word to signify the word itself (or in the case of “scare quotes” some other connotation other than the typical meaning).

Just to hammer this home that it really is a string you can take the symbol string and bind it to another symbol:

let message-symbol-string = ('message as string)

Meta…

And in fact you don’t even need to have assigned something to a symbol for it to “exist”:

print ('IHaventBeenAssignedToYet as string)
'hello

This ' character is called a “sugar-quote” and is used for syntax macros. There is another kind of quote called a “spice-quote” using the ` character which works at a deeper level.

A full explanation of the implications of the sugar-quote won’t be continued until Level 3. The spice-quote will be discussed at Level 4 as it is a novel concept in Scopes.

I will leave it here that you can use the special function sc_parse_from_string to see for yourself that these special characters are just syntax that will get expanded to concrete functions in the language:

sc_parse_from_string "hello"

'hello

sc_parse_from_string "'hello"

''hello

sc_parse_from_string "''hello"

sc_parse_from_string "`hello"

sc_parse_from_string "'hel'lo"

Operating On Symbols

As we saw above we can cast them to strings:

print ('newsymbol as string)

But we can also compare symbols like strings too:

'newsymbol == 'newsymbol
'newsymbol != 'othersymbol

You can also construct symbols from strings using the constructor:

print ('thingy == (Symbol "thingy"))

We will see later that this is very useful for programmatically accessing symbols from a module.

Lists

Some Context

In scripting languages like Python they often provide a number of primitive data structures like lists, tuples, and maps/dictionaries or even sets.

These are all very high-level concepts compared to the “structs and arrays” of low-level languages like C or C++ (without the standard library).

Scopes aims to combine the efficiency and control of languages like C/C++ while maintaining a relatively high-level veneer similar to Python.

Thus the base language only provides a single seemingly-high-level data structure called simply a list.

However, while similar in some aspects to the “list” of Python this list is very different in implementation and behavior and actually comes from the Scheme/Lisp heritage (LISt Processing).

Describing the full behavior of lists is a topic for Level 3 but here we describe some ways that basic lists can be used in place of mutable lists and maps from a language like Python.

In Level 2 we will also describe how you can get data structures that actually behave like Python lists (i.e. linked-lists), maps/dictionaries (i.e. hash-maps) from the standard library. Its worth forewarning that in the vast majority of cases you will probably want to employ these more classical data structures for performance reasons. The lists we are discussing here are flexible in a fundamental way but are only practically used in a specific context that won’t really be elaborated on until Level 3 when we talk about syntax macros.

We start with defining a simple list of numbers from 0 to 5 in a few different syntax forms:

'(0 1 2 3 4 5)

'(0,1,2, 3, 4 , 5)

Where we have the normal space delimiter, but also the comma character which lets you elide whitespace.

The empty list can be given as:

'()

Notice our friend sugar-quote (') from the Symbols section. You only need this on the outermost list when you define nested lists:

'(0
  ("red" "blue" "green")
  ()
  10
)

Note that to define sublists you do not need another ' character.

A Quick But Necessary Tangent: Why the ‘?

Again this is a topic for Level 3 but I’ll give you the gist here.

First try it out without the ':

$0 ► (0 1)
<string>:1:1: in fn <string>:1
    (0 1)
<string>:1:1: while checking expression
    (0 1)
error: cannot call value of type i32

You can see that scopes is trying to call the function 0 on the parameter 1. Thats kind of weird…

In Lisp & Scheme like languages the list is not only a data-structure for program data like numbers, strings, etc. but it is also the data structure in which the program itself is contained in. This property is called homoiconicity because code and data use the same (homo) syntax (iconicity). This basically makes it much easier to write programs that write other programs since a function can take in a piece of language syntax, rearrange it and output something else. These constructs are called macros and the practice in general is called “metaprogramming”. Its also similar to how compilers like LLVM work by getting some input intermediate representation (IR) code and rewriting it into an optimized form.

Observe that even normal looking language syntax can also be a list if you sugar-quote it:

'(print "hello")

This is how “code is data”; although as we saw above data is not always code.

This concept doesn’t really have a good analog in most scripting languages primarily because metaprogramming is something of a dangerous and complex feature that requires a lot of sophistication on the part of the programmer. And scripting languages are meant to be simple and not too complicated for beginner to intermediate programmers.

Don’t be fooled though. Metaprogramming is an incredibly powerful language feature that used correctly can be very useful. Scopes uses these features extensively and so we will have to become familiar with them. Just not now. Levels 3 and above will deal with them.

Enough metaprogramming I came here to manipulate some data!

List Creation & Manipulation

We have already shown a basic way to define lists manually. But you can also do this programmatically as well. The operations are a bit stripped down in Scheme fashion but its well known that these operations can be the foundation for arbitrary other manipulations.

First is the explicit list constructor:

let l = (list)
let ll = '()

print (l == ll)

Second is the cons function, which is a function taking two values, the second of which must be a list. It then returns a new list with the first element being the first argument and the rest of the elements are the elements of the second (list) argument. An example helps:

cons 0 '(0 1)
cons '() '(0 1 2)

Again I won’t go reeling into all the mind-expanding implications this has here. Lets keep things grounded for now.

Normally in Lisps/Schemes they would have the additional car and cdr functions for taking lists apart. In Scopes we just have the opposite of cons; decons.

decons '(0 1)
decons '(() 0 1 2)

There is also a similar concatenation operator to strings:

.. '(0 1) '(2 3)
'(5 6) .. '(7 8)

Lists can also be compared for equality:

'(0 1) == '(0 1)
'(0) == '(1)

Some Common Listy Structures

Having only a list may seem kind of limiting; and it is in order to maintain simplicity.

The most obvious omission is the lack of any kind of map type.

A common convention in Lisps is to encode maps in lists 2 ways:

plist: Property List
alist: Association List

A plist uses an un-nested list where every two elements are interpreted as a pair. Keys cannot be repeated (at least if you want it to work properly):

let plist = '( 
    "bob" 10
    "suzy" 12
    "jill" 8
)

This is often how keyword function arguments are implemented in Lisp-like languages.

An alist uses a single level of nesting where key-value pairs are sub lists of two elements:

let alist = '( 
    ("bob" 10)
    ("suzy" 12)
    ("jill" 8)
)

The more elements you have in your mapping the “better” the datastructure you will want roughly in order of the number of elements you have:

plist < alist < hash table

Where plists are used for very small numbers of elements (1-20). The boundary between alists and hash tables would require some benchmarking. If in doubt you should probably use a hash table.

I am unaware of any functions for dealing with these structures in Scopes, but I think it helps to understand how these things are used in practice since coming from a scripting language these kinds of things are not used.

To add key-value pairs to a plist you could use concatenation:

let plist = (.. '("dan" 15) plist)

And to add to an alist you would use cons:

let alist = (cons '("dan" 15) alist)

String Construction & Conversions

There are a few different kinds of string conversions that can occur so we outline them here

Converting Arbitrary Value Representations To Strings

You can get the implicit “stringified” version of the representations of primitives using either tostring or repr.

print (tostring 10)
print (repr 10)

This is distinguished from casting of types to string.

tostring give a plain kind of string whereas repr is meant for making it look pretty in different contexts. Here it returns the same result as tostring but on a terminal it gives the raw string: "\x1b[33m10\x1b[0m" and when you print it you get a colored 10.

Note that this stringification is not meant to be a reliable serialization of the values and is just meant for human inspection like for in logs or reporting the value on the terminal.

We will see in higher levels that these two functions are hookable similar to the Python “magic method” protocols e.g. __repr__.

Casting C-style “char” Arrays

While this is really a topic best covered in detail in Level 2, we will briefly introduce it here as it is useful in a few instances where deeper knowledge is not necessary.

That is converting C-style char arrays to strings. This is very useful for parsing command line arguments which are supplied at the most basic level like this from the script-launch-args function:

let source-path argc argv = (script-launch-args)

print "source-path:" source-path (typeof source-path)
print "argc:" argc (typeof argc)
print "argv:" argv

print "argv[0]:" (typeof (argv @ 0)) (string (argv @ 0))

If you run this script like:

scopes ./_bin/cli_conversion.sc Hello

You should see:

source-path: /home/salotz/tree/personal/devel/scopes-newbs/_bin/cli_conversion.sc string
argc: 1 i32
argv: $riroroxesisogul:(@ (@ i8))
argv[0]: (@ i8) Hello

Notice that the type of argv @ 0 is (@ i8) which is a pointer to a “char” (8-byte sized thing) array.

And here we are using the explicit string constructor to convert it to a string.

Arithmetic & Mathematics

We have kind of already seen arithmetic above in the section on Operators.

TODO

Basic Control Flow

Conditionals & Boolean Expressions

Boolean expressions are expressions which evaluate to either true or false.

true and true or false

0 > 3

"goodbye" != "hello"

This can be used in combination with the familiar if-else kind of syntax:

let valid? = true

if valid? (print "SO TRUEE!!")


let height = 3.3

let MIN_HEIGHT = 4

if (height < MIN_HEIGHT)
    print "You must be " MIN_HEIGHT "ft to ride"
else
    print "Step aboard!"


let color-pick = "red"

# choose a complementary color
let complement =
    if (color-pick == "blue")
        "yellow"

    elseif (color-pick == "red")
        "green"

    elseif (color-pick == "yellow")
        "purple"

    else
        "invalid-color"

if (complement != "invalid-color")
    print complement "is the complement to" color-pick
else
    print "You did not pick a primary color"

Some values can be implicitly interpreted as booleans:

if (not none) (print "boolable")

if (not 0) (print "boolable")
if 1 (print "boolable")

if inf (print "boolable")
if nan (print "boolable")

As we will see in type casting these values can be explicitly cast as bools:

0 as bool
1 as bool
-1 as bool

string and null types cannot be cast to booleans.

Branching Control Flow

Note that because of typing rules you may find some expressions give compiler errors that you would expect to work, e.g. this is a compiler error:

# INVALID
if true
    "yellow"
else
    3

Why is this? The error is basically explaining that the different branches of the if-else statement are returning incompatible types, string and i32.

To better explain this consider that you can write if-else expressions (and any other expression) like this:

let result =
    if true
        "yellow"
    else
        "blue"

print result

This is called roughly “expression oriented programming” because most things are made up of expressions and they are composable. This is because all expressions have “return” values which are implicitly the last value in the block, or in this case the last statement of each branch.

We’ll see this put to more familiar uses in the sections on functions and modules but its useful to introduce it here to drive the point home that this is not something special to them and can be used pretty much anywhere in the language.

With this understanding we can see that even in the case without the preceding let that the expression itself needs to be type checked.

Thinking in terms of a the variable result needing to have a predetermined type (even when not using it).

We will talk about types in depth at higher levels but for know you should know that for branching control flow, each branch must have compatible types.

This also explains why the examples where we are just printing in each branch works. print has a special return type void, which is similar to Python’s None when used this way and just indicates no return type. If each branch has return type void then the whole thing works. We could even explicitly return void:

if true
    print "yellow"
else
    void

Loops

The fundamental loop in scopes is very general, but maybe not exactly what you are used to from a langauge like Python.

Scopes does provide these comfy loops though and its very satisfying.

For-Loop

Here is a basic for loop like you would find in python:

local result = 0
for i in (range 10)
    result + 1

You can loop over the language level lists, but they need to be “quoted” by placing a single apostrophe ' at the beginning of the list just like in other Scheme languages. Otherwise it will try to call the function 0 on arguments 1 and 2.

for i in '(0 1 2)
    print i

The for-loop also supports the break and continue statements which are similar to those in other languages.

for i in (range 10)
    if (i == 1)
        print "continuing"
        continue;
    elseif (i > 2)
        print "breaking"
        break;
    else
        print "nothing"

In the for-loop it doesn’t make sense for break and continue to return any values which is why they have a semicolon at the end (see Defining & Calling Functions) but as we will see below this is possible.

While-Loop

local i = 0
while (i < 5)
    print i
    i += 1

General Loop

The general loop has a few more requirements but is more flexible.

There should be:

A path to “repeat” the loop
A path to break out of the loop

loop (a = 0)
    print a
    if (a < 10)
        repeat (a + 1)
    else
        break a

Technically the repeat is redundant and a bare value at the end of a scope will “return” it and continue the loop.

loop (a = 0)
    print a
    if (a < 10)
        a + 1
    else
        break a

You do need the break though, or it won’t compile as this will always be an infinite loop.

Fold-Loop

The other loop styles are very well suited to a mutation based method of constructing results. The “fold-loop” provides a more functional approach to constructing objects and is compatible with iteratively constructing immutable objects. We will see how this is practical later when we come across mutable & immutable objects.

Here is a simple example that increments a number:

let input = 0

let result =
    fold (result = input) for i in (range 3)
        result + 1

print result

First note that the input to the loop is not a local definition and is instead a let, which is not mutable. So we know it is not being mutated.

The second thing is that we have to actually accept the result of the loop as if it was a function. This is because the loop really is a kind of functional construct.

Thirdly, we can break apart the actual loop line. As normal we have the for i in (range 3) that is the same as the for-loop. The first part, fold (result = input), introduces the inputs to the loop. This is similar to a function call with a named argument where result is the argument name.

In the body of the loop we have the same as the other ones, where the last line is returned to the next iteration of the loop.

Note that the result symbol is immutable, equivalent to let result = input and cannot be mutated.

Here is another example with an immutable structure, the list.

let things = '(0 1 2 3)

let new-things =
    fold (new-things = '()) for thing in things
        let new-thing = ((thing as i32) + 1)
        cons new-thing new-things

print new-things

There are two small issues with this because of the specifics of lists, that are inconsequential to the example: 1. the results are in reverse because of how cons works and 2. we have to explicitly cast thing to an int.

We will see more realistic examples later.

Switch Statement

Switch statements are a compact and structured way to dispatch some code on a specific value.

While this isn’t something explicit in Python it is in general pretty common in programming languages and is sometimes preferrable to an if-else chain.

Here is a basic example which includes 3 cases for an integer and includes a default value.

let val = 2

switch val
case 0
    print "Hello"
case 1
    print "Goodbye!"
case 2
    print "Be gone!"
default
    print "Huh?"

Because of typing rules all of the case values should be able to casted to the same type and each piece of code run for the cases should have the same return type.

In this example there are no return types (or technically it is return type void) because we are just printing something.

Returning from a switch statement can be done as follows:

let val = 0

let result =
    switch val
    case 0
        "zero"
    case 1
        "one"
    default
        "unknown"

print result

For example the following would result in a compile error because the case types are different.

let val = 0:i32

# INVALID!!
let result =
    switch val
    case 0:i32
        "zero"
    case "hello"
        "hello"
    default
        "unknown"

This will also give a compilation error because the return types from each case block is different:

let val = 0

# INVALID!!
let result =
    switch val
    case 0
        "zero"
    case 1
        1
    default
        "unknown"

“GOTO”

Interestingly Scopes has a sort of well behaved “goto” statement by using the label and merge keywords.

label finish
    for i in (range 10)
        if (i > 3)
            print i
            merge finish

Here we are running a for-loop and in a certain condition we “merge” back this branch to a previously labelled location in the code.

This example is sort of equivalent to break statement we already introduced, however it is much more flexible.

We won’t elaborate on this much further as this is obviously an advanced feature.

We will note that this version is much safer than a normal GOTO that just specifies line numbers to jump to because the actual branching and merges of these branches must be handled or the compiler will complain. That and you can only jump to previously defined labels.

Type Casting

You can convert types using the as operator:

# constant
0 as f32

1.2 as i32

-1 as u32

# not constant
'a-symbol as string

Which is a static cast and happens at compile time (see caveats above).

Functions

Defining & Calling Functions

As tradition in Scheme-like functional-ish programming languages there are a variety of syntaxes for defining functions, due to them being higher order and possible anonymous. Here we go over the equivalent ways for defining functions to a simple language like Python.

First we can explicitly define a function with the fn syntax:

fn say-hello (name)
    print "Hello:" name

# and call them like you would guess
say-hello "Bob"

Multiple arguments can be given:

fn introduce (name age)
    print "My name age is:" name
    print "I am " age " years old"

# and call them like you would guess
introduce "Bob" 34

Some possibilities for organizing multiple arguments:

Horizontally, but with a bit more space:

fn introduce
    name age

    print "My name age is:" name
    print "I am " age " years old"

# and call them like you would guess
introduce "Bob" 34

Vertical layout with parens:

fn introduce (
    name
    age
    )

    print "My name age is:" name
    print "I am " age " years old"

# and call them like you would guess
introduce "Bob" 34

Vertical layout with naked notation:

fn introduce
    name
        age

    print "My name age is:" name
    print "I am " age " years old"

# and call them like you would guess
introduce "Bob" 34

Functions without any arguments can be called in two ways:

fn yell ()
    print "AHHHHH!!!!"

(yell)
yell;

There is also another function-like construct called inline. Which behaves very similarly but has properties that will only make sense in Level 2 when we talk about constant and dynamic values. Here is an example:

inline yell ()
    print "AHHHHH!!!!"

yell;

As you can see it appears to be the same! We will discuss how it is different later.

In some situations you might need the builtin call which is some syntax for explicitly applying a function to arguments:

fn add (x y)
    x + y

print (call add 1 2)

Return Values

As in all “blocks” in scopes the last value is returned, as was seen in the loop examples. The same is true for functions:

fn gimme ()
    "that"

print (gimme)

You can also use a return statement to be explicit:

fn get-over ()
    return "here"

print (get-over)

Scopes can perform “unpacking” in a general way similar to “tuple unpacking” in Python using the _ prefix operator. This is often used to “forward” multiple return values from function returns, rather than having to do the destructuring yourself.

fn args ()
    _ 1 2 3

let a b c = (args)
print a b c

You can unpack arguments for function parameters as well:

fn trio (a b c)
    print a
    print b
    print c

let args = '(0 1 2)

trio (unpack args)

Recursion With Functions

Recursion is achieved using this-function:

fn rec-count (n)
    print n
    if (n > 5)
        return;
    this-function (n + 1)

rec-count 0

Recursion is a bit more complicated than this however due to type checking. This will be discussed in much more detail in Level 2, so don’t be surprised if you get errors when trying this on your own.

Anonymous Functions AKA Lambdas

Scopes also supports anonymous unnamed functions (typically called lambdas).

An example with the typical usage of lambdas:

print ((fn (x) (x + 1)) 4)

You can assign the function to a symbol to simulate a normal declaration:

let lambda = (fn (x) (x + 1))

print (lambda 4)

let lambda2 =
    fn (x)
        x + 1

print (lambda2 4)

If you want to use the expanded syntax for an anonymous function you will probably need to make use of the call builtin.

let result =
    call
        fn (x)
            x + 1
        3

print result

—

Finally you can also use the “currying” notation:

print (((x) -> x + 1) 3)

This will be discussed elsewhere on its own for how to actually use it for Currying functions and functional programming but the syntax is simple enough here to simply use it for a lmabda.

Modules, Namespaces, & Scopes

Scopes provides a module system very much like python.

Importing Modules

You can import installed libraries and use their methods such as:

import String
let str = (String.String "hello")

Or you can dump the exported symbols (i.e. functions and variables) into your current namespace with the using keyword:

using import String
let str = (String "hello")

You can also directly bind a loaded module to a symbol:

let string_mod = (import String)
let str = (string_mod.String "hello")

You can also do some fancier imports although they are a little imperfect in their operation.

Firstly you can rebind particular symbols from a module to another name using the from keyword:

let myString = (from (import String) let String)

print (myString "Hello")

print (String "Hello")

However notice that the String is dumped into the local namespace. To get around this we can use another (newer) syntax which accepts keyed values for imported values:

from (import String) let
    str = String

print (typeof str)
print (str "hello")

print String

This still binds the String module name to the context. You can avoid this as well by using this syntax:

Writing Modules

Like in Python a module is implicitly defined for files. Unlike Python however is that the module only “returns” or “exports” the last thing in the file. This behavior is consistent with most other constructs in Scopes.

However, it is a little strange at first since a module can return not just a “module” or namespace but even single functions or values in the simplest case.

So you can export a function like this in the file first_mod.sc:

fn not-exported ()
    print "I don't do anything"

fn test()
    print "testing out the function"

Then import the function directly:

let test = (import .test_mod)

test;

Notice that unlike the other modules we have imported or the behavior in other languages that the module is just a single function.

Also notice that the first function not-exported is not available to be called from the import.

If you want to export all of the symbols in module you can use the builtin locals which is function that returns a namespace of all the locally defined functions.

For example if you have the module in a file hellomod.sc:

fn hello (name)
    (print "Hello" name)

do
    let hello
    locals;

And then import it like:

let hellomod = (import .hellomod)

hellomod.hello "Bob"

Lastly sometimes you can unintentionally return things from a module (especially when writing small scripts for learning). To stop this you can put a none at the end of a module.

let a = 3
none

Or more stylistically you can call the null function ():

let a = 3
;

We will see in higher levels that some values cannot be returned from modules and so we might guard the end of the module like this, rather than raising an error.

do-blocks

The do block can thought of being equivalent to defining and executing a new unnamed function.

let msg = "Do the do"

do
    print msg

As you can see it can use values in the surrounding scope (a “closure”).

But anything defined in the block is not available in the outer scope:

let name = "Bob"

do
    let other = "Alice"
    print "hello" name
    print "hello" other

# this would raise an error
# print other

The do-let-locals pattern from above is a nice way to export symbols from a module in a clean way. It should be used as the most common convention.

You can also use this block to customize what gets exported. Much like the __all__ magic variable in Python.

fn thing1 ()
    print "Thing1"

fn thing2 ()
    print "Thing2"


do
    let mything = thing1
    let thing2
    locals;

But what is do actually returning as a value?

Consider this code:

let scope =
    do
        let
            x = 1
            y = "Hello"
        locals;

print scope.x
print scope.y

This is essentially the module which we made above but instead of exporting it and using it in another module we are just using it right away.

As the variable name suggests the do block returns a “scope”, which we will talk more about below.

Executing a module

Similar to the common python refrain of if __name__ == "__main__": Scopes has a similar special value that can be used to conditionally execute code if a module is executed like scopes mod.sc rather than being imported using main-module?.

print "module code"

if main-module?
    print "running tests..."

Modules are Just First Class Scopes

We should talk about the namesake of the language Scope, and what makes it different from a module system like Python.

Python has this saying:

Namespaces are one honking great idea – let’s do more of those!

Which never got taken that seriously because there is no first-class concept of a “namespace” in Python. However, in Scopes we do have this first-class namespace and surprisingly its called a Scope.

So now you know where the name comes from.

Above we showed how to create a Scope using a do block and how that is used to export symbols as a module for consumption in other modules.

let scope =
    do
        let
            message = "hello"
            name = "bob"
        locals;

print (typeof scope)
for k v in scope (print k ":" v)

This is the simplest and easiest way to construct a Scope. However, there is a more explicit API that uses the type itself.

One way is to use the 'bind-symbols method:

let scope =
    'bind-symbols (Scope)
        message = "hello"
        name = "Bob"

run-stage;

print (typeof scope)
print scope.name

Note you need to do a “run stage” (with run-stage which will be talked about much later).

In essence this is all the locals function above does except in a convenient way just for everything in the local do-block scope. If you want complete control over what gets exported and under what name you are free to do so.

And you should also see that as a consumer of a Scope from another module you also have complete control over the Scope object. We will see where this comes in handy in cleaning up messy namespaces that are autogenerated from parsing C header files in Level 2.

Here is an example of this building on a constructed Scope:

let scope =
    do
        let
            message = "hello"
            name = "bob"
            junk = "You don't want me"
        locals;

# remove the "junk" symbol from the scope
let new-scope =
    fold (scope = (Scope)) for k v in scope

        let name = (k as Symbol as string)

        if (name != "junk")
            'bind scope (Symbol name) v
        else
            scope

run-stage;

print new-scope.message

# this is not in the scope
# print new-scope.junk

Scopes are actually a really useful as a mapping data structure for small numbers of static assets and can kind of replace a Python dict for a lot of use cases.

To hear more about this and the other utilities of Scopes see the section on them in Datastructures.

Characters

As an addition to the primitive types above there is a useful function for dealing with single characters.

using import UTF-8

# convert a single character string literal to a char
let char = (char32 "a")


print (typeof char)
print char

# TODO: move to later section, or introduce string prefixes before

# you can also use the string prefix 'c'
let char = c"a"

print (typeof char)
print char

Data Structures

So far the only kind of datastructure we have seen is the list. We also stressed that lists aren’t really the same as lists or maps in other languages and are really only used for either very simple structures with small amounts of data and for implementing the (sugar) syntax macros (a very advanced feature).

So what really are the comparable structures to lists, dicts, tuples, and sets like in Python?

Here is where we need to come to grips with the fact that while Scopes provides many of the comfy pleasantries of a language like Python, it really is intended to be in the same class of languages as C/C++, Ada, and Rust. That is underneath the scripting language like veneer Scopes is both statically typed and “low level” in the sense that it allows you to have complete control of your data structures.

So where in a language like Python you have built-in syntax for things like linked lists (roughly) using [a, b], hash maps {'a' : 1}, or tuples (1, "hello"). In low-level statically typed languages there usually isn’t specific syntax for any “blessed” high-level data structures. Instead you either implement them yourselves or import them from libraries; either the “standard library” which ships with the language implementation or perhaps a third-party one.

You can also do this in Python with Classes, but is considered bad-taste when the built-in types are sufficient.

The term “low-level” here is a bit ambiguous and roughly means a language that is more-or-less similar to the C programming language in terms of the basic datastructures that are built in to the language. Namely arrays and structs (and unions which are much less used).

Scopes intends to be one-to-one compatible with C programs and so has similar built-in types. We will see much more of this in action later.

We won’t talk about arrays or structs until Level 2 (since that is a C/C++ equivalent feature); but we can skip over them to discuss some easy to use high-level Python-like equivalents.

The above explanation was just to soften the blow of some divergence from the Pythonic simplicity we have seen up until now.

Tuples

The tuple is probably the simplest of these. As such we will use it as an example to describe general features of more complex types and class-like constructs.

A tuple is an immutable datastructure of a fixed size which can contain any combination of element types.

You can use the function tupleof to construct them directly:

let tup = (tupleof 1 2:f32 "hello")
print tup

You can get the values of the tuple in a few ways.

Via unpacking:

let tup = (tupleof 1 2:f32 "hello")

let a b c = (unpack tup)

print a b c

Explicitly accessing values via the @ syntax:

let tup = (tupleof 1 2:f32 "hello")

print (@ tup 2)
print (tup @ 1)

There are two methods

let tup = (tupleof (a = 1) (b = "hello"))
print tup

print ('emit tup 'a)

let tup = (tupleof (a = 1) (b = "hello"))

print ('explode tup)

And further you can actually define values to have keys associated with them, much like the Python namedtuple:

let namedtuple = (tupleof (a = 3) (b = "hello"))

print "a" namedtuple.a
print "b" namedtuple.b

Here we see the first use of the “dot” notation which is also used in Python.

But you can also use the @ selector instead of the keys:

let namedtuple = (tupleof (a = 3) (b = "hello"))

print "0" (namedtuple @ 0)
print "1" (namedtuple @ 1)

Before we dig into the other datastructures we will go over some of the common patterns to all of them using the tuple as an example.

Mutability

If you declare a datastructure as local you can mutate the components as long as they are the same type as in the construction or declaration.

For an indexed structure this is:

local things = (tupleof 3 "hello")

things @ 0 = 5
print (things @ 0)

Notice that you don’t need parens around the first part because of the infix operator precedence rules, but you can add them if you like:

local things = (tupleof 3 "hello")

(things @ 0) = 5
print (things @ 0)

local things = (tupleof (a = 3) (b = "hello"))

things.a = 4
print things.a

# NOT allowed
# things.a = "test"

fn scopetest ()
    local t-inside =
        tupleof
            (a = 3)
            (b = "hello")

    print (t-inside @ 0)

    t-inside


# This won't be mutable since it will rebind it as immutable in the
  outside scope
 
  let t-outside = (scopetest)

local t-outside = (scopetest)

print t-outside.a

t-outside.a = 10

print t-outside.a

Attributes, Methods, and Meta-Methods

In the above examples we have used a number of recurring conventions for retrieving and setting data using the “dot” operator or the @ operator.

let t = (tupleof (a = 3) (b = "hello"))

t @ 0
t.a
(t . a)
(. t a)

These are implemented via the “metamethods” system which is similar to the “magic methods” in Python.

These are protocols which can be customized by each type. This is also how operator overload is implemented.

The only difference when compared to operator overload in other languages is that this extends to general function-looking metamethods.

For instance functions like unpack as we have seen above. However these are different in that they are not infix operators like the dot operator.

let t = (tupleof (a = 3) (b = "hello"))

(unpack t)

On the dot operator there are 3 different syntaxes which should be described.

There is the “prefix” version (. t a), “infix” version (t . a), and the “sugar” version t.a.

The last one is of interest because it is actually a symbol that gets expanded to the prefix version.

Lastly we see the use of this syntax:

let t = (tupleof (a = 3) (b = "hello"))

('emit t)

This is called the “method” syntax.

Scopes

We saw how Scopes are the foundation of modules and first class namespaces in Scopes (the language). This brings up an interesting use case of a Scope as a kind of simple “map” type akin to a hash table or python dictionary.

We saw in the ‘List’ section how to implement some “map” types with simple lists, but these are undesirable as data structures because they are purely syntactic and don’t really exist at run time at all. Lists really only exist to contain syntax and to be manipulated for macros, the core metaprogramming feature of lisp-like languages.

But what if you aren’t metaprogramming? You just want to use something like Python. In this case a Scope might be a good choice for replacing a dictionary.

However, because Scopes is a low-level language there will be a necessary distinction between data-structures like Hash Maps (see ‘Maps’ below) which will dynamically allocate memory and static, immutable ones like the Scope.

Scopes are useful for if you know up front what all the entries will be in the map, or they rarely change.

For instance you might have the following kind of code somewhere defining some constants. This could be transformed into a Scope. Both are shown in this snippet:

# using variables
let
    retriever = "A loyal dog that fetches dead ducks."
    poodle = "A haute one, with curly locks."
    dachsund = "Roots out badgers."

print "Variable style"
print "retriever: " retriever
print "poodle: " poodle
print "dachsund: " dachsund


# using a Scope
let breeds =
    do
        let
            retriever = "A loyal dog that fetches dead ducks."
            poodle = "A haute one, with curly locks."
            dachsund = "Roots out badgers."
        locals;


print "\nScope style"
for breed description in breeds
    print description

As you can see in this example you can loop over all of the values in a Scope.

You can get values out of the scope using a lookup by using the getattr metamethod:

let breeds =
    do
        let
            retriever = "A loyal dog that fetches dead ducks."
            poodle = "A haute one, with curly locks."
            dachsund = "Kills badgers."
        locals;

# use the symbol syntax
print (getattr scope 'poodle)

# programatically construct the symbol
print (getattr scope (Symbol "dachsund"))

Scopes also have a number of other operators and metamethods you can use. Here are some useful ones.

Combine scopes together with ..:

let breeds =
    do
        let
            retriever = "A loyal dog that fetches dead ducks."
            poodle = "A haute one, with curly locks."
            dachsund = "Roots out badgers."
        locals;

let other-breeds =
    do
        let
            shepherd = "Can keep your sheep in line."
        locals;


let all-breeds = (.. breeds other-breeds)

run-stage;

print all-breeds.shepherd
print all-breeds.poodle

More Scopes Tricks

A neat trick to remove a level of indentation from let-do-let statements is to use the following syntax:

vvv bind config
do
    let A = 3
    locals;

print config.A

;

Explaining what vvv bind is doing is out of scope for now so just think of it as some syntactic sugar (which is actually it is).

Nested Scopes:

let breeds =
    do
        let
            retriever = "A loyal dog that fetches dead ducks."
            poodle = "A haute one, with curly locks."
            dachsund = "Roots out badgers."
        locals;


let dog-info =
    do
        let
            breeds = breeds
            dog = "A dog is a kind of quadruped."
        locals;

print dog-info.dog
print dog-info.breeds.poodle

When you pass scopes into functions you might find that you get an error when trying to access members. This is because if you use fn then the argument is not constant and member lookup via the . operator only supports constants.

Here is an example of how to do this and retrieving members in a constant manner using the inline function declaration:

vvv bind config
do
    let A = 3
    locals;

inline do-stuff (conf)
    print conf.A

do-stuff config

;

The other way is to access members using non-constant methods:

vvv bind config
do
    let A = 3
    locals;

fn do-stuff (conf)
    print ('@ conf 'A)

do-stuff config

;

Both have their tradeoffs to be considered.

Map

Basics of a Map.

using import Map
using import String

global mymap : (Map string i32)

'set mymap "a" 3:i32

try
    print ('get mymap "a")
except (e)
    print "'a' not in mymap "

Set

There is a Set class that you can use as well, here are the basics:

using import Set

# declare a set
local digits : (Set u8)

# add things to the set
'insert digits 0:u8
'insert digits 1:u8
'insert digits 2:u8
'insert digits 3:u8
'insert digits 4:u8
'insert digits 5:u8
'insert digits 6:u8
'insert digits 7:u8
'insert digits 8:u8
'insert digits 9:u8

# test for inclusion
print (1:u8 in digits)
print ('in? digits 1:u8)

# iterate over a set
for digit in digits
    print (tostring digit)

# adding duplicates has no effect
print "num digits:" (countof digits)

'insert digits 0:u8
'insert digits 1:u8
print "num digits after duplicates:" (countof digits)

# remove things from the set
print ('pop digits)
print "num digits after pop:" (countof digits)

;

Currently it doesn’t have too much functionality that you might expect from fancier versions. Perhaps this will improve in the future.

Enums

As you are probably already familiar with an Enum is a way to assign values a semantic meaning that is mapped to some other lower-level value, typically an integer.

Here is a basic enum in Scopes:

using import enum

enum Things
    A
    B

print "Things.A:" Things.A
print "Things.B:" Things.B

print "typeof Things:" (typeof Things)
print "typeof Things.A:" (typeof Things.A)

# useful attributes
print "Things.A.Literal:" Things.A.Literal
print "Things.A.Name:" Things.A.Name
print "typeof Things.A.Literal:" (typeof Things.A.Literal)
print "typeof Things.A.Name:" (typeof Things.A.Name)

In this example you can see both how to get the enum cases and how to get both the string name of the enum case via the Name attribute and how to get the underlying data value for it via Literal.

By default the Literal values be u8 values starting from 0, but you can customize this if you like:

using import enum

enum Things
    A = 3
    B = 5

print "A" Things.A.Literal
print "B" Things.B.Literal

using import enum

enum Different_Things
    A
    B = 3
    C

print "A" Different_Things.A.Literal
print "B" Different_Things.B.Literal
print "C" Different_Things.C.Literal

If you mix the styles it will restart counting from the last defined one.

There is a lot more to say on enums as they will become even more useful as a way of creating “tagged unions”. This will be elaborated on in Level 2 when we will be more concerned with the type system however.

—

One common use of enum is in switch statements. Here is an example just for fun:

using import enum

enum Actions plain
    Nothing = 0
    Terminate = 1

# generate the case somehow
let action = Actions.Nothing

# dispatch on the value
switch action
case Actions.Nothing
    print "doing nothing"

case Actions.Terminate
    print "Terminating"

default
    print "default"

Exceptions, Errors, Assertions, and Premature Program Exit

Error Propagation

Error propagation is much the same as you would expect syntactically. You raise errors (or call the error function) and then you can catch them with a try-~except~ block.

Here is about as simple as it gets:

try
    raise "Error"
except (e)
    print e

Notice that we aren’t raising any specific exception or error types and instead just a plain string is being raised. Any type of value can be raised in exceptions.

Just for fun:

try
    raise "🤯"
except (e)
    print e

If you use the error function it will raise an Error type object:

fn test-error ()
    if true
        error "WRONG!!"
    else
        print "right"

try
    test-error;
except (err)
    print "Something bad happened:"
    print err
    print (typeof err)
;

Also note that the symbol you bind the exception value to can be anything.

You can also use the else clause instead of except which will drop whatever the exception object is:

try
    raise "Help!"
else
    print "Dropped the error"

If you run the function without the try-except block it will raise an error:

fn test-error ()
    if true
        error "WRONG!!"
    else
        print "right"

test-error;

—

Exceptions in Scopes are monomorphic however which is a fancy way of saying you can’t have different kinds of error values propagated in the same try block.

So this code is invalid:

# INVALID
try
    if true
        raise "Error"
    else
        raise 1

except (e)
    print e

And just to bring the point home you will get a similar compiler error even if you try to use a function with multiple error types:

# INVALID
fn polymorph-errors ()
    if true
        raise "Error"
    else
        raise 1

polymorph-errors;

# try
#     polymorph-errors;
# except (e)
#     print e

Technically, the compilation error happens at function instantiation which occurs when the function is called (this will be discussed in Level 2), which is why its added in this snippet.

The recommended way of solving this is by using an enum (AKA union) type as the exception type. This will be discussed in Level 2.

—

Note that because of type checking, some things might not work as you expect depending on your biases.

For instance consider this invalid code:

fn test-error ()
    if true
        error "WRONG!!"

    else
        "right"

# INVALID
try
    test-error;
except (e)
    print "error occured"

;

This will result in a compiler error that roughly decsribes that the returning value from the except clause conflicts with that of the try block. This is because the test-error function returns a string and the except block returns nothing (or void).

To fix this code we would need to actually return a string from the except clause:

fn test-error ()
    if true
        error "WRONG!!"

    else
        "right"

try
    test-error;
except (e)
    "error occured"

;

This might seem strange, but in practice you really should be doing something like the following to handle error propagation:

fn test-error ()
    if true
        error "WRONG!!"

    else
        "right"

let result =
    try
        test-error;
    except (e)
        "error occured"

print result
;

This is similar to the if-else type branching that was explained previously.

We will see in more detail in higher levels on how to deal with these typing constraints, but this should be sufficient to avoid the inevitable confusion on this if you are used to a more dynamic language.

You can also create your own error types and even do things with them. This uses concepts that will be discussed in level 2 but the basics are shown here:

using import struct

struct myException
    msg : string

try
    raise (myException "an error occurred")
except (e)
    print e.msg

How Errors are Different

However, because Scopes is a typed language there are some limitations that might seem weird to a Python programmer. For instance the following code will not even compile:

error "Bare error here"
;

Nor will:

fn test-error ()
    error "Error"

test-error;

The reason is that to maintain typing any function with an error in it actually has the type signature dynamically modified to accomodate for the error.

In languages like C and Odin with no exceptions (which can also be turned off in C++) you typically have to of roll your own kind of error handling system where you are always returning both the value from the computation and the error itself if any. Although languages like Odin provide a specific support for making this simpler in the language. This article by the creator of Odin does a good job comparing the “normal control flow” expressions to those of Python in this article: https://www.gingerbill.org/article/2018/09/05/exceptions-and-why-odin-will-never-have-them/

The reason is complicated but ultimately comes down to performance and lower complexity.

In the “normal control flow” approach there is no exceptional (haha get it) behavior occuring.

However in languages like Python or C++, exceptions are implemented using some form of GOTO. That is control flow doesn’t follow the normal path you would expect it to in your code. This is all the “magic” it takes to be able to pass exceptions up the stack and continue execution elsewhere.

Its not that this is bad per se, but it just adds an extra layer of complexity into your code. The detractors of this kind of system have a point which is that this kind of complex system shouldn’t come stock in a low level language like C/C++, where performance is critical. Indeed it doesn’t come as a default in C and you have to use things like setjmp and longjmp to accomplish this kind of behavior (or a library that does it for you). Indeed many libraries and language features like coroutines, generators, etc. all use this to great effect.

In some sense Scopes (and also Rust has a similar system) has the best of both worlds in which instead of resorting to non-local control flow errors are implemented in the type system.

You could implement non-local exceptions in Scopes the same as coroutines but that would be a choice you could make in a specific library or project rather than the language as a whole.

There will be more discussions of the details of the changes in type signatures etc. in Level 2 when it is more appropriate.

Assertions

Scopes has the common assert function which you can use for quick checks of boolean expressions. However, instead of (like in Python) raising a special error when the assert value is false the program is aborted.

For these nothing will happen:

assert true

assert (not false)

assert (1 == 1)

But this will abort and dump core.

assert false

exit

TODO

Generators & Iteration

TODO

Documentation

Our main concern in this section will be with docstrings, a familiar feature.

Docstrings can be placed on just about anything.

""""The Docstring for the module.

    More details down here. Isn't it nice to only have the string
    delimiters written once at the top?


""""One of my favorite numbers.
let three = 3


fn foo (bar)
    """"Our favorite function.

        Inputs
        ------

        bar : i32

        Outputs
        -------

        baz : f32

    bar as f32

The docstrings on components can be retrieved by using the 'docstring method on their containing scope.

import itertools

# inline funcion docstring
print ('docstring itertools 'closest)

# module docstring
print ('module-docstring itertools)

Making sure to quote the symbol you want the docstring of.

Module Loading Mechanism & Configuration

Module Search Paths

Languages like Python typically have a mechanism by which you can configure the file paths and sources of modules which can be imported without using relative file paths.

That is typically the standard installation location of libraries, and includes the standard library.

But there are others including setting them by environment variables or directly manipulating them in a module.

For instance how does Scopes know where to find the String module?

First you will need to find where Scopes is installed via the compiler-dir:

print compiler-dir

If you look in this folder you will see a folder lib/scopes with a number of modules. By default Scopes will look in this folder for modules to load with import.

You can see what these default values are with the builtin __env Scope object:

print __env.module-search-path

You will see something like this (where <SCOPES_DIR> is the directory that Scopes is installed in):

("<SCOPES_DIR>/lib/scopes/packages/?.sc"
 "<SCOPES_DIR>/lib/scopes/packages/?/init.sc"
 "<SCOPES_DIR>/lib/scopes/?.sc"
 "<SCOPES_DIR>/lib/scopes/?/init.sc")

There are two distinct prefixes. The lib/scopes collection which is for the standard library modules distributed with Scopes itself. And the lib/scopes/packages collection which can be used to place third-party packages in.

This is very similar to Python and typically the installation of packages into the third party packages folder is managed by a package manager. See the Package Management section for more on package managers.

For each collection of packages we can see there are two separate search paths: <search-path>/?.sc and <search-path>/?/init.sc.

The first will find simple one-file modules and the second will find modules corresponding to the directory name containing the init.sc. Which is similar to the __init__.py files from Python modules, except that instead of being something that is hardcoded is actually just a convention and a result of the pattern matching used in the search paths.

Here is an example of a package that could be installed into <SCOPES_DIR>/lib/scopes/packages (see the side_quests/package folder):

package/
└── src
    ├── env.sc
    ├── show-mod-main.sc
    ├── single.sc
    └── things
        ├── init.sc
        ├── thing0.sc
        └── thing1.sc

The init.sc does not even need to contain any code.

This packaged installed into a search prefix would look like:

<search-path>/
    ├── single.sc
    └── things
        ├── init.sc
        ├── thing0.sc
        └── thing1.sc

And you would import them as such:

let single = (import single)
let things = (import things)
print things.thing0.thing
print things.thing1.thing

let thing0 = (import things.thing0)
let thing1 = (import things.thing1)

Other Search Paths

In addition to the __env.module-search-path for Scopes modules, there are also equivalent search paths for C modules which can be loaded into Scopes. Because C code is broken up into both binary shared libraries and header files there are two search paths: __env.library-search-path and __env.include-search-path respectively.

These search paths will be automatically searched by the shared-library and incude functions which will be discussed in Level 2 when we discuss loading C libraries.

Configuring the Environment

We have seen how the __env Scope object controls the search paths for modules. These scopes can be configured by using a __env.sc file in a project.

If you launch the Scopes compiler with the -e flag then it will look up the filesystem tree relative to the called module for the first __env.sc file it finds. This module will be executed before the main module and the returned result will be injected as the __env object to the module.

For instance in the side_quests/package example you can execute the env.sc module which prints the environment search paths:

# without the __env.sc file loaded
scopes ./src/env.sc

# with the __env.sc file loaded
scopes -e ./src/env.sc

You can see there are two extra search paths with the -e flag which are of the local side_quests/package/src directory:

"<scopes-newbs>/side_quests/package/src/?.sc"
"<scopes-newbs>/side_quests/package/src/?/init.sc"

This allows you to import and run the local modules without using relative imports:

scopes -e -c 'import single'

This is accomplished in the __env.sc file by redefining the __env scope and returning it from the module:

'bind-symbols __env

  module-search-path =
      cons
          # the current module in development
          .. module-dir "/src/?.sc"
          .. module-dir "/src/?/init.sc"
          __env.module-search-path

  include-search-path = __env.include-search-path

  library-search-path = __env.library-search-path

In this example we are altering the module-search-path by adding the src directory relative to the __env.sc filesystem location.

Because we are not mutating __env (as Scopes are immutable) we also need to define the other search paths, which we leave undisturbed.

The new __env symbol is exported from the module and passed to whatever is being called.

Its important to note that the __env.sc is not a part of the source code of the package, i.e. everything under src. This allows any consumer of the code to customize the search paths they want to use.

Additionally there is nothing standard about this particular layout and you can customize it as you wish. Although this layout is fairly common and clean for writing a package.

In the future there might be further ways to configure the __env object including extra arguments to scopes or environment variables. The __env.sc however will be the most flexible as you can run arbitrary Scopes code to configure it.

Module Entrypoints

Continuing with the side_quests/package example we show how installed modules can be called directly without specifying the path to them.

In the src/show-mod-main.sc file there are two print statements:

print "always"

if main-module?
    print "only main"

The first will always run and the second will only run when the main-module? value is true.

If you run the module like this you will see both printed:

scopes ./src/show-mod-main.sc

always
only main

However if you just import the module then only the first will run:

scopes -c "import .src.show-mod-main"

always

This is because the main-module? part is only true when the module itself is considered the entrypoint. When you import a module it is not considered the main module, since the main entrypoint is in the module importing it.

The main-module? entrypoint can be used to write tests of library modules, show examples, or even provide a simple application. Modules can exist for the sole purpose of being an application or you can add the main-module? just as an add on to a pure library module. The decision is up to you.

Users that are used to using Scopes could reasonably be expected to run the individual modules as applications, but more typically you would provide wrapper binaries that call out to the individual entrypoints as necessary without needing to prefix them.

A good example of this is the scopes REPL. It can be run from the application front-end scopes, or it can be run from the console module in the standard library:

scopes -m console

This showcases another way to run modules as main without needing the full filesystem path.

Because console.sc is in the standard library on the default search path you simple need to provide the module name.

This is the only example in the standard library of this. In our side_quests/package example we can leverage this by running with the environment to add the package to the search path:

scopes -e -m show-mod-main

Which should print both messages:

always
only main

You must run with the environment or the module will not be found.

Package Management

[2022-03-26 Sat 12:10] <- Environment Configuration

Scopes does not have a dedicated package manager like pip or npm. It may in the future, but in the current year there is a flourishing of language agnostic package managers that could be considered for use such as Nix, Guix, and Spack.

There is of course the distro package managers (like apt, dpkg, rpm, dnf, yum, zypper, xbps, pacman, etc.) which we can’t recommend using for these purposes, as these are oriented towards making a well functioning linux distro rather than an application oriented programming language development environment.

Currently, there is a package repo (snailpacks) for the Spack package manager being maintained including a build recipe for Scopes itself and a variety of C/C++ libraries for game and multimedia development.

Memory Model

An important part of a programming language is understanding when values are copied and/or referenced from the various variable assignments, which is sometimes called the “memory model”.

In languages like C/C++ this might refer to the system dealing with pointers and explicit references, whereas in Python its more about knowing when to use the copy functions and when to just use variable assignment.

Scopes sort of has both and here we will just cover the non-pointer or C++ style references similar to what would be found in Python. Look to Level 2 for more detailed information on references, pointers, copies, and move semantics and how they also relate to the borrow checker.

The TL;DR for this is very straightforward though. All variable assignment works with copies and there is no implicit referencing given the syntax we’ve seen so far.

Below we simply demonstrate this.

let assignment

With let it is immutable and so its simple. You can’t mutate the value so there is no chain of references to worry about.

let a = 3

let b = a

print "a" a
print "b" b

Somewhat obviously, if you redefine an immutable variable as something else, it takes on the new value.

let a = 3
let b = a
let b = 4

print "a" a
print "b" b

And if you redefine the first value ’a‘it doesn’t change the second value ’b’ because they don’t reference each other. b is a copy of the value a initially held:

let a = 3
let b = a
let a = 4

print "a" a
print "b" b

local & global assignment

local and global are exactly the same! But for different reasons.

The reason is that let exists purely at the syntactic level, whereas local and global exist at run time.

Nonetheless for this level you can think of them as the same.

local a = 3

local b = a

print "a" a
print "b" b

local a = 3
local b = a
b = 4

print "a" a
print "b" b

local a = 3
local b = a
a = 4

print "a" a
print "b" b

Other Helpful Miscellanea

Copying

You can copy values with the copy macro:

using import String

local a = (String "hello")

print "a:" a

local b = (copy a)
# local c = b

print "b copied from a:" b
# print "c aliased from b:" c

b = (String "goodbye")

print "b after mutating:" b
print "a after b is mutated:" a
# print "c after b is mutated:" c

;

Running the Interpreter (REPL) from a Module

You can run the “read eval print loop” (REPL) as it is known in the Lisp world from inside any module like this:

from (import console) let
    read-eval-print-loop


let x = 3
let y = 4

fn hello ()
    print "hello"

read-eval-print-loop (.. (globals) (locals)) false

Where the first argument is the scope that you wan’t available in the REPL and the second argument is whether or not to print the logo.

The scope you pass in probably should have the globals defined since that contains the language itself. In this example we also pass in the locally defined variables.

This is very useful for writing debugging modules that allow you to import a bunch of symbols or perform some construction and then drop you into the interpreter to play around with them.

You could also use this to construct an interpreter for different applications where you could include and exclude different parts of globals or redefine it completely.

There is also a third argument for specifying the history of the repl.

from (import console) let
    repl = read-eval-print-loop

do
    let
        x = 3
        y = 4

    repl
        (.. (globals) (locals))
        false
        (.. module-dir "/console.history")

This will make a history file in the same directory as the module.

Note that in this example we run the repl inside a fresh scope. You can use this to carefully craft the scope that is available. In this example the repl symbol that was imported is not passed along to the repl session scope unlike the first example.

Command Line Scripts

Scopes provides a way to use things passed to a program on the command line. The style is more similar to C and so we don’t expect all of this to be understood (i.e. pointers etc.). However this is something basic that you would probably expect to want to do:

let source-path argc argv = (script-launch-args)

print "source-path:" source-path
print "argc (number of arguments in argv):" argc
print "argv (the array of each string argument):" argv
print "argv[0] (the first string in the argument array):"  (string (argv @ 0))

You would run this like:

scopes ./_bin/cli_example.sc Hello

If you don’t provide an argument this will segfault, which you would normally want to handle appropriately.

Final Notes

Where are the Classes?

One of the most important features of Python is the use of defining classes yet we haven’t talked about them at all. Does Scopes have an equivalent?

The answer is yes, and more!

The short answer to the class equivalent is in the use of structs which we will see early on in Level 2 so just continue on to see that.

The longer answer is that because Scopes has extensive support for metaprogramming you aren’t required to use paradigms like Object Oriented Programming and classes to organize your code and data. Furthermore, because of this metaprogramming there are multiple paradigms that can coexist together. So you might find that using classes isn’t actually necessary if you aren’t forced to like in Python.

Files

level_1.org

Latest commit

History

level_1.org

File metadata and controls

Level 1: What you already know, just Scoped

Printing Output

Assignment, Binding, Values & Variables

Operators

Primitives & Simple Expressions

Overview

Special Values

Booleans

Primitive Strings

Integers & Unsigned Integers

Floating Point Numbers

Symbols

Defining Symbols

“Quoting” & Symbols

Operating On Symbols

Lists

Some Context

A Quick But Necessary Tangent: Why the ‘?

List Creation & Manipulation

Some Common Listy Structures

String Construction & Conversions

Converting Arbitrary Value Representations To Strings

Casting C-style “char” Arrays

Arithmetic & Mathematics

Basic Control Flow

Conditionals & Boolean Expressions

Branching Control Flow

Loops

For-Loop

While-Loop

General Loop

Fold-Loop

Switch Statement

“GOTO”

Type Casting

Functions

Defining & Calling Functions

Return Values

Recursion With Functions

Anonymous Functions AKA Lambdas

Modules, Namespaces, & Scopes

Importing Modules

Writing Modules

do-blocks

Executing a module

Modules are Just First Class Scopes

Characters

Data Structures

Tuples

Mutability

Attributes, Methods, and Meta-Methods

Scopes

More Scopes Tricks

Map

Set

Enums

Exceptions, Errors, Assertions, and Premature Program Exit

Error Propagation

How Errors are Different

Assertions

exit

Generators & Iteration

Documentation

Module Loading Mechanism & Configuration

Module Search Paths

Other Search Paths

Configuring the Environment

Module Entrypoints

Package Management

Memory Model

let assignment

local & global assignment

Other Helpful Miscellanea

Copying