Smalltalk: Lost and Found


After Almost 40 years, It’s Still Taking Us to School

Smalltalk is the first and still canonical object-oriented programming language. Alan Kay and other members of the Learning Research Group at Xerox PARC (Palo Alto Research Center) developed Smalltalk in the 1970s. Smalltalk use decreased over the decades, but it’s gaining attention as developers struggling with increasingly complex code look to abandon procedural metaphors for simpler higher level metaphors. We can still learn from Smalltalk even if we don’t use it. (Though we should consider using it.)

When examining Smalltalk we find today’s “object-oriented” tools missed a few essential concepts when they adopted object-orientation. Sacrificing these concepts means developers lost some powerful programming metaphors.

OOP promises organization, simplicity, flexibility, and reusability, yet we often experience disorganized, tangled, inflexible, and duplicate code. Today’s popular multi-paradigm languages make it easier to write procedural code rather than object-oriented code.

Pure object-oriented programming allows building systems with less shared state, less unwanted coupling, and even less code. It reduces time developers need to understand and change code. Readable, decoupled, and testable code means moving fast without breaking things. Ultimately it saves time, and time is money. I like money. You like money too? We should hang out.

Digging In to POOP (Pure OOP)

Smalltalk is an interpreted, dynamically typed, pure object-oriented language with objects all the way down. Multi-paradigm languages like Java and C# (and many others – not picking on Java and C#) are based on procedural programming. They build on primitive types and procedural constructs.

Smalltalk builds everything with objects including so-called “primitive types”. The language itself has only 6 reserved keywords and even those keywords are not built-in language constructs. Wikipedia has a good explanation:

…only six “keywords” are reserved in Smalltalk: true, false, nil, self, super, and thisContext. These are actually called pseudo-variables, identifiers that follow the rules for variable identifiers but denote bindings that the programmer cannot change. The true, false, and nil pseudo-variables are singleton instances. self and super refer to the receiver of a message within a method activated in response to that message… The only built-in language constructs are message sends, assignment, method return and literal syntax for some objects.

The language provides the minimal features necessary to build the rest of its features using objects and messages. Built-in constructs like control structures and loops are absent. Everything is implemented dynamically using late binding and dynamic dispatch.

The Base Difference is Metaphorical

Different levels of abstraction have different base elements that provide the metaphors we use to express program logic. Higher level metaphors hide more state.

A computer processor’s base elements are things like microprocessors, circuits, registers, and bits. A little higher, the processor’s operation codes are the base elements. Above that, assembly language uses mnemonics, directives, and macros. Higher yet, functional programming has functions and immutable values and hide the processor, op codes, and assembly language. Procedural programming hides the same things as functional programming, but its base elements are procedures, flow control structures, and mutable data structures.

Object-Oriented programming’s base element of abstraction is the entire computer: a black box with hidden internal state and data that communicates with other black boxes by sending messages. We don’t build OO programs with things like data, state, loops and conditions. We build OO programs with little computers sending stateless messages to each other. To quote Alan Kay:

…everything we can describe can be represented by the recursive composition of a single kind of behavioral building block that hides its combination of state and process inside itself and can be dealt with only through the exchange of messages. – Alan Kay, The Early History of Smalltalk

What’s the Big Idea?

Surprisingly, objects are not the focus of object-oriented programming. Alan Kay admits it was a mistake naming it “object-oriented” because it causes people to focus on the wrong idea. “Message passing” is the essential and powerful concept in OOP.

In his speech at the 1997 OOPSLA (Object-Oriented Programming, Systems, Languages & Applications) conference, Alan Kay said:

I think our confusion with objects is the problem that in our western culture we have a language that has very hard nouns and verbs in it. Our process words stink. I’ve apologized profusely over the last 20 years for making up the term object-oriented because as soon as it started to be misapplied I realized that I should have used a much more process-oriented term for it. Now the Japanese have an interesting word which is called ‘ma’, spelled in English just ‘m-a’. Ma is the stuff in between what we call objects. It’s the stuff we don’t see because we’re focused on the noun-ness of things rather than the process of things.

– Alan Kay OOPSLA 1997 37min

What he’s saying is that objects are not important. What flows between them is.

Messages flow between objects. Messages are expressions that declare a sending object’s intent without assuming a receiver’s internal state. They don’t represent methods. A message is just a name. An object can send any message to any other object. A receiving object looks for a method with the same name and executes it if it’s found. Messages always pass control to the receiver. Procedural constructs keep control in the calling context, making procedural constructs less reusable and difficult to change in a modular way.

Examples

Some of the Smalltalk code may look familiar as many languages are influenced by it, including AppleScript, CLOS, Dart, Go, Groovy, Java, Objective-C, Perl 6, PHP 5, Python, Ruby, and Scala.

(Keep in mind, these examples don’t show good design practices and only compare the written style of the languages.)

Conditional Statements

Java

if(creditCard.isValid()) {
    sale.finalize();
    // ... other statements...
}
else {
    sale.reject();
    // ... other statements ...
}

Smalltalk
Syntax notes:
“comment” Comments are enclosed in double quotes.

[ ] Square brackets is a block-closure literal, which creates an instance of the BlockClosure class. Think of it as an anonymous function that has closure semantics.

( ) Parentheses are used for grouping.

. A period ends an expression.

object message Unary message that takes no parameters, similar to object.message().

object message: argument or object message: argument1 continueMessage: argument2 are keyword messages; similar to doing object.message(parameter) or object.messageContinueMessage(parameter1, parameter2).

(creditCard isValid)
    ifTrue: [
        sale finalize.
        "... other statements ..."
    ]
    ifFalse: [
        sale reject.
        "... other statements..."
    ].

A procedural if/else control structure is an early-bound construct that inspects a Boolean value and executes the code in one branch or another. The program compiles and executes it in the same context because there is no dynamic dispatch. An if-statement’s branches effectively depend on our code, meaning the construct is not reusable and is a form of duplication. We can tell it’s not reusable because we have to rewrite the construct every time we want to apply conditional logic and fill in different code in the branches.

Every time we write an if-statement we repeat a call to the compiler’s executable code that evaluates conditions. The code evaluates the same way every time, but it’s built-in so the only way to customize conditional logic without dynamic dispatch is to copy the construct and write new branches. It’s like copying and pasting the logic and filling in new information.

Imagine getting a packet of documents to fill out and having to put your name and address on each document. Alternatively, imagine each document references a single document with your name and address on it. There’s less writing and less duplication.

The Smalltalk code sends the ifTrue:ifFalse: message to the Boolean object literal created by isValid. The Boolean object is an instance of one of the Boolean subclasses: the True class or False class. The Boolean object’s value doesn’t enter the calling context. It’s hidden in the object. The receiving object receives the message name, and the message receives the branches (BlockClosure objects) as arguments. Control leaves the calling context and the receiving Boolean object decides whether to invoke one branch or the other based on its value.

Smalltalk’s ifTrue:ifFalse: message is not executable. It’s just a name. The executable code that evaluates conditional logic is stored in only one place: in the matching method belonging to the Boolean object. The message binds to the method and executes it when it’s sent to the Boolean object.

One of pure object-orientation’s beneficial features is late binding everything. The procedural construct opposes this and marks a point where multi-paradigm languages break down. At the point we start the if-statement we leave the domain of objects and messages and talk directly to the compiler; hard-coding branches into the control flow. When we use any construct that talks directly to the compiler we lose the ability to change its behavior dynamically via late binding and polymorphism. This means leaving our high level metaphor for a low level one.

This is why modern language versions replace explicit looping with internal iteration using functional and declarative message metaphors. Why do we have to repeat what a loop looks like and how it’s implemented when we can just pass in the part that varies and reuse the construct?

You might be thinking, “Well, I don’t need conditional logic to use late binding because I have no intention of altering it”. It doesn’t matter. The procedural if-construct changes the metaphors we use. The metaphors available in a programming language reflect on application domain code, changing how it composes. Early binding constructs make code less composable and less reusable. A language that provides a construct promoting duplication reflects on our application code with further duplication.

Smalltalk’s messaging and focus on polymorphism allows greater composability. Programmers can reuse objects in unique ways with less code.

If we wanted to remove the duplication in Java and only pass varying blocks, the best we can do (without creating our own Boolean class) is hide the check inside a class and create methods that control access to the conditional statements. For example:

public class CreditCard {
    public void ifValidDoIfInvalidDo(Runnable validHandler, Runnable invalidHandler) {
            if(this.isValid()) {
                validHandler.run();
            else {
                invalidHandler.run();
            }
    }
}

and use it like this:

creditCard.ifValidDoIfInvalidDo(() -> customer.yey(), () -> customer.badNews());
creditCard.ifValidDoIfInvalidDo(() -> me.throwAParty(), () -> me.sadPanda());

Because objects are ubiquitous in Smalltalk, the condition is already reusable. We could implement the same method like this:

"Method definition in CreditCard class"

ifValid: validBlock ifInvalid: invalidBlock
    (self isValid) ifTrue: validBlock ifFalse: invalidBlock

and use it like this:

creditCard
    ifValid: [ customer yey ]
    ifInValid: [ customer badNews ]

creditCard
    ifValid: [ me throwAParty ]
    ifInValid: [ me sadPanda ]

Sandi Metz, author of Practical Object-Oriented Design in Ruby (the POODR book), did a great talk on object-oriented design at RubyConf 2015. She talks about Smalltalk and if-statements among some other interesting topics. I also recommend her book.

Loops

The rest of the language follows a similar declarative message style. Here are a few looping examples:

For loops
Java

for(int i = 0; i < 5; i++) {
    // Prints “hello” 5 times
    System.out.println(“hello”);
}

for(int i = 1; i <= 5; i++) {
    // Prints numbers 1 through 5
    System.out.println(i);
}

Smalltalk
Syntax notes:

; is a cascade. It allows us to send multiple messages to the same receiver.

'text' single quoted characters are string literals.

Here we send the timesRepeat: message to the number 5 passing an executable block.

The Transcript is the default output object to which we send the text 'hello' and cascade a carriage return message:

"Prints 'hello' 5 times"
5 timesRepeat: [ Transcript show: 'hello'; cr ].

Send the to:do: message to the number 1 passing a stop value 5 and an executable block. The block receives an argument, i, which is the current loop value:

"Prints numbers 1 through 5"
1 to: 5 do: [ :i | Transcript show: i; cr ].

Collections

Java

Arrays.asList(1, 2, 3).forEach((i) -> System.out.println(i));

Arrays.asList(1, 2, 3).stream()
    .filter((i) -> i % 2 == 1) // odd numbers
    .collect(Collectors.toList());

Smalltalk

#(1 2 3) do: [ :each | Transcript show: each; cr ].

#(1 2 3) select: [ :each | each odd ].

How Did We Lose Pure OOP?

Smalltalk became popular in the 80s and early 90s due to its expressiveness, simplicity, and advanced integrated tools (in many ways more advanced still than many tools we have today).

Classes and objects were used before Smalltalk in languages like Simula (1960s), but Smalltalk was the first language to exclusively use objects as the ubiquitous building block of programming, thus Alan Kay coined the term “object-oriented”.

Languages like C++ incorporated classes and objects, but C++ is not “object-oriented” in the way OO was intended. Even Alan Kay said “… I made up the term “object-oriented”, and I can tell you I did not have C++ in mind.” – Alan Kay, Speech at OOPSLA 1997

Some people (Dr. David West, author of “Object Thinking”) assert that Sun created Java specifically to kill Smalltalk because they were bitter about having to pay license fees. He tells the story in this video https://vimeo.com/77415896. Smalltalk was in high demand by companies like Sun and Apple, but the high licensing fees and royalties Xerox required convinced companies to seek alternatives. Sun eventually created Java which became popular because it had a familiar C-style syntax and because it added networking and web tools in a time when the web was becoming popular. Smalltalk was slower in moving into the web era.

The familiarity we gain in procedural languages is overshadowed by what we lose in expression. Kay’s thoughts on languages building in familiar procedural abstractions:

Once you have encapsulated in such a way that there is an interface between the inside and the outside it is possible to make an object act like anything. The reason is simply this: that what you have encapsulated is a computer. So you’ve done a powerful thing in computer science, which is to take the powerful thing you’re working on and not lose it by partitioning up your design space. This is the bug in data, procedures and data and procedural languages. I think this is the most pernicious thing about languages like C++ and Java, is that they think they’re helping the programmer by looking as much like the old thing as possible, but in fact they are hurting the programmer terribly by making it difficult for the programmer to understand what’s really powerful about this new metaphor.
– Alan Kay OOPSLA 1997 39min

Most of the languages created in Smalltalk’s wake overlooked the powerful metaphors Smalltalk gives to programmers, provided by message passing. Adding classes and objects do not alone make a language object-oriented. The OO buzzword, however, was still attached to products and was vital in marketing them to compete with Smalltalk. What sells better than buzzwords? (The answer is hotcakes.)

Marketing multi-paradigm languages as object-oriented eventually changed what we think object-oriented programming looks like. Presently, it looks like procedural programming with the same modularity and scalability issues that prompted development of higher level metaphors in the first place.

Countless other languages have this multi-paradigm-OO-schizophrenia. Multi-paradigm languages either force us, or make it easy to fallback to using procedural abstractions, which means we have yet to truly benefit from object-orientation.

Throw Away All-The-Things? No

So what? Are we supposed to throw away our languages? Of course not. Java, C#, Python, Ruby and many other languages are great, productive languages. However, complexity is our enemy and procedural metaphors make systems more complex. Recognizing the procedural pitfalls in our languages helps us write and refactor code to make it more flexible and reusable.

Thanks for reading! You win a pretzel! I’m out of pretzels. You win nothing.

Resources

The Deep Insights of Alan Kay

The Early History of Smalltalk by Alan Kay

Wikipedia: Smalltalk

Why Every Ruby Developer Should Learn Smalltalk

If-statements in Smalltalk

Advertisements

3 thoughts on “Smalltalk: Lost and Found

  1. Fascinating, thanks for writing this Pete (2+ years ago).

    I don’t get two things, and they’re the biggest two things:

    1) I learned assembly at a young age, so I’m always conscious of the instructions a CPU eventually has to run. I don’t get how a for loop isn’t, eventually, a procedural thing with a counter, a boolean test, and a branch (if). Some object deep down in the hierarchy (or late in the message passing?) has got to be doing a loop, right?

    2) I don’t get how “what you have encapsulated is a computer”. In what sense is each object like a complete computer? In that it takes and input, does something, and produces an output?

    • Hi gbell,

      Thanks for reading and for the great questions.

      1)
      This makes me think of a quote by Adele Goldberg, one of the Xerox PARC Smalltalk-80 team members, who said, “In Smalltalk, everything happens somewhere else.” Joking aside, you’re absolutely right. At some point, Smalltalk has to do the actual work, and the environment has to interface with the machine. The Smalltalk environment is incredibly clever and the language is unique, but like any high-level language, the code we write is a lie. A very useful and powerful lie, but a lie nonetheless. There is a lot of plumbing underneath the language to hide the fact that at some point, as you said, there are procedural things, boolean tests, and branches.

      I can’t pretend to know all the details of how this works because I am the farthest thing from a Smalltalk expert, but I do know a little and after doing some extra research I can tell you the following. The Smalltalk image runs on a virtual machine. Methods are compiled to bytecode and the interpreter processes the bytecode using stack-based activation records. The virtual machine is programmed to interface with the target platform, and the hardware, as you know is just shuffling bits in registers. It’s all very similar to Java (or probably more correctly, Java is very similar to Smalltalk).

      For example, the Number>>to:do method in Pharo Smalltalk, is implemented as:

      to: stop do: aBlock
      “Normally compiled in-line, and therefore not overridable.
      Evaluate aBlock for each element of the interval (self to: stop by: 1).”
      | nextValue |
      nextValue := self.
      [nextValue <= stop]
      whileTrue:
      [aBlock value: nextValue.
      nextValue := nextValue + 1]

      In bytecode this looks like the following…
      17 self
      18 popIntoTemp: 2
      19 pushTemp: 2
      20 pushTemp: 0
      21 send: <=
      22 jumpFalse: 34
      24 pushTemp: 1
      25 pushTemp: 2
      26 send: value:
      27 pop
      28 pushTemp: 2
      29 pushConstant: 1
      30 send: +
      31 popIntoTemp: 2
      32 jumpTo: 19
      34 returnSelf

      Some background on what we’re looking at… Thanks to Mariano Peck’s blog (link at bottom) we know arguments and temporary variables are numbered positionally, so stop -> 0, aBlock -> 1, nextValue -> 2. The compiled method is an array of bytes. The first column below (also called the program counter) is each instruction’s position in the array. It starts at 17 because there is another frame before it that is not shown. The second column is the hexadecimal bytecode instruction, and the third column is a human-readable annotation.

      Here it is with some annotations and the numerical temp variables replaced with their names…
      17 self // push self on stack
      18 popIntoTemp: nextValue // assign self to nextValue and pop stack
      19 pushTemp: nextValue // push nextValue on stack
      20 pushTemp: stop // push stop on stack
      21 send: <= // send <= to nextValue with stop as argument
      22 jumpFalse: 34 // exit loop at line 34 if false
      24 pushTemp: aBlock // push aBlock on stack
      25 pushTemp: nextValue // push nextValue on stack
      26 send: value: // send value: message to aBlock with nextValue as argument
      27 pop // ignore the return value
      28 pushTemp: nextValue // push nextValue on stack
      29 pushConstant: 1 // push literal 1 on stack
      30 send: + // send + message to nextValue with 1 as argument
      31 popIntoTemp: nextValue // assign res1ult to nextValue and pop stack
      32 jumpTo: 19 // start again from the loop condition at line 19
      34 returnSelf // return the receiver of to:do:

      Some methods, for example, <= have primitive annotations for the interpreter to supersede the bytecode and try to have the virtual machine execute optimized code…
      <= aNumber
      "Primitive. Compare the receiver with the argument and answer true if
      the receiver is less than or equal to the argument. Otherwise answer
      false. Fail if the argument is not a SmallInteger. Optional. No Lookup.
      See Object documentation whatIsAPrimitive. "

      ^super <= aNumber

      As they describe in the Pharo by Example book, "In Pharo everything is an object, and everything happens by sending messages. Nevertheless, at certain points we hit rock bottom. Certain objects can only get work done by invoking virtual machine primitives." (4.7 Primitives and Pragmas)

      2)
      My understanding is an object is like a computer in the sense that it is a black box that can receive messages, interpret the messages, and respond, all without the sender knowing how it was implemented and without having to know the internal state of the object.

      When creating Smalltalk, Alan Kay was influenced by his background in biology and his work with ARPA and early computer network technology. In biology, cells communicate by sending chemical messages. Cells that interact with the chemical message each have their own response implementation. If biology were like procedural programming a cell who wants to communicate with another cell would have to check what type of cell they are talking to and what the current state of the other cell is, and then decide what command to send it. This requires all senders to change any time a new type of receiver is introduced or the structure of a receiver changes.

      Alan Kay's experience with computer networks brought him to this conclusion: "…I realized that the bridge to an object-based system could be in terms of each object as a syntax directed interpreter of messages sent to it… The mental image was one of separate computers sending requests to other computers that had to be accepted and understood by the receivers before anything could happen. In today's terms every object would be a server offering services whose deployment and discretion depended entirely on the server's notion of relationship with the servee." (Early History of Smalltalk)

      So the mental model for objects are computers communicating similar to the way they communicate on a network. Computers don't know what type of computer they are talking to, and they don't check the internal state of the computer before talking to it. They have agreed-upon protocols and send messages to each other.

      Here are some other good quotations from The Early History of Smalltalk (link at bottom), which express the OO/Smalltalk philosophy.
      "What I got from Simula was that you could now replace bindings and assignment with goals. The last thing you wanted any programmer to do is mess with internal state even if presented figuratively. Instead, the objects should be presented as sites of higher level behaviors more appropriate for use as dynamic components."

      "Where does the special efficiency of object-oriented design come from? This is a good question given that it can be viewed as a slightly different way to apply procedures to data-structures. Part of the effect comes from a much clearer way to represent a complex system. Here, the constraints are as useful as the generalities. Four techniques used together—persistent state, polymorphism, instantiation, and methods-as-goals for the object—account for much of the power. None of these require an "object-oriented language" to be employed … an OOPL merely focuses the designer's mind in a particular fruitful direction. However, doing encapsulation right is a commitment not just to abstraction of state, but to eliminate state oriented metaphors from programming.

      Perhaps the most important principle—again derived from operating system architectures—is that when you give someone a structure, rarely do you want them to have unlimited privileges with it. Just doing type-matching isn't even close to what's needed. Nor is it terribly useful to have some objects protected and others not. Make them all first class citizens and protect all."

      I hope I've answered your questions. I'm happy to discuss further. Let me know, and thanks again for your questions!

      Resources
      http://sdmeta.gforge.inria.fr/FreeBooks/BlueBook/Bluebook.pdf
      https://marianopeck.wordpress.com/2011/05/21/introduction-to-smalltalk-bytecodes/
      http://worrydream.com/EarlyHistoryOfSmalltalk/

Comments!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s