2014/1/05

On using Go channels like Python generators

I really like Go, and I really like Python, so it’s natural that I apply patterns I’ve learned in Python to Go… even when it’s maybe not appropriate.

For example, in Python, it’s common to use generators to process a stream of data. Take this simple (unrealistic, inefficient) odd number generator:

>>> def odds(from_, to):
...     for n in range(from_, to):
...         if n % 2:
...             yield n
... 
>>> print(*odds(0, 10), sep=", ")
1, 3, 5, 7, 9

How would you do something like that in Go? The natural approach is to use channels. Go’s channels are a language primitive that allows you to send values from one light-weight green-ish thread (“go-routine”) to another. Many other languages would make these into methods in a standard library threading package, but since they’re language primitives in Go, they get used more often, even in simple programs.

Here is an incorrect first attempt at simulating generators using channels:

package main

import "fmt"

func Odds(from, to int, c chan int) {
    if from%2 == 0 {
        from += 1
    }
    for i := from; i < to; i += 2 {
        c <- i
    }
}

func main() {
    c := make(chan int)
    go Odds(0, 10, c)
    for v := range c {
        fmt.Println(v)
    }
}

This program ends with fatal error: all goroutines are asleep - deadlock! because the loop in for v := range c never quits. It never quits because the channel is never closed. An even worse version of this would be one where go Odds(0, 10, c) was just Odds(0, 10, c). That ends with a deadlock before even printing anything.

What is a non-terrible version of this code? We should move more of the responsibility into the Odds function. The Odds function should be in charge of creating the channel, spawning a go-routine, and closing the channel when the go-routine is done, so that the caller can be freed of that responsibility.

Here’s what that looks like:

package main

import "fmt"

func Odds(from, to int) <-chan int {
    c := make(chan int)
    go func() {
        defer close(c)
        if from%2 == 0 {
            from += 1
        }
        for i := from; i < to; i += 2 {
            c <- i
        }
    }()
    return c
}

func main() {
    for v := range Odds(0, 10) {
        fmt.Println(v)
    }
}

There are a couple of thing to notice here. First, the caller doesn’t need to use the go statement because the Odds function now has an anonymous function inside of it that it uses to spawn a go-routine. (This use of an anonymous function looks a bit like Javascript to me.)

Second, the caller doesn’t need to create its own channel to give to our pseudo-generator. Not only that, since the type of the channel is <-chan int, it’s perfectly clear that the channel is only for receiving ints, not for sending, and this is enforced at compile time.

This leads to the most important point: that it doesn’t deadlock at the end! That’s because the channel gets closed. Generally speaking, in Go the one doing the sending on a channel should also be the one to do the closing of the channel when it’s done sending. (Double closing a channel causes a runtime panic.) Since the Odds function returns a receive-only channel, the compiler actually enforces that the close cannot be done in the main function.

But notice how it gets closed, with defer close(c). In Python, the with statement is used for resource management things like remembering to close files and sockets. In Go, defer is used to ensure that a certain command happens whenever a certain function ends (whether in a normal return or an exception-like panic). As a convention, it’s a good idea to defer the close of something as soon as it’s applicable.

So far so good, but what about this scenario:

func main() {
    for v := range Odds(0, 10) {
        fmt.Println(v)
        if v > 5 {
            fmt.Println("Tired of counting all day.")
            return
        }
    }
}

The channel returned by Odds wasn’t exhausted, so it was never closed. Since the function returning was main, this isn’t a big deal in this case, but this is actually a somewhat subtle bug that can affect otherwise correct-looking code. It’s a memory leak, or more specifically a go-routine leak. Python and Go are both garbage collected, but there’s a subtle difference here. In Python if I write gen = odds(0, 10) and then next(gen) the resulting generator will be cleaned up shortly after gen goes out of scope. Similarly, in Go the channel created by c := Odds(0, 10) will be garbage collected once all references to it go away. Go-routines, however, only go “out of scope” when they return or panic, since it can have side-effects independent of whether anything in the main thread has a reference to it. (E.g.)

The standard way to prevent go-routine leaks is through the use of “quit channels.” However, this looks like it’s just going to re-introduce a lot of the complexity that we got rid from the first/incorrect example. We need to make up a new channel to pass around, figure out what value to send as a quit signal, etc.

A quit channel is more complicated than just letting a go-routine leak or run to exhaustion, but there are a few tricks we can take advantage of to make it less annoying:

package main

import "fmt"

func Odds(from, to int, quit <-chan struct{}) <-chan int {
    c := make(chan int)
    go func() {
        defer close(c)
        if from%2 == 0 {
            from += 1
        }
        for i := from; i < to; i += 2 {
            select {
            case <-quit:
                return
            case c <- i:
            }
        }
    }()
    return c
}

func main() {
    quit := make(chan struct{})
    defer close(quit)
    for v := range Odds(0, 10, quit) {
        fmt.Println(v)
        if v > 5 {
            fmt.Println("Tired of counting all day.")
            return
        }
    }
}

First of all, what is <-chan struct{}? type struct {} is a type for hanging methods off of with no associated data. chan struct{} is a channel that passes no data. You could also write chan bool but that has two potential values, true or false. struct{} lets your readers know, “There’s no data being passed here.” The only reason for the channel to exist is to synchronize things. No other information is being sent. The arrow in front of <-chan struct{} further specifies that the caller is the one who will be in charge of the quit channel, not the function.

Next, select (not to be confused with switch, which also has cases) is a statement in Go that lets you send or receive from multiple channels at once. If multiple channels are ready simultaneously, the runtime will pseudo-randomly select just one to use, otherwise the first channel to be ready will fire. In this case, it means “either send the value of i on the output channel or return if the quit channel has something.”

When you close a channel, from then on anyone listening to that channel receives the blank value for that type (0 for int, "" for string, etc.) Instead of using close, you could also send an empty struct:

func main() {
    quit := make(chan struct{})
    for v := range Odds(0, 10, quit) {
        fmt.Println(v)
        if v > 5 {
            fmt.Println("Tired of counting all day.")
            quit <- struct{}{}
        }
    }
}

close(quit) is less ugly than quit <- struct{}{}, but the real reason I think it’s better to use close gets at my next couple of points.

Why didn’t I put the make for the quit channel into Odds like I did for the chan int in the second example? It certainly could be done, but fundamentally a caller knows better than the function itself when the go-routine should quit, so the caller should have the responsibility. This lets us do shortcuts like this:

func main() {
    for v := range Odds(0, 10, nil) {
        fmt.Println(v)
    }
}

Since the caller knew the channel would be drained to exhaustion, it didn’t need to make a quit channel at all. In Go, nil channel never sends, but there’s nothing illegal or panic-inducing about listening to a nil channel, so Odds works just fine.

Furthermore, we can reuse the same quit channel for multiple functions. If we were sending the quit signal with quit <- struct{}{}, we’d have to send one message for each go-routine that reuses the same quit channel and needs to be closed. However because closed channels are always “sending” their blank value, we can stop all of the go-routines that are listening to the same quit channel by just doing close(quit) once.

Put it all together and this approach allows for more advanced (and Python generator-like) usages of channels. The second Project Euler problem is “By considering the terms in the Fibonacci sequence whose values do not exceed four million, find the sum of the even-valued terms.” Here is an over-engineered solution using Go’s channels:

package main

import "fmt"

func sendOrFail(n int64, out chan<- int64, quit <-chan struct{}) bool {
    select {
    case out <- n:
        return true
    case <-quit:
        return false
    }
}

func FibGen(quit <-chan struct{}) <-chan int64 {
    out := make(chan int64)
    go func() {
        defer close(out)
        var last, current int64 = 1, 2

        if ok := sendOrFail(last, out, quit); !ok {
            return
        }
        for {
            if ok := sendOrFail(current, out, quit); !ok {
                return
            }
            last, current = current, last+current
        }
    }()
    return out
}

func Evens(in <-chan int64, quit <-chan struct{}) <-chan int64 {
    out := make(chan int64)
    go func() {
        defer close(out)
        for n := range in {
            if n%2 == 0 {
                if ok := sendOrFail(n, out, quit); !ok {
                    return
                }
            }
        }
    }()
    return out
}

func TakeUntil(max int64, in <-chan int64, quit <-chan struct{}) <-chan int64 {
    out := make(chan int64)
    go func() {
        defer close(out)
        for n := range in {
            if n > max {
                return
            }
            if ok := sendOrFail(n, out, quit); !ok {
                return
            }
        }
    }()
    return out
}

func main() {
    var total int64

    quit := make(chan struct{})
    defer close(quit)
    for n := range Evens(TakeUntil(4000000, FibGen(quit), quit), quit) {
        total += n
    }
    fmt.Println("Total:", total)
}

Compare to similar Python code:

>>> def fibgen():
...     last, current = 1, 2
...     yield last
...     while True:
...         yield current
...         last, current = current, last + current
... 
>>> def take_until(max, gen):
...    for n in gen:
...       if n > max:
...           return
...       yield n
... 
>>> def evens(gen):
...    for n in gen:
...       if n%2 == 0:
...           yield n
... 
>>> sum(evens(take_until(4000000, fibgen())))
4613732

The Go version isn’t as simple as a basic approach using Python generators. It’s 73 lines versus only 20 lines of Python. (15 lines just for all of those closing }.) Still, Go has certain advantages, like being concurrent and type safe. In real code, you could imagine making and processing multiple network calls concurrently using techniques like this. The lowest level generator opens a socket to get some data from somewhere on the internet, then it pipes it results out to a chain of listeners that transform the data. In a case like that, Go’s easy concurrency could be an important advantage over Python.

2013/7/31

Bret Victor - The Future of Programming

"Hey Steve, it’s Marvin. Your cousin, Marvin Wozniak. You know that new information paradigm you’ve been looking for? Well listen to this!"

Via GvR

2013/2/11

Google Go: The Good, the Bad, and the Meh

“Go is not meant to innovate programming theory. It’s meant to innovate programming practice.” — Samuel Tesla

Normally, I just blog about… uh, stuff?… but I feel like writing about the Go programming language (or “Golang”) today, so instead today’s topic is computer stuff. For the record, the language I’ve programmed the most in has been Python, so that’s the perspective I’m analyzing it from.

The Good

One of the best things about Go is that it was clearly designed for working on large projects in a team with a modern version control system.

For example, one of the “controversial”† aspects of Go is that it uses case to specify visibility. Instead of having a system of public, private, and shared class members, visibility is at the level of the package. Lowercase names like var a are visible only from the current package, but uppercase names like var A are publicly visible, and so on for types, functions, struct members, methods, etc. It is blisteringly obvious in hindsight that enforcing visibility at the package level instead of the class level is the right way to do it. Who cares if the other class poking around in your class’s internals is your subclass or whatever? The question is whether it’s part of the same package, in which case, you can be expected to keep the classes in sync since you wrote them both. If it’s in a different package, then the other person wants to know what’s in the public facing “API” of the class. In Python something similar is done with the convention of appending _ to the front of names that are meant to be private, but Go’s convention is simpler and easier. It does take a little bit of time getting used to the idea that in name.Name the first thing is definitely a package name and the second thing may or may not be a type (unlike Python, where classes are usually the only thing with CamelCase names), but once you adjust, it’s fine and leads to better API design. When you write something, if you want someone importing your package to use it, use uppercase. If you want them to leave it alone, lowercase. Done.

† Controversial with [Internet controversialists]. People who complain about this are complainers. Features like this are useful because they let you know who likes to complain about things for no good reason. Go has a lot of features like this. [Edit: Based on comments, I have changed the word “idiots” to “Internet controversialists” in reflection if the fact that you can be a smart person and still love debating the color of the bike shed.]

Another example of Go’s modernity is the decision to make the directory the fundamental unit of packaging. While you can just compile and run an individual file using go run filename.go, Go encourages you to set the environmental variable $GOPATH and then put your projects into $GOPATH/src/projectname/. When you run go install project, it will compile that directory and put the executable into $GOPATH/bin/. Why is this useful? It leads to a couple of nice advantages. First, since it’s done by directory, you can make each package you write its own git (or whatever) repository. Second, if your project file gets too long, you can break it up into multiple files that automatically get stitched into one at compile time. Not a big deal, but nice. Third, there’s no need for something like Python’s virtualenv. If you need to have different versions of the same library installed simultaneously for two different projects, just switch between two different $GOPATHs. Fourth, there’s no question of Python’s relative import headaches. Imports are always either rooted in $GOPATH/src/ or in the built-ins ($GOROOT/src/pkg/).

Fifth, this leads to the convention of using domain names as directory names. So, if you have a github account, you’d store your local stuff in $GOPATH/src/github.com/username/project/. Now, the genius of this is that the go tool has the ability to download from most (all?) modern VCS systems. So, if you run go get github.com/userA/project/ it will use git to download the project from github.com and put it into the right place. Even better, if that project contains the line import "bitbucket.com/userB/library", the go tool can also download and install that library in the right place. So, in one stroke, Go has its own elegant solution to packaging and a modern, decentralized version of CPAN or PyPI: the VCS you were already using!

Deployment in Go is very easy too, since everything compiles down to a single binary. It’s practical to cross-compile things, so that you don’t have to install anything Go-related on your production server except the one binary you compiled on your development machine. That greatly simplifies the issue of library management.

The go tool in general can’t be praised enough. It compiles quickly (sorry XKCD fans), and it comes with a couple of awesome features. First, go fmt. Gofmt is also “controversial” with [Internet controversialists]. It’s great. It automatically makes your code line up with the Go standard. Some people complain that the standard doesn’t put the braces where they want them. Those people are why we need gofmt. Gofmt basically does for Go what syntactic whitespace does for Python: it makes sure that the average code you run into is written to be readable.

Second, go doc. Godoc autogenerates documentation based on the comments in your code. Since Go is a compiled language, I tend to treat it as a replacement for using help(whatever) at the REPL in Python. It works well. It only tells you about exported things (things with Uppercase names), but that’s all you should need to know if the API works. It has a couple of modes for giving you the docs, the most helpful of which is a local webserver that gives you the docs for everything you have installed in your current $GOPATH.

Third, go test. It’s a built in testing and benchmarking framework. It’s not mind blowing, but it works.

At this point, I suppose I should say something about the Go language itself. What I like about Go is that it is very readable. Let’s say I want to declare a pointer to a variable length array (what Python calls a list and Go calls a slice) of pointers to FooType objects:

var foo *[]*FooType

It reads very simply, left-to-right: “Variable ‘foo’ is a pointer to a ‘slice’ of pointers to FooType objects.” From the perspective of languages in the C-Java continuum, this is backwards, since the type follows the variable name, but once you get used to it it makes much more sense than vice versa. Functions also read very naturally:

func Foo(x int, y *Bar) error { ... }

"Function foo takes x, an int, and y, a pointer to a Bar, and returns an error (or a nil of type error).”

Of course, the two big pains of static typing are that sometimes you have to write out the type stuff even when it’s obvious what it has to be and that sometimes you just can’t easily do what you want to do. Fortunately, Go has basic type inference, so the first problem is not as big of problem. The := operator is used when you want the compiler to just figure out the type stuff for itself. So, in err := Foo(1, &Bar{}), the compiler knows that I want err to be of type error, since that’s the type that function Foo returns. Similarly, the second problem can be routed around by using the universal type interface {} and the runtime reflection tools. Throw in a broad enough interface and the compiler will usually just shut up and let you shoot yourself in the foot if you really need to. (But see also my complaint about generics in the “Bad” section below.)

Go also has a simpler solution to the whole nonlocal headache in Python. Since Python doesn’t have a way to declare a variable apart from giving it a value with =, there’s a question of what you want to do in the case of a closure: how do you know which variables to initialize and which not? The Python solution of nonlocal is inelegant, but Go’s closures are straightforward. Adding := would be a great simplification for Python 4000, but based on what I’ve read in the mailing lists, it’ll never happen.

Here’s a complete Go solution to Paul Graham’s Accumulator Generator Challenge:

package main

import "fmt"

func accgen(n int) func(int) int {
    return func(i int) int {
        n += i
        return n
    }
}

func main() {
    f := accgen(0)
    fmt.Println(f(1), f(1), f(1)) //Prints 1, 2, 3
}

The static typing adds a little noise, but it makes sense if you read it out loud: “Function accgen takes an int n and returns a function that takes an int and returns an int.” It can be confusing, but no more so than the concept of an accumulator in the first place.

Incidentally, another nice feature of Go is the Go Playground linked to above. It’s a combination of pastebin and a sandbox, and it helps ease the pain of not having a REPL. If you wonder, “Will this work in Go?” you can write a quick proof of concept on the Playground, run it (on Google’s server, which means it works on an iPad or whatever), then press “Format” to pretty it up and “Share” to generate a link to it.

In general, I really like the attitude that Go’s designers have towards the type system. For example, they straightened out the confusing mess of longs and shorts that had accumulated over thirty years with straightforward names: int8, int16, int32, and int64. There are also unsigned variants (uint8, etc.). By itself, int is 32-bits on most systems now, but they plan to change it to be 64-bits in future versions. Go also makes it easy to use the type system to do rudimentary sanity checks. For example, you can easily define a type like this:

type celsius float64

And then you can write your functions so that they only take celsius and not fahrenheit temperatures.

Or if you want a simple options bit-flag, you can do something like this:

package main

import "fmt"

type BFlag uint8

const (
    Grapes BFlag = 1 << iota
    Apples
    Bananas
    Mangos
)

func OrderBreakfast(flag BFlag) {
    if flag&Apples != 0 {
        fmt.Println("Apples it is!")
    } else {
        fmt.Println("All we had was apples.")
    }
}

func main() {
    f := Grapes | Apples | Bananas
    OrderBreakfast(f)
}

iota is a semi-magical variable for use in constant declarations that goes up by one each time. Since the other fruits are in the same constant declaration block but don’t have anything declared, Go automatically gives them the same type and assignment as the first constant (BFlag and 1 << iota), so Grapes is 0001, Apples is 0010, Bananas is 0100, and Mangos is 1000. The bit flag is simple enough, but since BFlag is its own type, you don’t have worry about someone accidentally sending a number to OrderBreakfast that’s just the result of adding up the tabs or something.

The designers of Go agree with the collective experience of the last twenty years of programming that there are three basic data types a modern language needs to provide as built-ins: Unicode strings, variable length arrays (called “slices” in Go), and hash tables (called “maps”). Languages that don’t provide those types at a syntax level cannot be called modern anymore. (And what’s up with all the languages that claim all you need are linked lists? I’m sorry, this is not 1958, and you are not John McCarthy.) Go strings are UTF-8 because Go was designed by the guys who invented UTF-8, so why not?

The designers of Go also seem to have come to the conclusion that object oriented programming had some good ideas but was mostly overblown. So, they don’t provide inheritance. (!) But before you freak out, however, they do provide embedding and interfaces, which make it possible to do most of the stuff you would want to do with inheritance anyway. I think at this point, it’s well accepted that multiple inheritance was mostly a headache for little payoff, so Go just takes that insight to the next level. If you just want to copy the behavior of a class with a few overrides, it’s pretty simple.

Here’s an example that illustrates what I mean:

package main

import "fmt"

type FooBaz interface {
    Foo()
    Baz()
}

First I declare the FooBaz interface. Something is a FooBaz if it can Foo() and Baz(). Note that this is basically statically compiled duck-typing. All you need to do is have these methods and you’re in. There’s no need to declare that you implement FooBaz beyond just implementing it.

type A struct{}

func (a A) Foo() {
    fmt.Println("I'm an A! Foo.")
}

func (a A) Baz() {
    fmt.Println("I'm an A! Baz.")
}

Next I declare the A type. Just for simplicity, I make it an empty struct. Then I give A two methods, Foo and Baz. The way you add a method to a type is like a function declaration, except that before the function name (method name), you put the type in parentheses as well as a name for the receiver (like this or self in other languages). Unlike Python’s convention of always using self, idiomatic Go doesn’t always use the same receiver name, but instead the receiver name tends to be based on the type’s name. In this case, my receiver name is a, but I never use it (and type A is an empty struct anyway, so I can’t modify its instances), so it doesn’t really matter. Because methods are treated basically like functions with the privilege of overloading the type of the first argument (Go doesn’t allow other forms of overloading), Go lets you put method names wherever you want, as long as its in the same package as the type its the method of.

type B struct {
    A
}

func (b B) Baz() {
    fmt.Println("I'm a B! Baz.")
    fmt.Println("I'm also an A!?")
    b.A.Baz()
}

Here’s where it starts to get interesting. B is a struct with an A embedded in it. That means when you call a B object with .Foo that call gets passed into the A that the B is holding, since there is no B.Foo method. On the other hand, since there is a B.Baz method, it calls that when you do .Baz() on the object. The Baz method then calls the A in itself (like a superclass call in inheritance languages). Of course, this isn’t required, but I just do it to show that you can.

func FooAndBaz(f FooBaz) {
    fmt.Println("Calling!")
    f.Foo()
    f.Baz()
}

func main() {
    var a A
    var b B
    FooAndBaz(a)
    FooAndBaz(b)
}

Finally, we have a function that takes anything that fulfills the FooBaz interface, so I send it a and b which are of types A and B. The final output is:

Calling!
I'm an A! Foo.
I'm an A! Baz.
Calling!
I'm an A! Foo.
I'm a B! Baz.
I'm also an A!?
I'm an A! Baz.

This doesn’t have all the complexity and nuances of a true inheritance system… And that’s a good thing! If you’re just trying to override a couple of methods, it’s easy to do and easy to understand. If you want the nuance and complexity of a traditional OO inheritance system, you can specify exactly what you want it to do yourself.

Interfaces are one of the key features of Go. As I said before, it’s basically compile time duck typing. As long as you have walk and quack methods with the right signatures, good enough.

The final good thing to mention about Go is the concurrency. It works like you thought concurrency should work before you learned about concurrency. It’s nice. You don’t have to think about it too much. If you want to run a function concurrently, just do go function(arguments). If you want to communicate with the function, you can use channels, which are synchronous by default, meaning they pause execution until both sides are ready.

A simple example:

package main

import "fmt"

func printer(ch chan int, done chan struct{}) {
    for i := range ch {
        fmt.Println(i)
    }
    done <- struct{}{}
}

func main() {
    ch := make(chan int)
    done := make(chan struct{})
    go printer(ch, done)
    ch <- 1
    ch <- 2
    ch <- 3
    close(ch)
    <-done
}

The printer function prints the values it receives from a channel of ints. The main function spawns the printer as its own goroutine (green thread, basically) then sends it some work. There’s some nuances to this (for example, in this case, 2 won’t finish sending until the printer is ready for it), but again, it basically works like you think it should. It makes it fairly simple to get things go concurrently and/or in parallel without ripping your hair out. Since execution halts when the main function is complete, I threw in the done channel as a quick and dirty way to make sure that everything gets printed before the program ends. You could also use a sync.WaitGroup in a more realistic example.

The Bad

Those are the things I like about Go. What do I dislike?

This is just a small thing, but for some reason, Go doesn’t suppose the use of a #! on the starting line, which means you can’t just use uncompiled files as utility scripts.

In general, I think that Go tends not to be very DRY. Go is meant to make performance issues pretty obvious, so it doesn’t have a lot of sugar for things. Where this especially shows up is in the lack of generics. Idiomatic Go code has a couple of different tricks for using interfaces so that you don’t need generics… but sometimes you just have to bite the bullet and copy-and-paste a function three times so you have three differently typed versions of it. If you’re trying to make your own data structure, you can just use interface {} as a universal object type, but then you lose compile time type safety. If you want to make a universal sum function, on the other hand, there’s no really good way to do it. Probably the easiest thing is to add a header like type MyNum int32 at the top, write your functions to take MyNums, and then if you later decide to use a float instead, at least you don’t have to rewrite the type of all of your functions, but that’s not as convenient as real generics.

Along the same lines, Go can be a little too low level in spots. For example, while it has a string type and all of the same ways of manipulating strings as Python, in Go (like in really old versions of Python), these are functions in the strings package instead of just being methods on string objects. I guess they do this for performance/memory reasons,‡ but it seems chintzy to me.

‡ People sometimes complain that a basic “Hello World!” program in Go is too large (like 2MB or something, IIRC), but that’s because it statically compiles in the fmt package, which has Unicode support. If all strings required the equivalent of the strings package, Hello World would be even bigger.

Finally, Go is still a new language, which sometimes means that library you need hasn’t been implemented yet or it’s still too immature for practical use. Fortunately, this problem is rapidly solving itself.

The Meh

What am I passionately neutral about in Go?

Some people get very worked up about how Go “doesn’t have exceptions.” First of all, this is untrue. Go has panic, which works almost exactly like an exception. However, idiomatic Go code isn’t supposed to use panic for expected exceptions. Instead you’re supposed to return an error-type object along with your normal return type using multiple return values. In practice, Go’s error handling system is basically the same as Java’s. panic is like an unchecked exception: divide by zero in the wrong place and the whole stack blows up on you. error objects are like checked exceptions: if something you call returns an error you either have to ignore the error entirely (bad), do something about it (good), or pass the buck to someone else (neutral). The only difference is that these response are respectively written as

//Ignore
value, _ = package.MightReturnAnError()

//Deal with it
value, err = package.MightReturnAnError()
if err != nil {
    do something 
    ...
}

//Pass the buck
value, err = package.MightReturnAnError()
if err != nil { return nil, err }

instead of being written in terms of try and catch or except. I do wish the compiler was a little stricter though. It’s perfectly legal to write

fmt.Println("Hello")

even though Println returns an integer and an error. I sort of wish they made you explicitly silence the error by writing _, _ = fmt.Println("Hello") instead. Then again, maybe that would be a pain? On top of that, since basically everything returns errors of the interface type error, it can sometimes be a little hard to figure out exactly which concrete errors something will return. For example, the documentation on fmt.Println says “It returns the number of bytes written and any write error encountered.” OK, but which exact errors should I be prepared to deal with?

That brings me to my next bit of neutralness: the Go compiler has no warnings, only errors. Now, obviously, this is The Right Thing to Do. We all know that a compiler warning ought to be treated like an error, and ignoring warnings is wrong, and it’s better for the compiler to just force you to fix it than for it to enable you to do the wrong thing. …But sometimes you really want to do the wrong thing anyway. The other day there was a big stink online about how the Go compiler won’t let you have any unused variables or unused imports. Technically speaking, this isn’t true. If you just throw in some _, the compiler will shut up and let you import stuff you don’t use or create variables and never use them. On the other hand, it is kind of annoying having to shut up the compiler when you’re just hacking things out. On the third hand though, the temptation to do things the wrong way is exactly why the compiler shouldn’t let you do things the wrong way “just for now.” So, in the end (on my fourth hand), I’m pretty neutral about it.

Finally, Go doesn’t have operator overloading, function/method overloading, or keyword arguments. You can see why—those features are bad for performance and they lead to “clever” code that does bad things—but you also sometimes miss them. It’s a tradeoff, and I see why they made the choice they did, but I also see the value of the other choice.

Conclusion

Go is rad. It’s not my everyday language (that’s still Python), but it’s definitely a fun language to use, and one that would be great for doing big projects in. If you’re interested in learning about Go, I recommend doing the tour then reading the spec while you put test programs into the Playground. The spec is very short and quite readable, so it’s great way to learn Go.

2012/8/10

2012/7/19

My early metrics book, Controlling Software Projects: Management, Measurement, and Estimation (Prentice Hall/Yourdon Press, 1982), played a role in the way many budding software engineers quantified work and planned their projects. […] The book’s most quoted line is its first sentence: “You can’t control what you can’t measure.” This line contains a real truth, but I’ve become increasingly uncomfortable with my use of it.

Implicit in the quote (and indeed in the book’s title) is that control is an important aspect, maybe the most important, of any software project. But it isn’t. Many projects have proceeded without much control but managed to produce wonderful products such as Google Earth or Wikipedia.

To understand control’s real role, you need to distinguish between two drastically different kinds of projects:

  • Project A will eventually cost about a million dollars and produce value of around $1.1 million.

  • Project B will eventually cost about a million dollars and produce value of more than $50 million.

What’s immediately apparent is that control is really important for Project A but almost not at all important for Project B. This leads us to the odd conclusion that strict control is something that matters a lot on relatively useless projects and much less on useful projects. It suggests that the more you focus on control, the more likely you’re working on a project that’s striving to deliver something of relatively minor value.

— Tom DeMarco via plasmasturm.org, which is consistently the most Daoist programming link blog I subscribe to.

2012/5/21

The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures. […]

Yet the program construct, unlike the poet’s words, is real in the sense that it moves and works, producing visible outputs separate from the construct itself. It prints results, draws pictures, produces sounds, moves arms. The magic of myth and legend has come true in our time. One types the correct incantation on a keyboard, and a display screen comes to life, showing things that never were nor could be.

[…]

Not all is delight, however […] One must perform perfectly. The computer resembles the magic of legend in this respect, too. If one character, one pause, of the incantation is not strictly in proper form, the magic doesn’t work. Human beings are not accustomed to being perfect, and few areas of human activity demand it.

— Fred Brooks, The Mythical Man-Month (via plasmasturm)

2010/11/13

2009/9/14

2009/8/13

Josh Bloch - How To Design A Good API and Why it Matters

Guido van Rossum, creator of Python says, "Watch it at least once a year".

This talk screams “Chinese philosophy” to me, everything from exemplar models to make right the names to learning (學) over thinking (思) to affecting the tips by taking care with the root. He even wrote up his findings as a series of aphorisms! I can’t tell if this is because they were right or because I read them into places where they don’t belong due to an over-application of the principle of charity to the originals in my interpretations.

The one point that seems to differ to me is that I can’t off the top of my head think of a parallel to hiding the implementation details of an API. I can see though why one would want to keep an API small, since one has to have trustworthiness (信, 誠, etc.) to keep the API working in the same way in the future. I guess that the reason for this difference is that in the social world, you shouldn’t try to refactor the underpinnings of some process, since things are almost never neatly modular. In computers, however, conceptual modularity is a fiction that can be maintained with a not unreasonable amount of diligence (though it does take some work; that’s why they say “Goto Considered Harmful,” etc.).

It may seem interesting that Guido, the creator of such a dynamic and plainspoken language as Python, should be recommending a talk by one of the creators of Java, a static and verbose language. However, the principles of good API design are more or less universal. What’s interesting about Java though is that because it is suited to use by large “enterprise”-type teams rather than an individual programmer or a small group of programmers, it more concretely embodies the social dynamics of Chinese philosophy. In Python, one ought to be a good citizen and use normal variable names and not monkeypatch and stay out of people’s private class/module members, etc., etc. But in Java these things are not just good ideas but absolutely vital practices. Having a large number of unrelated individuals (who are, to be honest, probably not the best programmers) attempt to work together on an enterprise application would be doomed from the start without the social straightjacket of classes, interfaces, public and private, “design patterns,” and so on to keep the masses inline behind the sage at the lead of development.

2009/3/22

A Map Doesn’t Help You in the Dark

I’ve seen some quite negative reactions to Doug Bowman’s post, which insinuated he was ungrateful for his position at Google. Most of this seemed to hinge on the phrase “I can’t operate in an environment like that." But I think this phrase was widely misinterpreted. It doesn’t mean "I don’t like working in this environment". Rather, it means "You are forcing me to deliver an inferior result based on a flawed belief."

That belief is that data can’t lie.

The frustration for someone in Doug’s position is that he is hired as an expert in his field, eager to share his experiences. The problem, though, is the people in Doug’s position are often hired under the incorrect assumption that a designer has amassed information in their career, not experiences. That assumption leads to a second flawed assumption: that all decisions will be based on hard facts.

Theocacao: Measuring the Design Process