MEAPEdition
ManningEarlyAccessProgram
GoinAction
Version7
Copyright2015ManningPublications
FormoreinformationonthisandotherManningtitlesgoto
www.manning.com
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
Welcome
Thank you for purchasing the MEAP for Go in Action. We’ve been writing this book in
one form or another for almost four years now, so we’re very excited to release it to you:
our intrepid early adopter. This book was written for an intermediate level developer who
has some experience with other programming languages and is curious about Go.
Our goal in writing Go in Action is not just to introduce you to the Go language, but to
share our experiences using Go in high-volume production environments. As the book
progresses, we’ll show you patterns for success and hopefully debunk some anti-patterns
too.
Our initial release will include the first two chapters, which are focused on getting you
acquainted with this new programming language. Chapters 1 and 2 will give you an
overview of the language and the workflow that you’ll use as a Go developer. By the end of
Chapter 2, you will be comfortable reading and understanding most Go code.
Most of the chapters in this book will be broken into two parts: Part I will teach you the
concepts that power the language, and Part II will show you how to apply those concepts in
your projects.
You should be getting updates to this book every two to three weeks. Sometimes we’ll
release the chapters with only Part I, and release Part II a little later; sometimes we’ll release
whole chapters at once. We’re counting on your feedback to help us shape this book into an
indispensable tool for professional developers who want to become proficient with Go.
Please join us on the Manning Author Online Forum. We’ll be reading your comments,
providing help and feedback, and using your suggestions to make the book better before it is
published.
The source code for the examples in the book is available at
https://github.com/goinaction/code Feel free to download and play along! We look forward
to your feedback as we finalize this book.
Brian Ketelsen
William Kennedy
Erik St. Martin
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
brief contents
1 Introducing Go
2 Go Quick Start
3 Packaging and Tooling
4 Arrays, Slices and Maps
5 Building User Defined Types
6 Concurrency
7 Concurrency Patterns
8 Standard Library
9 Writing Network Applications in Go
10 Debugging and Testing Go Programs
11 Reflection
12 Making Your Applications More Performant
13 Interoperating with C Libraries
14 Using Go for Devops
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
1
1
Introducing Go
In this chapter
The Case for Go
Using the Go Tools
Prerequisites - Things you should know before reading this chapter
You should be familiar with a command prompt or shell
You should have Go installed according to the guidelines in the
preface of this book
You should be familiar with at least one other programming
language
Computers have evolved, but programming languages haven't kept the same pace
of evolution. The cell phone you carry probably may have more CPU cores than
the first computer you used. High powered servers now have 64, 128, or even more
cores, but we're still programming using the techniques we've been using for a
single core.
The art of programming has evolved too. Most programs aren't written by a
single developer any more, they're written by teams of people sitting in different
time zones, working at different times of the day. Large projects are broken up into
smaller pieces and assigned to programmers who then deliver their work product
back to the team in the form of a library or package that can be used across an
entire suite of programs.
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
2
Today's programmers and companies believe more and more in the power of
Open Source Software. Go is a programming language that makes sharing code
easy. Go ships with tools that make it simple to use packages written by others, and
Go makes it easy to share your own packages too.
In this chapter you'll see how Go is different from other programming
languages. Go rethinks the traditional Object Oriented development you might be
used to while still providing an efficient means for code reuse. Go makes it easier
for you to effectively use all of the cores on your expensive server, and Go takes
away the penalty of compiling a very large project.
As you read this chapter, you'll get a feeling for the many decisions that shaped
the creation of Go, from its concurrency model to its lightning fast compiler. You'll
appreciate the tools that ship with Go to make your life as a developer easier.
You'll see why so many developers are choosing Go when they start up that new
project.
1.1 Why Do We Need Another Programming Language?
It isn't easy to pick a programming language when you're starting a new project, or
even picking a platform for your team. Every programming language comes with a
set of tradeoffs that make the decision a difficult one. Let's look at some of the
more obvious tradeoffs you will find as you look at new languages.
1.1.1 Choosing Static vs. Dynamic Languages
If you've ever spent time writing code using a dynamic language, you'll know why
people love them. It's so easy to throw together a script that manipulates some data
without formally worrying about the data types. This flexibility comes with some
grief. You can't always presume that the data you receive is the data you're
expecting, and you can get some pretty crazy results from dynamic languages
when you feed them unexpected data. The solution is writing more code and lots of
tests.
Conversely, static languages give you type safety which can frustrate you with
repetition. Code, Compile, Test... Code, Compile, Test... Lather, Rinse, Repeat!
Compiling is a productivity killer. But with compiled languages, deploying your
compiled binaries is a breeze and the burden of testing bizarre edge cases is a little
more palatable.
We need something better that provides the benefits of dynamic and interpreted
languages with the speed and safety of static and compiled languages.
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
3
1.1.2 How Will Your Language Handle Concurrency
Very few programming languages have given more than a passing thought to
concurrency. Some dynamic languages like Ruby and Python implement a Global
Interpreter Lock into their runtimes to ensure thread-safety and mitigate
race-conditions, it comes at the cost of reducing performance, as the interpreter can
only be executing in one thread at a time. Other languages, like Java, have the
primitives needed for concurrency but often require tons of boilerplate code to
write concurrent applications properly.
We need a language that makes concurrency a first-class citizen.
1.1.3 Are The Batteries Included?
One of the best gauges to determine the utility of a programming language is how
much you can get done without using external libraries. In many languages, it's
hard to do anything beyond simple text manipulation without resorting to installing
gems, eggs, or CPAN modules. And forget about trying to do some serious work
like creating a network service without finding, downloading, testing and learning
some third party code.
Wouldn't it be nice if there was a language where nearly all the heavy lifting
was already done for you?
1.2 Solving Modern Programming Challenges With Go
The Go team went to great lengths to solve the problems facing software
developers today. Developers have to make an uncomfortable choice between
rapid development and performance when choosing a language for their projects.
Some languages like C or C++ offer fast execution, while other languages like
Ruby or Python offer rapid development. Go bridges these competing worlds and
offers a high performance language with features that make development fast.
As you explore Go, you'll find well planned features and concise syntax. As a
language, Go is defined not only by what it includes, but by what it doesn't include.
Go has a concise syntax with very few keywords to memorize. Go has a compiler
that's so fast, you might not notice it at all. As a developer you will spend
significantly less time waiting for your project to build. Because of the built-in
concurrency features of Go, your software will scale to use the resources available
without forcing you to use special threading libraries. Go uses a simple and
effective type system that takes much of the overhead out of object-oriented
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
4
development and lets you focus on code reuse. Go also has a garbage collector, so
you don't have to manage your own memory. Let's look quickly at these key
features of Go.
1.2.1 Programs Compile Quickly
Compiling a large application in C or C++ takes more time than getting a cup of
coffee. Figure 1.1 is an XKCD classic excuse for messing around in the office. Go
offers lightning quick compiles by using a smart compiler and simplified
dependency resolution algorithms. When you build a Go program, the compiler
only needs to look at the libraries that you directly include, rather than traversing
the dependencies of all the libraries that are included in the entire dependency
chain like Java, C and C++. Consequently, many Go applications compile in under
a second. The entire Go source tree compiles in just under twenty seconds on
modern hardware.
Figure 1.1 Working Hard? (via XKCD)
Writing applications in dynamic languages makes you productive quickly
because there are no intermediate steps between writing code and executing that
code. The tradeoff is that dynamic languages don't offer the type safety that static
languages do, and often need a very comprehensive test suite to avoid discovering
incorrect type bugs at runtime.
Imagine writing a large application in a dynamic language like Javascript and
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
5
you come across a function that expects to receive a field called ID. Is that an
integer, a string, a UUID? The only way to find out is to look at the source if you
have it, or try to execute the function with a number or a string and see what
happens. In Go, you wouldn't spend a second of time wondering, because the
compiler will catch type differences for you.
1.2.2 Concurrency is Built In
One of the hardest things to do as a programmer is to write an application that
effectively uses the available resources of the hardware running it. Modern
computers have many cores, but most programming languages don't have effective
tools for utilizing those additional resources easily. They often require a lot of
thread synchronization code, which is prone to errors.
Go's concurrency support is one of its strongest features. Goroutines are like
threads, but use far less memory and require much less code to implement.
Channels are typed memory queues that let you send messages between goroutines.
This facilitates a programming model where you send data between goroutines,
rather than letting the goroutines fight to use the same data. Let's look at these
features in more detail now.
GOROUTINES
Goroutines are functions that run concurrently with other goroutines, including the
entry point of your program. In other languages, you would use threads to
accomplish the same thing, but in Go, many goroutines execute on a single thread.
For example, if you write a web server and you want to handle different web
requests simultaneously, you would have to write a lot of extra code to use threads
in C or Java, but Go's net/http library has concurrency built in using goroutines.
Each inbound request automatically processes on it's own goroutine. Goroutines
use less memory than threads and the Go runtime will automatically schedule the
execution of goroutines from a configurable pool of threads. This makes your
application much more efficient with significantly less development effort.
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
6
Figure 1.2 Many Goroutines Execute On A Single OS Thread
If you want to execute some code concurrently while you move on to
accomplish other things, a goroutine is perfect for the job. Here is a quick example:
func logError(errorMessage string){
... some error logging code here
}
// Elsewhere in your code after you've discovered an error.
go logError("something dire happened")
That keyword go is all you need to launch the logError() function as a
goroutine and be run concurrently with other goroutines. This means you can
continue executing the rest of your application while the logging happens
concurrently, which often results in greater perceived performance for your end
users. Goroutines have minimal overhead, and it isn't uncommon to spawn tens of
thousands of them without incurring memory bloat or processing overhead. We'll
explore goroutines and concurrency more in-depth in Chapter 6.
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
7
CHANNELS
Channels are typed memory queues that act as a pipeline to enable safe data
communication between goroutines. When used correctly, channels help you to
avoid problems typically seen in programming languages that allow shared
memory access.
The hardest part of concurrency is ensuring that your data is not unexpectedly
modified by concurrently running processes, threads, or goroutines. When multiple
threads change the same data without locks or synchronization, heartache always
follows. When you have global variables and shared memory, you are required to
use complicated mutexes or locks to prevent unsynchronized changes to the same
variables.
Channels help to solve this problem by providing a pattern that makes data safe
from concurrent modification. Channels help to enforce the pattern that only one
goroutine should modify the data at any time. You can see an example of this flow
in Figure 1.3, where channels are used to send data between several running
goroutines. Imagine an application where many different processes need to know
about or modify data sequentially. Using goroutines and channels, you can model
this process safely.
Figure 1.3 Using Channels To Safely Pass Data Between Goroutines
Each goroutine in Figure 1.3 has exclusive control of the data it receives on a
channel. The goroutine can modify the data, or act upon it, then send it down the
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
8
pipeline to the next goroutine for processing. Because of the guarantees that
channels provide, there are no locks or mutexes required to safely modify the data.
1.2.3 Go's Type System Enables Code Reuse
Go rethinks traditional object orientation for a flexible hierarchy-free type system
that enables code reuse with minimal refactoring overhead. It is still
object-oriented development, but without the headaches. If you've ever spent a
week trying to determine how to set up your abstract classes and interfaces in a
complex Java or C++ program, you'll appreciate the simplicity of Go's type system.
Go developers simply embed types to re-use functionality in a design pattern called
Composition. Other languages use composition, but it is often deeply tied to
inheritance, which can make it complicated and difficult to use. Your types in Go
are composed of smaller types, which is in contrast to traditional inheritance based
models.
In addition, Go has a unique interface implementation that allows you to model
actions, rather than modeling types. You don't need to declare that you are
implementing an interface in Go, the compiler does the hard work of determining
whether your type satisfies the interfaces you are using. Many interfaces in Go's
standard library are very small, exposing only a few functions. In practice this
takes some time to get used to, especially if you've been writing in very strong
object-oriented languages like Java.
TYPES ARE SIMPLE
Go has built-in types like int and string as well as user-defined types. A typical
user-defined type in Go will have typed fields to store data. If you've seen structs
in C, Go's user-defined types look and operate similarly. However, types may also
declare methods that operate on that data. But rather than building a long
inheritance structure---Sedan extends Car extends Vehicle---Go developers build
small types---Wheel and Door---and embed them into larger ones. Figure 1.4
demonstrates the difference between inheritance and composition.
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
9
Figure 1.4 Inheritance Vs. Composition
GO INTERFACES MODEL SMALL BEHAVIORS
Interfaces allow you to express the capabilities of a type. If a type implements an
interface, it means that a type has a specific set of capabilities. You don't even need
to declare that you're implementing an interface, you just need to write the
implementation. Other languages call this Duck Typing -- if it quacks like a duck,
then it can be a duck-- and Go does it well. In Go, if your type implements the
methods of an interface, it can be used anywhere that interface is expected, no
declaration required.
In a strictly object-oriented language like Java, interfaces are all encompasing.
You're often required to think through a large inheritance chain before you're able
to even start writing code. Here's an example of a java interface:
interface Animal {
public void eat();
public void move();
}
Implementing this interface in Java requires you to create a class that fulfills all
of the promises made in the Animal interface and explicitly declare that you
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
10
implement the interface. In contrast, a Go interface typically represents just a
single action. One of the most common interfaces you will use in Go is the
io.Reader. The io.Reader interface provides a simple way to declare that your type
has data to be read in a way that other functions in the standard library understand.
Here's the definition:
type Reader interface {
Read(p []byte) (n int, err error)
}
To write a type that implements the io.Reader interface, you only need to
implement a Read function that accepts a slice of bytes and returns an integer and
possible error.
This is a radical departure from the interface systems used in other object
oriented programming languages. Go's interfaces are smaller, and more aligned
with single actions. In practice, this allows significant advantages in code reuse.
You can implement an io.Reader on nearly any type that has data available, and
then pass it to any Go function that knows how to read from a Reader. The entire
networking library in Go is built using the io.Reader interface, because it allows
you to separate the network implementation required for each different network
operation from the functionality of your application. It makes interfaces fun,
elegant, and flexible. That same io.Reader enables simple operations with files,
buffers, sockets and any other data source. Using a simple interface allows you to
operate on data efficiently, regardless of the source.
1.2.4 Memory Management Is Easy
Improper memory management causes applications to crash, leak memory and
even crash the operating system. Go has a modern garbage collector that does the
hard work for you. In other systems languages, like C or C++, you need to allocate
a piece of memory before you can use it, then de-allocate it when you're done. If
you fail to do either of these correctly, you'll have program or system crashes. It
isn't easy to track a piece of memory when it is no longer needed; threads and
heavy concurrency make it even harder. When you write code with garbage
collection in mind, Go's garbage collection adds very little overhead to program
execution time, but reduces development effort significantly. Go takes the tedium
out of programming and leaves the bean counting to the accountants.
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
11
1.3 Hello, Go
It's much easier to get the feel of a programming language by seeing it in action.
Let's look at the traditional Hello World! application written in Go:
package main
import "fmt"
func main(){
fmt.Println("Hello World")
}
Go programs are organized as packages.
The import statement allows you to use external code. The "fmt" package, in this
case, which is provided by the standard library allows you to format and print data.
The main() function is what gets executed when you run your application - just like
in C.
This sample program prints a familiar phrase on your screen when you run it.
But how should you run it? Without installing Go on your computer, you can use
almost all of Go right from your web browser.
1.3.1 Introducing the Go Playground
There is a tool that allows you to edit and run Go code from your web browser
called the Go Playground. Fire up a web browser and navigate to
http://play.golang.org We'll wait for you. It should look a lot like Figure 1.5:
Figure 1.5 The Go Playground
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
12
The code in the browser window is editable right on the screen. Click the RUN
button and see what happens! The code executes and its output is printed at the
bottom of the screen like Figure 1.6:
Figure 1.6 Output From Go
Playground Shown In Your
Browser Window
You can change the code, for example to make the greeting text in a different
language. Go ahead and change the greeting inside the fmt.Println() function and
hit RUN again. Did you see your new greeting appear?
NOTE
Sharing Go Code
Go developers use the Playground to share code ideas, test
theories, and debug their code, as you soon will too. Every time you
create a new application on the Playground, you can click the
Share button to get a sharable URL that anyone else can open. Try
this one: http://play.golang.org/p/EWIXicJdmz
The Go Playground is the perfect way to demonstrate an idea for a coworker or
friend who is trying to learn something, and it's a great way for you to solicit help
too. On the Go IRC channels, in the Go mailing lists, and countless emails sent
among Go developers, you'll see Go Playground applications being created,
modified, and shared.
1.4 Introducing Kiev
In this YET TO BE WRITTEN SECTION, we'll introduce Kiev, which is a project
that will span many chapters of the book providing real-world examples of Go
code.
1.5 Summary and Best Practices
Go is a modern, garbage-collected language with a workflow that helps you to feel
productive while you are using it, mostly because of the great tools that are
provided with the language, like the go command, a focus on enabling easy
concurrency, and a clean syntax that is easy to remember.
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action
13
Go is modern, fast, and comes with all the batteries included
Go has concurrency built-in
Go uses interfaces as the building blocks of code reuse
©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and
other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders.
https://forums.manning.com/forums/go-in-action