Learn Go Reflections and Generic Designs with a Practical Example

Reflection is a powerful capability available in some programming languages that allows a program to examine, introspect and modify its own structure and behavior at runtime. Essentially, reflection enables code to "reflect" on itself, retrieving information about types, variables, functions and more programatically during execution.

In the Go programming language, reflection is primarily provided by the reflect package in the standard library. The reflect package allows you to dynamically inspect and manipulate arbitrary values, types and functions at runtime. This opens up a variety of advanced use cases, from building generic algorithms to implementing plugin systems and dependency injection frameworks.

In this post, we‘ll take a deep dive into Go‘s reflection capabilities and see how they can be applied to a real-world scenario. Specifically, we‘ll use reflection to build a generic application configuration mechanism that allows us to dynamically wire up and invoke analytics functions based on a config file. Along the way, you‘ll learn about key parts of the reflect API and some best practices for generic Go programming. Let‘s get started!

A Quick Tour of Go‘s reflect Package

At the heart of reflection in Go is the reflect package. This provides a set of types and functions for inspecting and manipulating arbitrary Go values at runtime. Let‘s look at a few key parts of the API that we‘ll be using later.

The entry point to reflection is the reflect.ValueOf() and reflect.TypeOf() functions. These take an arbitrary Go value and return a reflect.Value or reflect.Type struct respectively containing metadata about that value:

x := 42
xVal := reflect.ValueOf(x)  // reflect.Value
xType := reflect.TypeOf(x)  // reflect.Type

reflect.Value and reflect.Type have a variety of methods for introspecting the underlying value or type. For example:

fmt.Println(xVal.Kind())  // "int"
fmt.Println(xType.Kind()) // "int"
fmt.Println(xVal.Int())   // 42

Another key part of reflect is the ability to inspect and call methods. reflect.Value has a MethodByName() function that returns a callable reflect.Value representing the method with a given name:

type Greeter struct {}

func (g *Greeter) Greet(name string) {
    fmt.Printf("Hello, %s!", name)
}

g := &Greeter{}
gVal := reflect.ValueOf(g)
greetMethod := gVal.MethodByName("Greet") 

greetMethod.Call([]reflect.Value{reflect.ValueOf("world")})
// Output: Hello, world!

Here we get a reflect.Value for the Greet method and then invoke it using Call(), passing a slice of reflect.Value arguments.

There‘s a lot more to the reflect package, but these are some of the key pieces we‘ll use. Next let‘s see how to apply this to a real use case.

Building a Generic Config-Driven Application

Let‘s consider the following scenario: we‘re building an analytics tool that processes data from an inventory database and generates various statistical reports. There are many possible types of reports ranging from simple counts to complex machine learning models. We want the exact analyses that are run to be configurable via a config file, without having to modify code.

A clean architecture for this is to define each type of analysis as a separate function, and then dynamically invoke the configured ones at runtime. For example:

func TotalBooksByAuthor(db *sql.DB) (map[string]int, error) { ... }

func AveragePriceByCategory(db *sql.DB) (map[string]float64, error) { ... }

We could have many such functions, each taking a database connection and returning some analysis result. We then define the desired analyses in a config file:

# analytics.yaml
analyses:
  - name: book_counts
    func: TotalBooksByAuthor

  - name: avg_prices  
    func: AveragePriceByCategory

At runtime, we read this config and execute the specified functions. But how do we bridge the gap between the function names in the config and the actual func values in our code? This is where reflection comes in!

Wiring Up Functions with Reflection

The key idea is to use reflect.ValueOf() and MethodByName() to dynamically look up the configured functions by name and get back something callable. Here‘s a sketch of how it works:

// readConfig returns a list of analysis functions to run
func readConfig() ([]reflect.Value, error) {
    // Read the YAML config into a struct   
    var config struct {
        Analyses []struct {
            Func string `yaml:"func"`
        } `yaml:"analyses"`
    }
    // ...parse YAML... 

    // Look up each function by name
    var funcs []reflect.Value
    for _, a := range config.Analyses {
        fn := reflect.ValueOf(Analytics{}).MethodByName(a.Func)
        if !fn.IsValid() {
            return nil, fmt.Errorf("analytics: no such function %q", a.Func) 
        }
        funcs = append(funcs, fn)
    }
    return funcs, nil
}

We define a struct type matching the shape of our YAML config, parse the file, and then iterate over the "analyses" list. For each one, we look up the specified .Func using MethodByName() on a reflect.Value of an Analytics struct (the container for our functions). If found, we save the function value, otherwise we error out.

The result is a []reflect.Value holding our callable function values, wired up purely from the config file. We can then invoke them like so:

db, _ := sql.Open(...)
funcs, _ := readConfig()

for _, fn := range funcs {
    retVals := fn.Call([]reflect.Value{reflect.ValueOf(db)})
    result := retVals[0].Interface()
    // ...do something with result...
}

For each function, we Call() it passing the database connection as an argument. The return value comes back as a []reflect.Value which we convert to an interface{} using Interface().

And there we have it – a fully generic, config-driven analytics system using reflection! The beauty is that we can add new analysis functions just by defining them in code and updating the config file – no other wiring or "glue" code is needed.

A Quick Note on Type Safety

Astute readers may have noticed that our readConfig() function is not type safe – if an analysis function doesn‘t match the expected signature of func(*sql.DB) (interface{}, error), we‘ll get a panic at runtime when we try to Call() it.

We can make this type safe by having our analysis functions implement an interface:

type AnalysisFunc interface {
    Analyze(*sql.DB) (interface{}, error)
}

func TotalBooksByAuthor(db *sql.DB) (map[string]int, error) { ... }
func (TotalBooksByAuthor) Analyze(db *sql.DB) (interface{}, error) {
    return TotalBooksByAuthor(db)
}

Now in readConfig() we can safely cast our looked-up function to AnalysisFunc before calling it:

fn := reflect.ValueOf(Analytics{}).MethodByName(a.Func)
if !fn.IsValid() {
    return nil, fmt.Errorf("analytics: no such function %q", a.Func)
}
analysisFunc, ok := fn.Interface().(AnalysisFunc)
if !ok {
    return nil, fmt.Errorf("analytics: invalid signature for %q", a.Func)
}
funcs = append(funcs, analysisFunc.Analyze)

This gives us a compile-time guarantee that all our analysis functions have the correct signature.

Reflections on Using Reflection

As we‘ve seen, reflection is a powerful tool for building generic, dynamically-configured systems in Go. However, it‘s important to note that reflection is not without its costs:

• Reflection can be harder to understand and maintain than explicit code. Developers need to be familiar with the reflect API and mentality.

• Reflection relies on runtime information, which means many errors (like missing methods or invalid types) are only caught at runtime, not compile time. Judicious use of interfaces can mitigate this.

• Calling functions via reflection is slower than calling them directly. For performance-critical paths, reflection may not be suitable.

So when should you use reflection? In my experience, reflection is best suited for building platforms and frameworks where the exact types and behaviors are not known at compile time. Some examples are plugin systems, dependency injection, serialization frameworks, and API mocking tools.

For most application-level code, it‘s better to start with a direct, compile-time approach and only reach for reflection if truly needed. When you do use reflection, try to localize it to key areas and provide type-safe interfaces around it.

Learning More

This post has been a whirlwind tour of Go reflection and how it can enable powerful generic designs. We‘ve only scratched the surface of what‘s possible with reflection. To learn more, I recommend the following resources:

• The official reflect package docs: https://golang.org/pkg/reflect/
• Go‘s blog post on Laws of Reflection: https://blog.golang.org/laws-of-reflection
• reflectwalk, a Go library for "walking" complex structures using reflection: https://github.com/mitchellh/reflectwalk

I also highly recommend practicing using reflection by building out your own tools and frameworks. Some ideas:

• Implement a generic JSON serializer/deserializer
• Build a simple web framework that allows controller methods to take custom request and response types
• Write a parallel testing library that can run subtests across a shared fixture

Whatever you choose, have fun reflecting on the power of reflection! By mastering this advanced technique, you open up a whole new world of possibilities in Go programming.