Reflective Metaprogramming on Go: loop for each by means of the reflect package

For many programmers who use or wish to use Go in practice, the lack of parametric polymorphism mechanisms in the language is a great sadness. But not everything is as bad as it may seem at first glance.

Of course, you cannot write generic programs in Go, for example in the C ++ templates style, which would have practically no effect on CPU time. There is no such mechanism in the language and it is quite possible that it is not expected.

On the other hand, the language is a fairly powerful built-in package reflect , which allows for the reflection of both objects and functions. If you do not put speed at the center, then with the help of this package you can achieve interesting and flexible solutions.

In this article, I will show how to implement for each in the form of a type-independent reflexive function.

Problem

In the Go language, for searching elements of a collection ( Array , Slice , String ), the for range construct is used:

 for i, item := range items { // do something } 

Similarly, you can select items from the Channel :

 for item := range queue { // do something } 

In general, this covers 80% of the needs for the for each loop. But the built-in construction for range has pitfalls that are easy to demonstrate with a small example.

Suppose we have two structures Car and Bike (imagine that we are writing code for a car shop):

 type Car struct{ Name string Count uint Price float64 } type Bike struct{ Name string Count uint Price float64 } 

We need to calculate the cost of all the cars and motorcycles that we have in stock.
To do this with a single loop in Go, a new type is needed that summarizes the access to the fields:

 type Vehicle interface{ GetCount() uint GetPrice() float64 } func (c Car) GetCount() uint { return c.Count; } func (c Car) GetPrice() float64 { return c.Price; } func (b Bike) GetCount() uint { return b.Count; } func (b Bike) GetPrice() float64 { return b.Price; } 

Now the total cost can be calculated by traversing vehicles using for range :

 vehicles := []Vehicle{ Car{"Banshee ", 1, 10000}, Car{"Enforcer ", 3, 15000}, Car{"Firetruck", 4, 20000}, Bike{"Sanchez", 2, 5000}, Bike{"Freeway", 2, 5000}, } total := float64(0) for _, vehicle := range vehicles { total += float64(vehicle.GetCount()) * vehicle.GetPrice() } fmt.Println("total", total) // $ total 155000 

In order not to write a loop each time we can write a function that takes the type []Vehicle as an input and returns a numerical result:

 func GetTotalPrice(vehicles []Vehicle) float64 { var total float64 for _, vehicle := range vehicles { total += float64(vehicle.GetCount()) * vehicle.GetPrice() } return total } 

Separating this code into a separate function, oddly enough, we lose flexibility because The following problems appear:

• Restriction of strict typing of elements. Since Since the for / range construction is strongly typed and does not produce element types, it is necessary to explicitly indicate the expected type of elements in the function signature. As a consequence, it is not possible to transfer the []Car or []Bike slice directly, although both types - both Car and Bike - satisfy the conditions of the Vehicle interface:

 cars := []Car{ Car{"Banshee ", 1, 10000}, Car{"Enforcer ", 3, 15000}, Car{"Firetruck", 4, 20000}, } fmt.Println("total", GetTotalPrice(cars)) // Compilation error: cannot use cars (type []Car) as type []Vehicle in argument to GetTotalPrice 

• Restriction of strict typing of the collection. For example, there is no possibility to pass instead of the []Vehicle slice the map[int]Vehicle dictionary:

 cars := map[int]Vehicle{ 1: Car{"Banshee ", 1, 10000}, 2: Car{"Enforcer ", 3, 15000}, 3: Car{"Firetruck", 4, 20000}, } fmt.Println("total", GetTotalPrice(cars)) // Compilation error: cannot use vehicles (type map[int]Vehicle) as type []Vehicle in argument to GetTotalPrice 

In other words, for / range does not allow you to select an arbitrary part of the code and wrap it in a function without losing flexibility.

Decision

The described problem in many languages ​​with strict typification is solved by involving the parametric polymorphism mechanism (generics, templates). But instead of parametric polyformism, the authors of Go presented a built-in reflect package that implements the mechanism of reflection.
On the one hand, reflection is a more resource-intensive solution, but on the other hand, it allows you to create more flexible and intelligent algorithms.

Type Reflection (reflect.Type)

In fact, in the package reflect there are two types of reflection - this is reflection.Type type reflect.Type and reflection.Value value reflect.Value . Type reflection describes only type properties, therefore two different variables with the same type will have the same type reflection.

 var i, j int var k float32 fmt.Println(reflect.TypeOf(i) == reflect.TypeOf(j)) // true fmt.Println(reflect.TypeOf(i) == reflect.TypeOf(k)) // false 

Since in Go, types are constructed on the basis of basic types, then for classification there is a special enumeration with the type Kind:

 const ( Invalid Kind = iota Bool Int Int8 Int16 Int32 Int64 Uint Uint8 Uint16 Uint32 Uint64 Uintptr Float32 Float64 Complex64 Complex128 Array Chan Func Interface Map Ptr Slice String Struct UnsafePointer ) 

Thus, having access to the reflection.Type type of reflect.Type , you can always find out the type of type that allows you to dispatch without defining the full type of the variable. For example, it is enough to know that a variable is a function, without going into details which particular type this function has:

 valueType := reflect.TypeOf(value) switch valuteType.Kind() { case reflect.Func: fmt.Println("It's a function") default: fmt.Println("It's something else") } 

For convenience of writing, we will call the type reflection a certain variable with the same name, but with the Type suffix:

 callbackType := reflect.TypeOf(callback) collectionType := reflect.TypeOf(collection) 

In addition to the type to which the type belongs, with the help of reflection of the type, you can find out the remaining static information about the type (that is, information that does not change at run time). For example, you can find out the number of function arguments and the type of expected argument at a certain position:

 if callbackType.NumIn() > 0 { keyType := callbackType.In(0) // expected argument type at zeroth position } 

Similarly, you can access the description of the members of the structure:

 type Person struct{ Name string Email string } structType := reflect.TypeOf(Person{}) fmt.Println(structType.Field(0).Name) // Name fmt.Println(structType.Field(1).Name) // Email 

The size of the array can also be found through reflection type:

 array := [3]int{1, 2, 3} arrayType := reflect.TypeOf(array) fmt.Println(arrayType.Len()) // 3 

But the size of the slice through the reflection of the type is already impossible to know, because this information changes at run time.

 slice := []int{1, 2, 3} sliceType := reflect.TypeOf(slice) fmt.Println(sliceType.Len()) // panic! 

Reflection values ​​(reflect.Value)

Similar to type reflection, in Go there is a reflection of the reflect.Value value, which reflects the properties of a particular value stored in a variable. It may seem like a rather trivial reflection, but because in Go, a variable with the interface{} type can store anything - a function, a number, a structure, etc., and the reflection of the value is forced to provide access to all the probable possibilities of the object in a more or less safe form. Which, of course, generates a rather long list of methods.

For example, the reflection of a function can be used to call - just pass the list of arguments to the type reflect.Value :

  _callback := reflect.ValueOf(callback) _callback.Call([]reflect.Value{ values }) 

Reflection of the collection (slice, array, string, etc.) can be used to access the elements:

 _collection := reflect.ValueOf(collection) for i := 0; i < _collection.Len(); i++ { fmt.Println(_collection.Index(i)) } 

Reflection of the dictionary works in a similar way - to get around it, you need to get a list of keys through the MapKeys method and select elements via MapIndex :

 for _, k := range _collection.MapKeys() { keyValueCallback(k, _collection.MapIndex(k)) } 

With the help of the reflection of the structure you can get the values ​​of the members The names and types of members should be derived from the reflection of the type of structure:

 _struct := reflect.ValueOf(aStructIstance) for i := 0; i < _struct.NumField(); i++ { name := structType.Field(i).Name fmt.Println(name, _struct.Field(i)) } 

Reflective loop for each

So, if you go back to for each, then it is desirable to get a function that would accept a collection and a callback function of an arbitrary type, thus the responsibility for the type negotiation would be on the user.

Since the only possibility in Go to transfer an arbitrary type of function is to specify the interface{} type, then in the function body it is necessary to perform checks based on the information contained in the callbackType type reflection:

• make sure that the callback function is really a function (via the calbackType.Kind() method)
• find out the number of expected arguments ( callbackType.NumIn() method)
• in case of failure, call panic()

The result is approximately the following code:

 func ForEach(collection, callback interface{}) { callbackType := reflect.TypeOf(callback) _callback := reflect.ValueOf(callback) if callbackType.Kind() != reflect.Func { panic("foreach: the second argument should be a function") } switch callbackType.NumIn() { case 1: // Callback expects only value case 2: // Callback expects key-value pair default: panic("foreach: the function should have 1 or 2 input arguments") } } 

Now you need to design a helper function that will crawl the collection.
It is more convenient to pass the callback to it not in a typeless form, but as a function with two arguments that accepts the reflection of the key and the element:

 func eachKeyValue(collection interface{}, keyValueCallback func(k, v reflect.Value)) { _collection := reflect.ValueOf(collection) collectionType := reflect.TypeOf(collection) switch collectionType.Kind() { // loops } } 

Since Since the collection passage algorithm depends on the type that can be obtained through the Kind() method of reflection type, then for dispatching it is convenient to use the switch-case construction:

 switch collectionType.Kind() { case reflect.Array: fallthrough case reflect.Slice: fallthrough case reflect.String: for i := 0; i < _collection.Len(); i++ { keyValueCallback(reflect.ValueOf(i), _collection.Index(i)) } case reflect.Map: for _, k := range _collection.MapKeys() { keyValueCallback(k, _collection.MapIndex(k)) } case reflect.Chan: i := 0 for { elementValue, ok := _collection.Recv() if !ok { break } keyValueCallback(reflect.ValueOf(i), elementValue) i += 1 } case reflect.Struct: for i := 0; i < _collection.NumField(); i++ { name := collectionType.Field(i).Name keyValueCallback(reflect.ValueOf(name), _collection.Field(i)) } default: keyValueCallback(reflect.ValueOf(nil), _collection) } 

As you can see from the code, traversing an array, a slice and a string is the same. The dictionary, channel and structure have their own traversal algorithm. If the genus of the collection does not fall under one of the above, the algorithm tries to pass the collection itself to the callback, note that the reflection index nil (which returns a call to IsValid() returns false ) as the key.

Now that you have a function that produces a typeless bypass of the collection, you can adapt it to a call from the ForEach function by wrapping it into a closure. This is the final decision:

 func ForEach(collection, callback interface{}) { callbackType := reflect.TypeOf(callback) _callback := reflect.ValueOf(callback) if callbackType.Kind() != reflect.Func { panic("foreach: the second argument should be a function") } switch callbackType.NumIn() { case 1: eachKeyValue(collection, func(_key, _value reflect.Value){ _callback.Call([]reflect.Value{ _value }) }) case 2: keyType := callbackType.In(0) eachKeyValue(collection, func(_key, _value reflect.Value){ if !_key.IsValid() { _callback.Call([]reflect.Value{reflect.Zero(keyType), _value }) return } _callback.Call([]reflect.Value{ _key, _value }) }) default: panic("foreach: the function should have 1 or 2 input arguments") } } 

It should be noted that in the case where the callback function is waiting for the transfer of two arguments (key / value pairs), it is necessary to check the key's correctness, since it may not be valid. In the latter case, a null object is constructed based on the key type.

Examples

Now it's time to demonstrate what our approach gives. If we return to the problem, we can now solve it in this way:

 func GetTotalPrice(vehicles interface{}) float64 { var total float64 ForEach(vehicles, func(vehicle Vehicle) { total += float64(vehicle.GetCount()) * vehicle.GetPrice() }) return total } 

This function, in contrast to the one given at the beginning of the article, is much more flexible, since allows you to calculate the amount regardless of the type of collection and does not require to bring the type of elements to the Vehicle interface:

 vehicles := []Vehicle{ Car{"Banshee ", 1, 10000}, Bike{"Sanchez", 2, 5000}, } cars := []Car{ Car{"Enforcer ", 3, 15000}, Car{"Firetruck", 4, 20000}, } vehicleMap := map[int]Vehicle{ 1: Car{"Banshee ", 1, 10000}, 2: Bike{"Sanchez", 2, 5000}, } vehicleQueue := make(chan Vehicle, 2) vehicleQueue <- Car{"Banshee ", 1, 10000} vehicleQueue <- Bike{"Sanchez", 2, 5000} close(vehicleQueue) garage := struct{ MyCar Car MyBike Bike }{ Car{"Banshee ", 1, 10000}, Bike{"Sanchez", 1, 5000}, } fmt.Println(GetTotalPrice(vehicles)) // 20000 fmt.Println(GetTotalPrice(cars)) // 125000 fmt.Println(GetTotalPrice(vehicleMap)) // 20000 fmt.Println(GetTotalPrice(vehicleQueue)) // 20000 fmt.Println(GetTotalPrice(garage)) // 15000 

And a small benchmark for two identical cycles, which clearly shows how flexibility is achieved:

 // BenchmarkForEachVehicles1M total := 0.0 for _, v := range vehicles { total += v.GetPrice() } 

 //BenchmarkForRangeVehicles1M total := 0.0 ForEach(vehicles, func(v Vehicle) { total += v.GetPrice() }) 

 PASS BenchmarkForEachVehicles1M-2 2000000000 0.20 ns/op BenchmarkForRangeVehicles1M-2 2000000000 0.01 ns/op 

Conclusion

Yes, there is no parametric polyformism in Go. But there is a package of reflect , which provides extensive opportunities in the field of metaprogramming. The code using reflect course, looks much more complicated than the typical code on Go. On the other hand, reflexive functions allow you to create more flexible solutions. This is very important when writing application libraries, for example, when implementing the concept of Active Record .

So, if you do not know in advance how other programmers will use your library and the speed limit for you is not the main goal, then, quite possibly, reflexive metaprogramming will be the best choice.

Github source code