Dependencies Generics. They often appear on the list of problems in the Go community, but there is one problem that is rarely remembered - the organization of the code for your package.

Each Go application I worked with seems to have its own answer to the question "How should I organize the code?". Some applications put everything in one package, while others group logic by type or module. Without a good strategy that all team members adhere to, you will sooner or later see that the code is heavily scattered across numerous packages. We need a standard for the design of code in Go applications.

I suggest a better approach. By following a set of simple rules, we can ensure that the code is unbound, easily testable and the project structure is complete. But before we dive into the details, let's look at the most commonly used approaches to structuring Go code.

The most frequent wrong approaches

There are a few of the most common approaches to organizing Go code, and each has its drawbacks.

Approach # 1: Monolith

Placing all the code in one package actually works very well for small applications. This ensures that you are not faced with the problem of circular dependencies, because, inside your application, you generally have no dependencies.

According to experience, this approach works great for applications up to 10 thousand lines of code. Then it becomes very difficult to understand and isolate parts of the monolith code.

Approach # 2: Rails style

The second approach is to group the code according to its functionality. For example, all handlers go to one package, controllers to another, and models to third. I have seen this approach many times with former Rails developers (including myself).

But with this approach there are two problems. First, you get monstrous names. You will have names like controller.UserController , in which you duplicate the package name in the type name. In general, I consider myself an ardent supporter of a careful approach to names. I am sure that good names are your best documentation, especially when you make your way through the code. Names are also often an indicator of the quality of a code — this is the first thing that another programmer will see when he first encounters your code.

But the biggest problem, in fact, is circular dependencies. Your functional types, separated by packages, may need each other. And this will only work if these dependencies are one-sided, but in most cases your application will be more complicated.

Approach # 3: Grouping by module

This approach is similar to the previous one, with the exception that we group the code by module, not by function. For example, you can split the code so that you have user and accounts packages.

Here we have the same problems. Again, horrible names like users.User and the same problem with circular dependencies, when our accounts.Controller users.Controller should interact with users.Controller . users.Controller and vice versa.

Best approach

The code organization strategy that I use in my projects includes 4 principles:

1. Root (main) package for domain types
2. Package grouping by dependencies
3. Use common mock package
4. The main package brings together dependencies

These rules help isolate packages and establish a clear language within the application. Let's see how each of these points works in practice.

1. Root package for domain types

Your application has a logical high-level language that describes the interaction of data and processes. This is your domain. If you are writing an e-commerce application, then your domain will include concepts such as customers, accounts, credit card debit and inventory. If you are Facebook, then your domain is users, likes and interactions. In other words, this is something that does not depend on the chosen technology.

I have domain types in the main, root package. This package contains only simple data types like User, in which there is only data about the user or the UserService interface for saving and querying user data.

It might look something like this:

 package myapp type User struct { ID int Name string Address Address } type UserService interface { User(id int) (*User, error) Users() ([]*User, error) CreateUser(u *User) error DeleteUser(id int) error } 

This makes your root package extremely simple. It can also include types that perform some actions, but only if they depend entirely on other domain types. For example, you can add a type that periodically polls your UserService. But he should not make calls to external services or save to the database. These are implementation details.

The root package should not depend on other packages inside your application!

2. Package grouping by dependencies

Since it is not allowed to have external dependencies in the root package, we must move these dependencies to other subpackages. In this approach, nested packages exist as an adapter between your domain and your implementation.

For example, your UserService may be implemented as a PostgreSQL database. You can add the postgres package to the application, which provides the postgres.UserService implementation:

 package postgres import ( "database/sql" "github.com/benbjohnson/myapp" _ "github.com/lib/pq" ) // UserService represents a PostgreSQL implementation of myapp.UserService. type UserService struct { DB *sql.DB } // User returns a user for a given id. func (s *UserService) User(id int) (*myapp.User, error) { var u myapp.User row := db.QueryRow(`SELECT id, name FROM users WHERE id = $1`, id) if row.Scan(&u.ID, &u.Name); err != nil { return nil, err } return &u, nil } // implement remaining myapp.UserService interface... 

This completely isolates the dependency on PostgreSQL, which greatly simplifies testing and opens up a simple way to migrate to another database in the future. It also allows you to create a dynamically changing architecture, if in the future you want to support other implementations, for example, on BoltDB .

It also makes it possible to create implementation layers. For example, you want to add an LRU cache in memory before your PostgreSQL implementation. Then you simply add a UserCache that implements the UserService, and in which you wrap your PostgreSQL implementation:

 package myapp // UserCache wraps a UserService to provide an in-memory cache. type UserCache struct { cache map[int]*User service UserService } // NewUserCache returns a new read-through cache for service. func NewUserCache(service UserService) *UserCache { return &UserCache{ cache: make(map[int]*User), service: service, } } // User returns a user for a given id. // Returns the cached instance if available. func (c *UserCache) User(id int) (*User, error) { // Check the local cache first. if u := c.cache[id]]; u != nil { return u, nil } // Otherwise fetch from the underlying service. u, err := c.service.User(id) if err != nil { return nil, err } else if u != nil { c.cache[id] = u } return u, err } 

We can see this approach also in the standard library. io.Reader is a domain type for reading bytes, and its implementations are grouped by dependencies - tar.Reader, gzip.Reader , multipart.Reader . And they can also be used in several layers. You can often see os.File wrapped in bufio.Reader , which is wrapped in gzip.Reader , which, in turn, is wrapped in tar.Reader .

Dependencies between dependencies

Your dependencies usually do not live by themselves. You may want to store user data in PostgreSQL, but financial transaction data may be in an external service like Stripe . In this case, we wrap our dependence on Stripe in a logical domain type - let's call it TransactionService .

By adding our TransactionService to the UserService we unleash (decouple) our two dependencies:

 type UserService struct { DB *sql.DB TransactionService myapp.TransactionService } 

Now our dependencies communicate exclusively with the help of our domain language. This means that we can switch from PostgreSQL to MySQL or switch from Stripe to another payment processor and do not have to change anything in dependencies.

Do not be limited to external dependencies.

This may sound strange, but I also isolate the dependencies on the standard library using the same method. For example, the net/http package is just another dependency. We can isolate it by adding an attached http package to the application.

It may seem strange to have a package with the same name as the standard library, but this is intentional. You will not have name conflicts if you do not use net / http elsewhere in your application. The benefit of duplicating the name will be that you isolate all the HTTP code inside your http package:

 package http import ( "net/http" "github.com/benbjohnson/myapp" ) type Handler struct { UserService myapp.UserService } func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) { // handle request } 

Now your http.Handler works as an adapter between your domain and the HTTP protocol.

3. Using a common mock package

Because our dependencies are isolated from others through domain types and interfaces, we can use these points of contact to introduce caps (mock).

There are several stub libraries like GoMock that will generate code for you, but I personally prefer to write them personally. It seems to me that most of the tools for plugs are unnecessarily complicated.

The plugs I use are usually very simple. For example, the stub for the UserService looks like this:

 package mock import "github.com/benbjohnson/myapp" // UserService represents a mock implementation of myapp.UserService. type UserService struct { UserFn func(id int) (*myapp.User, error) UserInvoked bool UsersFn func() ([]*myapp.User, error) UserInvoked bool // additional function implementations... } // User invokes the mock implementation and marks the function as invoked. func (s *UserService) User(id int) (*myapp.User, error) { s.UserInvoked = true return s.UserFn(id) } // additional functions: Users(), CreateUser(), DeleteUser() 

Such a stub allows me to embed functions in all places where the myapp.UserService interface is used to validate arguments. return the expected values ​​or embed erroneous data.

Suppose we want to test our http.Handler, which we added just above:

 package http_test import ( "testing" "net/http" "net/http/httptest" "github.com/benbjohnson/myapp/mock" ) func TestHandler(t *testing.T) { // Inject our mock into our handler. var us mock.UserService var h Handler h.UserService = &us // Mock our User() call. us.UserFn = func(id int) (*myapp.User, error) { if id != 100 { t.Fatalf("unexpected id: %d", id) } return &myapp.User{ID: 100, Name: "susy"}, nil } // Invoke the handler. w := httptest.NewRecorder() r, _ := http.NewRequest("GET", "/users/100", nil) h.ServeHTTP(w, r) // Validate mock. if !us.UserInvoked { t.Fatal("expected User() to be invoked") } } 

Our stub allowed to completely isolate this unit-test and test only part of the HTTP protocol.

4. The main package combines dependencies together

With all these addiction packages, you can ask how they all come together. And this is exactly the job for the main package.

main package organization

An application can consist of several binary executables, so we will use the standard Go agreement on the location of the main package in the cmd / subdirectory. For example, our project may have an executable file myappctl server, plus an additional myappctl binary to manage the server from the terminal. We place the file as follows:

 myapp/ cmd/ myapp/ main.go myappctl/ main.go 

Dependency injection at compile time

The term “dependency injection” has received a bad reputation. Usually, people immediately start thinking about Spring's XML files. But in fact, this term means that we are passing dependencies to an object, and not an object looking for them ourselves.

The main package is the place where the choice of which dependencies to embed in which objects occurs. Since this package usually simply interconnects various pieces of the application, it is usually quite small and simple code:

 package main import ( "log" "os" "github.com/benbjohnson/myapp" "github.com/benbjohnson/myapp/postgres" "github.com/benbjohnson/myapp/http" ) func main() { // Connect to database. db, err := postgres.Open(os.Getenv("DB")) if err != nil { log.Fatal(err) } defer db.Close() // Create services. us := &postgres.UserService{DB: db} // Attach to HTTP handler. var h http.Handler h.UserService = us // start http server... } 

It is also important to understand that the main package is also an adapter. It connects the terminal with your domain.

Conclusion

Application design is a complex issue. It is necessary to take a lot of decisions and without a good set of principles, the problem becomes even worse. We looked at several approaches to structuring Go applications and looked at their shortcomings.

I’m sure that dependency-based code organization simplifies the design and makes the code more understandable. First we define the language of our domain. Then, isolate the dependencies. Next, create stubs for tests. And at the end, we glue it all together using the main package.

Look at this approach in your next application. If you have questions or want to discuss application design, I am available on Twitter at @benbjohnson or as benbjohnson on the Slack Go channel .