Practical use in Go: organizing access to databases
A few weeks ago, someone created a topic on Reddit with the request:
What would you use as Go's best practice for accessing a database in (HTTP or other) handlers, in the context of a web application?
The answers he received were varied and interesting. Some people suggested using dependency injection, some supported the idea of using simple global variables, others suggested putting the connection pool pointer in x / net / context (the context package is used with golang 1.7).
As for me? I think that the correct answer depends on the project.
What is the overall structure and size of the project? What approach do you use for testing? How will the project develop in the future? All of these things, and much more, partly affect which approach is right for you.
In this post we consider four different approaches to organizing your code and structuring access to a pool of connections to a database.
This post is a free translation of the original article . The author of the article offers four approaches to organizing database access in an application written in golang
Global variables
The first approach that we consider is common and simple - take a pointer to the pool of connections to the database and put it in a global variable.
To make the code look beautiful and conform to the DRY principle (Don't Repeat Yourself - rus. Do not repeat), you can use the initialization function that will establish a global pool of connections from other packages and tests.
I like specific examples, let's continue working with the online store database and the code from my previous post . We will consider creating simple applications with MVC (Model View Controller) with a similar structure - with HTTP handlers in the main application and a separate model package containing global variables for the database, the InitDB () function, and our database logic.
bookstore ├── main.go └── models ├── books.go └── db.go File: main.go
package main import ( "bookstore/models" "fmt" "net/http" ) func main() { models.InitDB("postgres://user:pass@localhost/bookstore") http.HandleFunc("/books", booksIndex) http.ListenAndServe(":3000", nil) } func booksIndex(w http.ResponseWriter, r *http.Request) { if r.Method != "GET" { http.Error(w, http.StatusText(405), 405) return } bks, err := models.AllBooks() if err != nil { http.Error(w, http.StatusText(500), 500) return } for _, bk := range bks { fmt.Fprintf(w, "%s, %s, %s, £%.2f\n", bk.Isbn, bk.Title, bk.Author, bk.Price) } } File: models / db.go
package models import ( "database/sql" _ "github.com/lib/pq" "log" ) var db *sql.DB func InitDB(dataSourceName string) { var err error db, err = sql.Open("postgres", dataSourceName) if err != nil { log.Panic(err) } if err = db.Ping(); err != nil { log.Panic(err) } } File: models / books.go
package models type Book struct { Isbn string Title string Author string Price float32 } func AllBooks() ([]*Book, error) { rows, err := db.Query("SELECT * FROM books") if err != nil { return nil, err } defer rows.Close() bks := make([]*Book, 0) for rows.Next() { bk := new(Book) err := rows.Scan(&bk.Isbn, &bk.Title, &bk.Author, &bk.Price) if err != nil { return nil, err } bks = append(bks, bk) } if err = rows.Err(); err != nil { return nil, err } return bks, nil } If you run the application and make a request for / books, you should get an answer similar to:
$ curl -i localhost:3000/books HTTP/1.1 200 OK Content-Length: 205 Content-Type: text/plain; charset=utf-8 978-1503261969, Emma, Jayne Austen, £9.44 978-1505255607, The Time Machine, HG Wells, £5.99 978-1503379640, The Prince, Niccolò Machiavelli, £6.99 Using global variables is potentially suitable if:
- all database logic is in one package;
- your application is quite small and tracking global variables should not cause you problems;
- approach to testing means that you do not need to check the database and do not run parallel tests.
In the example above, using global variables is great. But what will happen in more complex applications when database logic is used in several packages?
One option is to call InitDB several times, but this approach can quickly become clumsy and it looks a bit strange (it is easy to forget to initialize the connection pool and get the null pointer to panic at runtime). The second option is to create a separate configuration package with the exported database variable and import "yourproject / config" into each file, where necessary. If it is not clear what is at stake, you can see an example .
Dependency injection
In the second approach, we will consider dependency injection. In our example, we obviously want to pass a pointer to the connection pool, to our HTTP handlers, and then to our database logic.
In the real world, applications probably have an additional level (competitively safe) in which there are elements to which your handlers have access. These may be pointers to a logger or cache, as well as a pool of database connections.
For projects in which all your handlers are in the same package, a neat approach is to have all the elements in the custom Env type:
type Env struct { db *sql.DB logger *log.Logger templates *template.Template } ... and then define your handlers and methods in the same place as Env. This provides a clean and distinctive way to create a pool of connections (and for other elements) for your handlers.
Full example:
File: main.go
package main import ( "bookstore/models" "database/sql" "fmt" "log" "net/http" ) type Env struct { db *sql.DB } func main() { db, err := models.NewDB("postgres://user:pass@localhost/bookstore") if err != nil { log.Panic(err) } env := &Env{db: db} http.HandleFunc("/books", env.booksIndex) http.ListenAndServe(":3000", nil) } func (env *Env) booksIndex(w http.ResponseWriter, r *http.Request) { if r.Method != "GET" { http.Error(w, http.StatusText(405), 405) return } bks, err := models.AllBooks(env.db) if err != nil { http.Error(w, http.StatusText(500), 500) return } for _, bk := range bks { fmt.Fprintf(w, "%s, %s, %s, £%.2f\n", bk.Isbn, bk.Title, bk.Author, bk.Price) } } File: models / db.go
package models import ( "database/sql" _ "github.com/lib/pq" ) func NewDB(dataSourceName string) (*sql.DB, error) { db, err := sql.Open("postgres", dataSourceName) if err != nil { return nil, err } if err = db.Ping(); err != nil { return nil, err } return db, nil } File: models / books.go
package models import "database/sql" type Book struct { Isbn string Title string Author string Price float32 } func AllBooks(db *sql.DB) ([]*Book, error) { rows, err := db.Query("SELECT * FROM books") if err != nil { return nil, err } defer rows.Close() bks := make([]*Book, 0) for rows.Next() { bk := new(Book) err := rows.Scan(&bk.Isbn, &bk.Title, &bk.Author, &bk.Price) if err != nil { return nil, err } bks = append(bks, bk) } if err = rows.Err(); err != nil { return nil, err } return bks, nil } Or use closures ...
If you do not want to define your handlers and methods in Env, an alternative approach would be to use handler logic to close and close the Env variable as follows:
File: main.go
package main import ( "bookstore/models" "database/sql" "fmt" "log" "net/http" ) type Env struct { db *sql.DB } func main() { db, err := models.NewDB("postgres://user:pass@localhost/bookstore") if err != nil { log.Panic(err) } env := &Env{db: db} http.Handle("/books", booksIndex(env)) http.ListenAndServe(":3000", nil) } func booksIndex(env *Env) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { if r.Method != "GET" { http.Error(w, http.StatusText(405), 405) return } bks, err := models.AllBooks(env.db) if err != nil { http.Error(w, http.StatusText(500), 500) return } for _, bk := range bks { fmt.Fprintf(w, "%s, %s, %s, £%.2f\n", bk.Isbn, bk.Title, bk.Author, bk.Price) } }) } Dependency injection is a good approach when:
- all your handlers are in the same package;
- all handlers have the same dependency set;
- approach to testing means that you do not need to check the database and do not run parallel tests.
Once again, you can use this approach if your handlers and database logic are distributed across multiple packages. One way to achieve this is to create a separate configuration package, the exported type of Env. One way to use Env in the example above. As well as a simple example .
Use of interfaces
We will use the dependency injection example a little later. Let's modify the model package so that it returns a custom database type (which includes sql.DB ) and implement the database logic in the form of a DB type.
We get a double advantage: first we get a clean structure, but more importantly, it opens up the potential to test our database in the form of unit tests.
Let's change the example and enable the new Datastore interface, which implements some of the methods, in our new DB type.
type Datastore interface { AllBooks() ([]*Book, error) } We can use this interface throughout our application. Updated example.
File: main.go
package main import ( "fmt" "log" "net/http" "bookstore/models" ) type Env struct { db models.Datastore } func main() { db, err := models.NewDB("postgres://user:pass@localhost/bookstore") if err != nil { log.Panic(err) } env := &Env{db} http.HandleFunc("/books", env.booksIndex) http.ListenAndServe(":3000", nil) } func (env *Env) booksIndex(w http.ResponseWriter, r *http.Request) { if r.Method != "GET" { http.Error(w, http.StatusText(405), 405) return } bks, err := env.db.AllBooks() if err != nil { http.Error(w, http.StatusText(500), 500) return } for _, bk := range bks { fmt.Fprintf(w, "%s, %s, %s, £%.2f\n", bk.Isbn, bk.Title, bk.Author, bk.Price) } } File: models / db.go
package models import ( _ "github.com/lib/pq" "database/sql" ) type Datastore interface { AllBooks() ([]*Book, error) } type DB struct { *sql.DB } func NewDB(dataSourceName string) (*DB, error) { db, err := sql.Open("postgres", dataSourceName) if err != nil { return nil, err } if err = db.Ping(); err != nil { return nil, err } return &DB{db}, nil } File: models / books.go
package models type Book struct { Isbn string Title string Author string Price float32 } func (db *DB) AllBooks() ([]*Book, error) { rows, err := db.Query("SELECT * FROM books") if err != nil { return nil, err } defer rows.Close() bks := make([]*Book, 0) for rows.Next() { bk := new(Book) err := rows.Scan(&bk.Isbn, &bk.Title, &bk.Author, &bk.Price) if err != nil { return nil, err } bks = append(bks, bk) } if err = rows.Err(); err != nil { return nil, err } return bks, nil } Due to the fact that our handlers now use the Datastore interface, we can easily create unit tests for responses from the database.
package main import ( "bookstore/models" "net/http" "net/http/httptest" "testing" ) type mockDB struct{} func (mdb *mockDB) AllBooks() ([]*models.Book, error) { bks := make([]*models.Book, 0) bks = append(bks, &models.Book{"978-1503261969", "Emma", "Jayne Austen", 9.44}) bks = append(bks, &models.Book{"978-1505255607", "The Time Machine", "HG Wells", 5.99}) return bks, nil } func TestBooksIndex(t *testing.T) { rec := httptest.NewRecorder() req, _ := http.NewRequest("GET", "/books", nil) env := Env{db: &mockDB{}} http.HandlerFunc(env.booksIndex).ServeHTTP(rec, req) expected := "978-1503261969, Emma, Jayne Austen, £9.44\n978-1505255607, The Time Machine, HG Wells, £5.99\n" if expected != rec.Body.String() { t.Errorf("\n...expected = %v\n...obtained = %v", expected, rec.Body.String()) } } Request context (Request-scoped context)
Finally, let's look at the use of context in the scope of the request and the transfer of a pool of connections to the database. In particular, we will use the x / net / context package.
Personally, I'm not a fan of application-level variables in the context of a request scope — it looks awkward and burdensome to me. The x / net / context package documentation also advises:
Use context values only for the visibility of data within the request that the API processes and entry points transmit, and not to pass optional parameters to the function.
However, people use this approach. And if your project contains many packages, and the use of the global configuration is not discussed, then this is quite an attractive solution.
Let's adapt the bookstore example for the last time, passing context to our handlers using the template suggested in the wonderful article from Joe Shaw
File: main.go
package main import ( "bookstore/models" "fmt" "golang.org/x/net/context" "log" "net/http" ) type ContextHandler interface { ServeHTTPContext(context.Context, http.ResponseWriter, *http.Request) } type ContextHandlerFunc func(context.Context, http.ResponseWriter, *http.Request) func (h ContextHandlerFunc) ServeHTTPContext(ctx context.Context, rw http.ResponseWriter, req *http.Request) { h(ctx, rw, req) } type ContextAdapter struct { ctx context.Context handler ContextHandler } func (ca *ContextAdapter) ServeHTTP(rw http.ResponseWriter, req *http.Request) { ca.handler.ServeHTTPContext(ca.ctx, rw, req) } func main() { db, err := models.NewDB("postgres://user:pass@localhost/bookstore") if err != nil { log.Panic(err) } ctx := context.WithValue(context.Background(), "db", db) http.Handle("/books", &ContextAdapter{ctx, ContextHandlerFunc(booksIndex)}) http.ListenAndServe(":3000", nil) } func booksIndex(ctx context.Context, w http.ResponseWriter, r *http.Request) { if r.Method != "GET" { http.Error(w, http.StatusText(405), 405) return } bks, err := models.AllBooks(ctx) if err != nil { http.Error(w, http.StatusText(500), 500) return } for _, bk := range bks { fmt.Fprintf(w, "%s, %s, %s, £%.2f\n", bk.Isbn, bk.Title, bk.Author, bk.Price) } } File: models / db.go
package models import ( "database/sql" _ "github.com/lib/pq" ) func NewDB(dataSourceName string) (*sql.DB, error) { db, err := sql.Open("postgres", dataSourceName) if err != nil { return nil, err } if err = db.Ping(); err != nil { return nil, err } return db, nil } File: models / books.go
package models import ( "database/sql" "errors" "golang.org/x/net/context" ) type Book struct { Isbn string Title string Author string Price float32 } func AllBooks(ctx context.Context) ([]*Book, error) { db, ok := ctx.Value("db").(*sql.DB) if !ok { return nil, errors.New("models: could not get database connection pool from context") } rows, err := db.Query("SELECT * FROM books") if err != nil { return nil, err } defer rows.Close() bks := make([]*Book, 0) for rows.Next() { bk := new(Book) err := rows.Scan(&bk.Isbn, &bk.Title, &bk.Author, &bk.Price) if err != nil { return nil, err } bks = append(bks, bk) } if err = rows.Err(); err != nil { return nil, err } return bks, nil } PS The author of the translation will be grateful for the errors and inaccuracies of the translation.
')
Source: https://habr.com/ru/post/332122/
All Articles