Creating a web application on Go in 2017. Part 2

So, our application will have two main parts: client and server. (What is the year now?). The server part will be on Go, and the client part will be on JS. Let's first talk about the server side.

Go (server)

The server part of our application will be responsible for the initial maintenance of everything necessary for JavaScript and everything else, such as static files and data in JSON format. This is all, only two functionalities: (1) static and (2) JSON.

It is worth noting that static maintenance is optional: statics can be served in a CDN, for example. But the important thing is that this is not a problem for our Go application - unlike the Python / Ruby application, it can work on a par with Ngnix and Apache serving static. Delegating the distribution of static files to some other application is not particularly required to ease the load, although it makes sense in some situations.

For simplicity, let's imagine that we are creating an application that serves a list of people (only first and last names), stored in database tables, and that's it. The code is here - https://github.com/grisha/gowebapp .

Directory structure

As my experience shows, separating the functionality between packages at an early stage is a good idea in Go. Even if it is not entirely clear how the final version will be structured, it’s best to keep everything in its unfolded state whenever possible.

For a web application, in my opinion, this layout makes sense:

# github.com/user/foo foo/ # package main | +--daemon/ # package daemon | +--model/ # package model | +--ui/ # package ui | +--db/ # package db | +--assets/ #    JS    

Top level: main package

At the top level, we have the main package, and its code is in the main.go file. The main advantage is that in this situation go get github.com/user/foo is the only command required to install the entire application in $GOPATH/bin .

The main package should be as minimal as possible. The only code that is here is the analysis of the arguments of the command. If the application had a configuration file, I would place the parsing and verification of this file into another package, which would most likely call the config . After this, main should transfer control to the daemon package.

Here is the basis of main.go :

 package main import ( "github.com/user/foo/daemon" ) var assetsPath string func processFlags() *daemon.Config { cfg := &daemon.Config{} flag.StringVar(&cfg.ListenSpec, "listen", "localhost:3000", "HTTP listen spec") flag.StringVar(&cfg.Db.ConnectString, "db-connect", "host=/var/run/postgresql dbname=gowebapp sslmode=disable", "DB Connect String") flag.StringVar(&assetsPath, "assets-path", "assets", "Path to assets dir") flag.Parse() return cfg } func setupHttpAssets(cfg *daemon.Config) { log.Printf("Assets served from %q.", assetsPath) cfg.UI.Assets = http.Dir(assetsPath) } func main() { cfg := processFlags() setupHttpAssets(cfg) if err := daemon.Run(cfg); err != nil { log.Printf("Error in main(): %v", err) } } 

The following code takes three parameters: -listen , -db-connect and -assets-path , nothing special.

Using structures for clarity

In the cfg := &daemon.Config{} we create a daemon.Config object. Its main purpose is to present the configuration in a structured and understandable format. Each of our packages defines its own Config type, which describes the parameters it needs, and which may include settings for other packages. We see an example of this in the processFlags() above: flag.StringVar(&cfg.Db.ConnectString, ... Here db.Config included in daemon.Config . In my opinion, this is a very useful technique. Using structures also leaves the possibility of serializing settings in the form of JSON, TOML or something else.

Using http.FileSystem to maintain statics

http.Dir(assetsPath) in setupHttpAssets is a preparation for how we will serve statics in the ui package. This is done in such a way as to leave the possibility for another implementation of cfg.UI.Assets (which is the http.FileSystem interface), for example, to give this content from RAM. I will talk about this in more detail later, in a separate post.

In the end, main calls daemon.Run(cfg) , which actually starts our application and blocks until the end of the work.

Package daemon

The daemon package contains everything related to running the process. This includes, for example, which port will be tapped, a user log will be defined here, as well as everything related to a polite restart, etc.

Since the task of the daemon package is to initialize the connection to the database, it needs to import the db package. He is also responsible for listening to the TCP port and launching the user interface for this listener, so he needs to import the ui package, and since the ui package needs to have access to the data provided by the model package, it also needs to import the model package.

The daemon module skeleton looks like this:

 package daemon import ( "log" "net" "os" "os/signal" "syscall" "github.com/grisha/gowebapp/db" "github.com/grisha/gowebapp/model" "github.com/grisha/gowebapp/ui" ) type Config struct { ListenSpec string Db db.Config UI ui.Config } func Run(cfg *Config) error { log.Printf("Starting, HTTP on: %s\n", cfg.ListenSpec) db, err := db.InitDb(cfg.Db) if err != nil { log.Printf("Error initializing database: %v\n", err) return err } m := model.New(db) l, err := net.Listen("tcp", cfg.ListenSpec) if err != nil { log.Printf("Error creating listener: %v\n", err) return err } ui.Start(cfg.UI, m, l) waitForSignal() return nil } func waitForSignal() { ch := make(chan os.Signal) signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM) s := <-ch log.Printf("Got signal: %v, exiting.", s) } 

Note that Config includes db.Config and ui.Config , as I mentioned.

All action takes place in Run(*Config) . We initialize the connection to the database, create an instance of model.Model and run ui , passing it the settings, pointers to the model and the listener.

Package model

The purpose of the model is to separate how the data is stored in the database from the ui , as well as to provide the business logic that the application can have. This is the brain of your application, if you like.

The model package must define a structure ( Model looks like a suitable name), and a pointer to an instance of this structure must be passed to all functions and methods ui . In our application, there should be only one such instance - for additional confidence, you can implement this programmatically using a singleton, but I do not think that this is so necessary.

Alternatively, you can do without the Model structure and simply use the model package itself. I do not like this approach, however this is an option.

The model must also define structures for the data entities with which we are dealing. In our example, this will be the Person structure. Its members must be exported (named with a capital letter), because other packages will access them. If you use sqlx , here you must specify the tags that bind the elements of the structure to the names of the columns in the database, for example, db:"first_name" .

Our Person Type:

 type Person struct { Id int64 First, Last string } 

Here we do not need tags, because the names of the columns correspond to the names of the elements of the structure, and sqlx takes care of the register so that Last corresponds to the column with the name last .

The model package should NOT import db

Somewhat counterintuitively, model should not import db . But it should not because the db package needs to import the model , and cyclical imports are prohibited in Go. This is the case when interfaces come in handy. model must specify an interface that db should satisfy. So far we only know that we need a list of people, so we can start with this definition:

 type db interface { SelectPeople() ([]*Person, error) } 

Our application does not do much, but we know that it lists people, so our model, most likely, should have the People() ([]*Person, error) method People() ([]*Person, error) :

 func (m *Model) People() ([]*Person, error) { return m.SelectPeople() } 

For everything to be neat, it is better to place the code in different files, for example, the Person structure must be defined in person.go , etc. But for readability, here is the single-file version of our model package:

 package model type db interface { SelectPeople() ([]*Person, error) } type Model struct { db } func New(db db) *Model { return &Model{ db: db, } } func (m *Model) People() ([]*Person, error) { return m.SelectPeople() } type Person struct { Id int64 First, Last string } 

db package

db is the actual implementation of the interaction with the database. This is where SQL statements are constructed and executed. This package also imports model , p.ch. he will need to create these structures from the database data.

First of all, db should provide the InitDB function, which will establish a connection to the database, as well as create the necessary tables and prepare SQL queries.

Our simplified example does not support migrations, but, theoretically, this is where they should be performed.

We use PostgreSQL, which means we need to import the pq driver. We will also rely on sqlx and we need our model . Here is the beginning of the implementation of our db :

 package db import ( "database/sql" "github.com/grisha/gowebapp/model" "github.com/jmoiron/sqlx" _ "github.com/lib/pq" ) type Config struct { ConnectString string } func InitDb(cfg Config) (*pgDb, error) { if dbConn, err := sqlx.Connect("postgres", cfg.ConnectString); err != nil { return nil, err } else { p := &pgDb{dbConn: dbConn} if err := p.dbConn.Ping(); err != nil { return nil, err } if err := p.createTablesIfNotExist(); err != nil { return nil, err } if err := p.prepareSqlStatements(); err != nil { return nil, err } return p, nil } } 

The exported InitDb() function creates an instance of pgDb , which is the Postgres implementation of our model.db interface. It contains everything you need to communicate with the database, including prepared queries, and implements the methods necessary for the interface.

 type pgDb struct { dbConn *sqlx.DB sqlSelectPeople *sqlx.Stmt } 

Below is the code for creating tables and preparing queries. From the point of view of SQL, everything is rather simplistic and, of course, there is room for improvement:

 func (p *pgDb) createTablesIfNotExist() error { create_sql := ` CREATE TABLE IF NOT EXISTS people ( id SERIAL NOT NULL PRIMARY KEY, first TEXT NOT NULL, last TEXT NOT NULL); ` if rows, err := p.dbConn.Query(create_sql); err != nil { return err } else { rows.Close() } return nil } func (p *pgDb) prepareSqlStatements() (err error) { if p.sqlSelectPeople, err = p.dbConn.Preparex( "SELECT id, first, last FROM people", ); err != nil { return err } return nil } 

Finally, we need to provide a method that implements the interface:

 func (p *pgDb) SelectPeople() ([]*model.Person, error) { people := make([]*model.Person, 0) if err := p.sqlSelectPeople.Select(&people); err != nil { return nil, err } return people, nil } 

Here we take advantage of sqlx to execute the query and build a slice from the results, simply by calling Select() (Note: p.sqlSelectPeople is of type *sqlx.Stmt ). Without sqlx we would have to iterate through the rows of the result, processing each with the help of Scan , which would be more verbose.

Beware of one very subtle point. people could be defined as var people []*model.Person and the method would work the same way. However, if the database returns an empty rowset, the method returns nil , rather than an empty slice. If the result of this method is later encoded in JSON, then it will become null , not [] . This can cause problems if the client side does not know how to handle null .

That's all for db .

ui package

In the end, we need to serve all of this via HTTP, and this is exactly what ui does.

Here is a very simplified version:

 package ui import ( "fmt" "net" "net/http" "time" "github.com/grisha/gowebapp/model" ) type Config struct { Assets http.FileSystem } func Start(cfg Config, m *model.Model, listener net.Listener) { server := &http.Server{ ReadTimeout: 60 * time.Second, WriteTimeout: 60 * time.Second, MaxHeaderBytes: 1 << 16} http.Handle("/", indexHandler(m)) go server.Serve(listener) } const indexHTML = ` <!DOCTYPE HTML> <html> <head> <meta charset="utf-8"> <title>Simple Go Web App</title> </head> <body> <div id='root'></div> </body> </html> ` func indexHandler(m *model.Model) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { fmt.Fprintf(w, indexHTML) }) } 

Note that indexHTML almost nothing. This is almost 100% all the HTML that our application will use. It will change a little when we proceed to the client side of the application, just a few lines.

Also note how the handler is defined. If this idiom is not familiar to you, it is worth spending a few minutes (or a day) to absorb it completely, since it is very common in Go. indexHandler() is not the handler itself, it returns a handler function. This is done this way so that we can pass *model.Model through the closure, since the signature of the HTTP handler function is fixed and the pointer to the model is not one of its parameters.

While we in indexHandler() do nothing with the pointer to the model, but when we get to the actual implementation of the list of people, we need it.

Conclusion

The above is, in fact, everything you need to know to create a basic web application on Go, at least from Go. In the next article I will deal with the client part and we will complete the code of the list of people.

Continuation