A simple guide for application configuration using the Go language
Introduction
When we develop programmes for ourselves, we frequently include some important options right in the source code, for example, the port on which our server will listen, the name of the database we are using, etc. But what if you try to let people use your programme? There is a good chance that your users will utilise the same database name that you are using during the development stage. They may even choose to use the same server port. Security is another concern. Do you desire everyone to be aware of your database password? It is not a good idea to maintain credentials in your code.
Let's talk about how to prevent these issues and keep our application credentials isolated from the source code using efficient configuration approaches offered by the Go language.
What is the Go language?
Go is a statically typed, compiled high-level programming language. It was created in 2007 at Google by Robert Griesemer, Rob Pike, and Ken Thompson, although it wasn't released as an open-source programming language until 2009. In the Go language, programmes are put together using packages for effective dependency management.
It is a well-liked option for developing scalable network services, online applications, and command-line utilities since it is made to be straightforward, effective, and quick to understand.
It offers explicit support for concurrent programming and is strongly typed and garbage-collected. Packages are the building blocks of programmes, and their inherent qualities enable effective dependency management.
Syntax:
The components of a Go file are as follows:
Package declaration
Import packages
Functions
Statements and expressions
Look at the following instance for a better understanding,
package main
import ("fmt")
func main() {
fmt.Println("Hello World!")
}
Output:
Advantages of the Go language
It is flexible, clear, and simple to read.
Its Concurrency enables many processes to operate efficiently and simultaneously.
It offers a comprehensive standard library.
Garbage collection is a crucial aspect of Go. Go excels at granting extensive control over memory allocation, and the most recent iterations of the garbage collector have significantly decreased latency.
It verifies the embedding of the type and the interface.
Go is made with a focus on performance and low memory utilisation, and it is intended to be quick and effective. This makes it suitable for both the development of high-performance network services and the resolution of challenging computing issues.
Go features built-in garbage collection, which takes care of memory management for you automatically. As a result, manual memory management is no longer necessary, which lowers the risk of memory leaks and other issues that can result from human memory management.
Types are established at compile time because Go is a statically typed language. As a result, type-related defects are easier to detect before they occur and type safety is strengthened.
Go is similar to Java in that it supports platform independence. As the code is compiled and transformed into binary form, which is the smallest possible format, and because of its modular design and modularity, it doesn't need any dependencies. Any platform, server, and application you use can build its code.
Real-time applications of the Go language
Docker: a set of tools for deploying Linux containers
Openshift: a cloud computing platform as a service by Red Hat.
Kubernetes: The future of seamlessly automated deployment processes
Dropbox: migrated some of its critical components from Python to Go.
Netflix: for two parts of their server architecture.
InfluxDB: is an open-source time series database developed by InfluxData.
Why do we choose the Go language for Configuration?
Go includes a large number of packages that can handle application configurations effectively. With Go, we can potentially spend less time configuring the app and more time developing it. Go accomplishes this by ensuring that we can swiftly build up our settings.
The main purpose of using the Go application for configuration is to secure the sensitive data of your software by improving security and preventing anyone from discovering the values kept in repositories such as secret keys, database credentials, credentials required to access apps from third parties, and many other types of sensitive information are kept there.
Go language supports three effective approaches for application configuration. Let’s explore each and everyone with examples.
Three ways to do configuration in the Go application
Flags
Using flags is the simplest yet verbose method of giving values to programmes. These are entered into the command line as parameters just once, during startup.
$ ./mywebapp --port=8080
They are suggested in situations where a configuration file would be cumbersome or where the settings don't change during execution, such as in ephemeral contexts where resources are rebuilt at each deployment.
In some circumstances, such as when expressing the location of a configuration file or quickly altering the behaviour of an app, flags are the ideal choice. They should not be used for secret settings, though, as command lines leave traces in log files and automation programmes like Jenkins.
The implementation of a flag in Go is demonstrated in the example below:
package config
import (
"flag"
)
type Config struct {
DBHost string
DBPort string
DBUser string
DBPassword string
// ...
}
func Parse() Config {
dbHost := flag.String("db-host", "localhost", "database host.")
dbPort := flag.String("db-port", "5432", "database port.")
dbUser := flag.String("db-user", "postgres", "database user.")
dbPass := flag.String("db-password", "postgres", "database password.")
flag.Parse()
c := Config{
DBHost: *dbHost,
DBPort: *dbPort,
DBUser: *dbUser,
DBPassword: *dbPass,
}
return c
}
Using flags has the advantage of offering documentation through usage arguments. Moreover, default values can be offered without the need for helper functions.
For more information on the purpose of the flags, we can run our programme with the -h flag. It will print out the usages.
If necessary, we may also specify the flags using a run command that makes use of environment variables.
go run main.go -DATABASE_URL $DATABASE_URL -ORDER_SVC_ADDR $ORDER_SVC_ADDR -DATABASE_MAX_IDLE_CONNECTIONS $DATABASE_MAX_IDLE_CONNECTIONS
It can be tested as follows:
package config_test
import (
"code.com/config"
"os"
"testing"
)
func TestParse(t *testing.T) {
os.Args[1] = "-db-host=hostname"
os.Args[2] = "-db-port=1234"
os.Args[3] = "-db-user=user"
os.Args[4] = "-db-password=pass"
c := config.Parse()
if c.DBHost != "hostname" {
t.Errorf("Expected dbHost to be 'hostname'. Got %s", c.DBHost)
}
if c.DBPort != "1234" {
t.Errorf("Expected dbPort to be '1234'. Got %s", c.DBPort)
}
if c.DBUser != "user" {
t.Errorf("Expected dbUser to be 'user'. Got %s", c.DBUser)
}
if c.DBPassword != "pass" {
t.Errorf("Expected dbPassword to be 'pass'. Got %s", c.DBPassword)
}
}
For CLI applications, using flags would be the preferred technique.
Environment variables
Some cloud services offer environment variable configuration capabilities. It's simple to set environment variables using the export command. Using the environment variable in the application is substantially important when developing production-grade applications.
Consider a programme with numerous functionalities, each of which requires access to a database. You set up each feature's DB information, including DBURL, DBNAME, USERNAME, and PASSWORD.
The environment variables can be divided into categories like PROD, DEV, or TEST. Just add the environment as a prefix to the variable.
A typical OS package offered by GO contains functions that can be used to interact programmatically with environment variables. The functions that can be used to communicate with environment variables are listed below.
os.Setenv()
This function adds value to the system environment. To set an environment variable, you must give a key and value pair.
os.Getenv()
If you need to retrieve environment variables, the critical parameter that you must give will retrieve the value of the key, if it exists.
os.Clearenv
The environment variables are all removed by this function.
Let's put this strategy into practice.
Let's assume that the database host, port, user, and password are the only 4 configuration parameters.
package config
import (
"os"
)
type Config struct {
DBHost string // Host of database server
DBPort string // ...
DBUser string
DBPassword string
// ...
}
func Parse() Config {
c := Config{
DBHost: getEnvOrDefault("DB_HOST", "localhost"),
DBPort: getEnvOrDefault("DB_PORT", "5432"),
DBUser: getEnvOrDefault("DB_USER", "postgres"),
DBPassword: getEnvOrDefault("DB_PASSWORD", "postgres"),
}
return c
}
func getEnvOrDefault(key string, defaultVal string) string {
v := os.Getenv(key)
if v == "" {
return defaultVal
}
return v
}
Here, a Config struct with the variables and a Parse function is defined. To set default values or parse other types of variables, we can construct helper functions like getEnvOrDefault.
Moreover, we may initialise our configuration as follows in our main.go file:
package main
import "github/mtekmir/go-config-management/config"
func main() {
conf := config.Parse()
// ...
}
We can quickly test the configuration package.
package config_test
import (
"os"
"testing"
"code.com/config"
)
func TestParse(t *testing.T) {
t.Cleanup(func() {
os.Clearenv()
})
os.Setenv("DB_HOST", "hostname")
os.Setenv("DB_PORT", "1234")
os.Setenv("DB_USER", "user")
os.Setenv("DB_PASSWORD", "pass")
c := config.Parse()
if c.DBHost != "hostname" {
t.Errorf("Expected dbHost to be 'hostname'. Got %s", c.DBHost)
}
if c.DBPort != "1234" {
t.Errorf("Expected dbPort to be '1234'. Got %s", c.DBPort)
}
if c.DBUser != "user" {
t.Errorf("Expected dbUser to be 'user'. Got %s", c.DBUser)
}
if c.DBPassword != "pass" {
t.Errorf("Expected dbPassword to be 'pass'. Got %s", c.DBPassword)
}
}
I want to evaluate products from the standpoint of the customer. We can test utilising the public API of the package by designating the package of tests as **_test. In this approach, until the behaviour of the package is altered, the tests won't need any modification. This allows us to refactor the internal implementation without affecting the tests.
Configuration files
The most easily maintained method of keeping configuration is through configuration files. These are text files containing documentation and validation that are structured and have these features. They can be versioned, but they must do so in a separate repository because doing so in the application's repository would propagate changes throughout all environments and make information about secrets public (e.g. database password). Any file format, including JSON, YAML, and TOML, may be used in this approach.
Consider a server that executes numerous cron jobs. JSON files can be used to configure them.
We may parse the configuration information from the JSON file after storing the cron job configuration in a separate struct.
package config
import (
"encoding/json"
"flag"
"fmt"
"io/ioutil"
"os"
)
type Config struct {
// ...
CronConfigs CronConfigs
}
type CronConfigs struct {
InventoryCron CronConfig `json:"inventoryCron"`
InvoicesCron CronConfig `json:"invoicesCron"`
}
type CronConfig struct {
Schedule string `json:"schedule"`
Description string `json:"desc"`
Disabled bool `json:"disabled"`
NotifyEmail []string `json:"notifyEmail"`
}
func Parse() (Config, error) {
// ...
cronConfPath := flag.String("cron-config-file", "cron_config.json", "path of cron config file")
flag.Parse()
file, err := os.Open(*cronConfPath)
if err != nil {
return Config{}, fmt.Errorf("failed to open config file. %v", err)
}
bb, err := ioutil.ReadAll(file)
if err != nil {
return Config{}, fmt.Errorf("failed to read config file. %v", err)
}
var cc CronConfigs
if err := json.Unmarshal(bb, &cc); err != nil {
return Config{}, fmt.Errorf("failed to unmarshal config file. %v", err)
}
conf := Config{
CronConfigs: cc,
}
return conf, nil
}
Let's configure the config file path so that, if necessary, we can specify an alternative one. The file will then be opened and its contents decoded.
And in our main.go we can initialize our config just like before.
package main
import (
"code.com/config"
)
func main() {
conf, err := config.Parse()
if err != nil {
// ...
}
}
By generating a configuration file in the config/testdata directory, we can test this.
With the -cron-config-file flag, we can now pass the path of the test configuration file in our test.
package config_test
import (
"os"
"testing"
"code.com/config"
"github.com/google/go-cmp/cmp"
)
func TestParse(t *testing.T) {
expectedConf := config.Config{
CronConfigs: config.CronConfigs{
InventoryCron: config.CronConfig{
Schedule: "30 0 * * *",
Description: "Cron to calculate inventory stats",
Disabled: false,
NotifyEmail: []string{"jdoe@gmail.com"},
},
InvoicesCron: config.CronConfig{
Schedule: "10 0 * * *",
Description: "Cron to generate invoices",
Disabled: true,
},
},
}
os.Args[1] = "-cron-config-file=testdata/cron_config.test.json"
c, err := config.Parse()
if err != nil {
t.Fatalf("failed to parse config. %v", err)
}
if diff := cmp.Diff(expectedConf, c); diff != "" {
t.Errorf("Configs are different (-want +got):\n%s", diff)
}
}
For comparing complex types in tests, I employ go-cmp. In my perspective, it shows differences more clearly than testify/assert.
Conclusion
A vital component of any application is configuration support. It ought to be one of the first things you learn while learning to code and the first thing you carry out when starting a new project.
Secrets like passwords, access tokens, and encryption keys are frequently found in configuration variables. That might be harmful if those secrets were to leak. There are some efficient open-source solutions available for this specific issue such as the Go language. They provide a wide range of packages to safeguard, keep an eye on, and verify how your secrets are being used.
Go includes a large number of packages that can handle application configurations through flags, environment variables, and configuration files.
Go is an attempt to combine the efficiency and safety of a statically typed, compiled language with the programming simplicity of an interpreted, dynamically typed language. With capabilities for networked and multicore computing, it also aspires to be contemporary.
Comments
Post a Comment