Golang project structuring — Ben Johnson way
Project code organisation is an ever evolving problem. As all wise developers put it, “it always depends on the requirement”. But following a standard structure will help in keeping the code base cleaner and improve the productivity of the team. Ben Johnson proposes one such standard in his article here which I highly recommend to give a read before going through this. In this article I just tried to explain the concepts involved in this structuring with a simple example. Lets get started!
Note: In order to concentrate more on the concept, i have picked a simple application called getpets that provides CRUD functionalities for pets. For such a simple application, this code organisation will be a overkill. We may just go with a flat application structure. But here I used this structuring just to explain the concepts. You can refer to wtf, chronograf to see how larger projects are using this structuring.
Ben Johnson purposes 4 principles to structure our code.
1. Root Package is for domain types
2. Group subpackages by dependency
3. Use a shared mock subpackage
4. Main package ties together dependencies
I also borrowed few concepts from standard go project layout like having our executable binaries under cmd dir and all the other packages inside the pkg dir. We can have separate package at root level for other project files like config, resource, CI/CD etc,. The code used in this blog can be found in here. Our final project structure looks something like this.
Now let’s go in detail of what above principles mean and also see how we could structure our getpets application to adhere to these principles.
Root Package is for domain types:
This principle is similar to Domain Driven development. Here, we define all the objects involved in our domain and the set of interactions/operations we perform on those objects inside a separate package. This package should not have dependency to any of the other packages in our project. Below is the domain definition of the getpets app.
We have defined our domain object Pet struct. Also we have declared the operations on the domain object as interfaces. Why interfaces? This way of defining the required operations as interfaces allows us to separate out our domain concepts from our underlining implementation. Our domain definition requires it needs a PetService that satisfies the requirement of performing CURD operations of Pet and is independent of how it is actually implemented. This way of separating the implementation details is called hexagonal-architecture. In hexagonal architecture, we use the terminologies, ports and adaptors to define the separate components. Ports denote the contract(PetSvc and PetDB interfaces) we have declared in the domain. Adaptors denote the components implementing these contracts. This kind of styling helps us remove circular deps and also makes testing easier(more on this later).
Group Subpackages by dependency:
Next, we group all the subpackages based on their functionalities. All our business logic can go into a package called app, database logic go into a package called db, etc. In our getpets app, we can see the subpackages like app, db, http, mock under pkg dir. An important point to note here is that all our subpackages must communicate between them only using our common domain language. As we saw above all these subpackages are adaptors/implementer to the ports/interfaces defined in our domain package. A subpackage PACK1 depending on PACK2 should only speak in terms of the interfaces defined in our domain package for PACK2. This helps us maintain a pluggable architecture and swap the implementation details at any time without affecting the dependent packages.
Let’s make this more clear with an example from the getpets app, we have petSvc defined in app package that implements the PetSvc interface defined in our domain package.
The petSvc satisfies the requirement of CRUD operation of Pets as defined by the PetSvc interface in our domain package. But to satisfy this requirement it needs a storage system of type PetDB interface defined in the domain package, (i.e., app package now depends on the db package). This dependency is defined in the petSvc with the type domain PetDB(interface). With this we can decouple petSvc from the implementation of PetDB. Now we can inject any implementation (may it be mongo, mysql, redis, etc) that satisfies PetDB interface into petSvc without needing any change in petSvc. In getpets app we have two implementations for our PetDB interface, an in-memory based store and a mongo based store.
Notice that both the implementations return only an object of type domain.PetDB.
Use a shared mock subpackage:
As our system subpackages communicate only via domain package language, we can easily inject mock implementations for any of the layer for testing purpose. Below is a mock implementation for the service layer of our getpets app,
Now if we want to test only the http functionalities like routes and handlers we can mock the service layer and only test that particular functionality. We will check the code for the same along with the explanation of next principle.
Main package ties together dependencies:
With all different subpackages implementing our domain package interfaces, it’s the duty of the main package to put everything together. In our main package we create an instance of subpackage for our required implementation and inject them into the dependent subpackage. Below is the main package of our getpets app-
from line 29–34, we build our getpets server with a mock version of PetSvc. As we discussed above we have injected the mock service layer to test only the http functionalities. In this was can plugin any version of the depended layer without any code change to the dependent layer.
from line 38–46 we are building our actual server. We are creating an instance for the mongo implementation of our PetDB and injecting it into the instantiation of the PetSvc followed by adding routes and starting the server. It is at the main package that we decide which version of a subpackage we need to inject.
Below are the advantages of following such pattern:
1. Testability:
With such a pluggable system, we can test the functionality of each layer separately by injecting a mock version of the dependent layers. We saw an example of mocking the service layer and testing only the http layer independently. You can also find in the repo a example test for the service layer along by injecting an in-memory version of the database without needing any db setup.
2. Clear separation between layers:
In our domain package we can see an interface for each layer in our application. This helps us to have a clear boundary between each layer.
3. Avoids circular dependencies:
As all our subpackages only speak in terms of our domain package language, there is no need for us to import any subpackage inside any other subpackage and hence we will not face the issue of circular dependencies.
4. Team wide understandable domain language:
Having a common domain language throughout the system that is consistent across dev and product team helps to improve team productivity.
Below are the disadvantages that We see with this pattern:
1. Affects readability:
Another approach for structure code is to group packages by module. With this we group all the dependencies of objects inside a same package. In getpets app, we would have a package call pets under which we would have the db, service, handlers and routes related to pet object defined. This helps in better readability as all related code is together.
2. Overkill for small project:
This is a disadvantage with any structuring other than flat application structure when it comes to small projects. This kind of separation of layers and pluggable architecture is very helpful when it comes to larger projects where we may need to test each layer independently and also to tackle the circular dependencies. But it is definitely an overkill for small projects.
That’s it for this blog. Please feel free to comment your views on this blog. Thanks for your time reading this blog and hope it was useful.
Happy learning and sharing.
Resources:
https://medium.com/@benbjohnson/standard-package-layout-7cdbc8391fc1
https://peter.bourgon.org/go-best-practices-2016/
https://www.youtube.com/watch?v=oL6JBUk6tj0
https://blog.gopheracademy.com/advent-2016/go-and-package-focused-design/
https://www.youtube.com/watch?v=MzTcsI6tn-0
https://www.youtube.com/watch?v=6SBjKOwVq0o
https://changelog.com/gotime/102
