A Cake for Kotlin: Exploring framework-less dependency injection for Kotlin
This article is about how to do dependency injection without frameworks in Kotlin, but let’s start out discussing Scala. Since 2009, a small but highly successful team at Knowit has been delivering solutions to one of our customers using Scala as their backend programming language. More than 20 such applications are currently in production. The team has seen major changes to its composition every year, with people joining and leaving. To date, about 35 contributors have worked on the projects, meaning that an average of about 3 people have had to learn to work with the tech stack in use every year. To make things even more difficult, almost no one has had prior knowledge of Scala before joining the team, and more than half have joined the team straight out of university.
The ability to deliver high quality solutions under these conditions is completely dependent on the project being “beginner-friendly” — including the technology stack and the code. There were many strategies and choices involved in how we achieved this (most of them on the interpersonal level), but when it comes to the code base the following points have been important:
- High unit test coverage, achieved by making internal control classes easily mockable — it turns out most developers are happy to write tests if it is easy to do so.
- An easy-to-read code style, enforced through code-reviews, with emphasis on readability and maintainability over terseness and fancy language-features.
- Using the language compiler to statically check correctness where possible.
- As few as possible third party frameworks and libraries — reducing the amount of things to learn for a new developer.
- No “magic” frameworks. Code does what it looks like it does, and dependencies and delegation is clear enough that you never wonder where an implementation comes from.
To achieve testability, and therefore unit test coverage, we need easy to use and understand dependency injection/inversion of control. Regrettably, dependency injection in JVM-languages is also often something that interferes with the rest of our list: Frameworks often rely on automatic injection of delegates, which can often appear “magical” to someone without experience with the framework in question — where does something come from that’s declared as “by inject” or “@Autowired”? Such injection also happens at runtime in many frameworks, leading to situations where the production environment complains about “missing beans” or the like, even though the compiler — and sometimes even the test environment — was entirely happy. Annotation- and/or XML-hell either clutters up code or hides away connections in files that aren’t obviously connected to the code you’re reading.
I’m painting the picture blacker than it needs to be. Clearly, it’s possible to write successful programs with frameworks such as Spring, Guice, Koin, Dagger, and so on. And in some cases the more advanced features that some of these frameworks bring to the table may be needed. But the challenges are still real, and in the aforementioned team at Knowit, we use no such frameworks. Instead, we rely on the language mechanisms of Scala to allow us to do dependency injection for us.
The Cake Pattern
The Cake Pattern is a fairly well known way of modularizing Scala code to achieve inversion of control/dependency injection in a way where the injections are checked by the compiler, without the need for frameworks. The pattern is controversial, with some going as far as calling it an anti-pattern. While I don’t agree with all of the points made against it, I freely admit that it isn’t a silver bullet, and that it does have some issues. The way we’ve used it at Knowit however, it has been a great boon, and central to our success with Scala.
Lets have a look at how the Cake Pattern works:
For every “module” in your code, define a trait (basically a Scala interface which allows implementation) with only the signatures of the methods that module will be providing to other modules. This trait is the Service Definition. As an example, it could be named SomethingService.
Define another trait, the Service Component, where you list all other modules that this module depends on as self-type requirements: Other traits that must be included when using the current trait, even though it doesn’t inherit from them. Continuing our example, it could be called SomethingServiceComponent. The Service Component also defines an abstract constant typed with the Service Definition — this constant will be how other modules access this one, and overriding it (and thereby initializing it) is how we perform injection.
Finally, define a Service Implementation class as an inner class to the Service Component trait, which implements the Service Definition and does the actual work. As an inner class of the Service Component, the implementation class gains access to the enclosing Service Components self-types, and can freely call methods on other components, without the need of passing an object of the component type into the implementation class.
As an example, let’s look at a Hello World service which depends on another service (not listed here) that allows the lookup of names based on person ids:
An application based on the Cake Pattern comes together in the Component Registry class, which inherits every Service Component in the program, and initializes the constants declared by each of them. By subclassing the component registry, we can replace some or all of the services with mocks, making unit testing easy.
That’s basically it. You may have noticed the main flaw of the Cake Pattern already: There is a fairly large amount of “ceremonial” code involved in achieving what we want. There are some other real difficulties as well: Significantly, it can be hard to access methods that aren’t exposed through the Service Definitions for testing — doing so usually involves type casting to the nested classes. And we suspect that the amount of self-types that need to be tracked in a large system can make the compiler’s work slow and memory-consuming — a suspicion we’ve as yet not been able to confirm through experiment, but which is based among other things on the need to increase the compilers available memory in some of our projects.
Even with these challenges however, more than ten years of experience with it at Knowit show that when used properly, Cake Pattern code can be used to create readable, testable, statically checked code, which is easy to understand for new developers.
Moving to Kotlin
I love Scala — I think it’s an awesome programming language, and it has a lot of features I miss when I write other JVM languages. But it does have some issues in practice: Firstly, it is possible (and sometimes even tempting) to write code in ways that are hard to understand for the uninitiated. Second, compile times can become unmanageable as a code base increases in size. These two things make Kotlin look very enticing in comparison, and new projects started at Knowit today most often choose Kotlin over Scala.
As someone who’s gotten used to the benefits (and inured to the difficulties) of the Cake Pattern however, the options for dependency injection most commonly available in Kotlin aren’t very tempting. Frameworks seem to be the way most people have chosen to go, and many of the more popular ones either have “magic” syntax, perform the dependency injection run-time, or both.
The only framework-less alternative I have been able to find discussed online is using the Service Locator pattern. I prefer to avoid depending on default values, however — they seem like you’re just waiting for you to forget to mock something in a test. (The astute reader will notice that this can be a problem with the Test Registry approach to the Cake Pattern as well, although in that case the problem is at least localized to only the Test Registry.) And while I agree with that article with constructor injection being preferable to field injection, I dislike needing a constructor parameter for every dependency. In some of the larger systems I’ve worked on, modules have depended on up to 15–20 other modules, and that many arguments seem untidy to me. Thus, I started experimenting — and I’ve found a way to take inspiration more closely from the Cake Pattern when modularizing Kotlin Code. The result is a kind of Cake Pattern for Kotlin — a “Kake” if you will.
Replacing Self-types
When trying to implement the Cake Pattern in Kotlin, the first issue that has to be solved is the lack of self-types in that language. This is fairly easily circumvented, we can use subtyping on the component definition level (the Service Component in the previous example) instead of self-typing: In the standard cake pattern, a CarService might depend on a RegistrationNumberService, expressed as the CarServiceComponent having the RegistrationNumberServiceComponent as a constraint on its self-type. In our variant, we’ll instead let CarServiceComponent inherit from RegistrationNumberServiceComponent. As the components are interfaces with an abstract constant representing the service implementation, the result is a component interface where all the relevant constants are available — much as if we were using self-types.
The constants being abstract is paramount here. If they were initialized, inheritance would create a heap of problems — or dare I say a “Diamond Problem”. In short, we avoid that because the abstract properties are realized at the same time, so there’s only one implementation that any references can be pointing at.
The Registry will be a subclass of all the components, and the resolution of the abstract property naming collisions will allow us to inject the actual dependency once and for all across the application.
I’ll give you a code example in a moment, but let’s consider a couple of issues first: Purists will look askance on the solution because we’re using inheritance without an obvious is-a relationship between the types, but we’ll ignore them for now. More importantly, the components in the original Cake Pattern actually contain the implementation classes as inner classes. That already creates issues in the original, such as when trying to access the implementation class to test methods that aren’t exposed through the Service Definition. When using inheritance on the components to represent dependencies, this leads to a nightmare of inherited implementation classes, a potentially confusing situation that we want to avoid (even if the language perhaps handles it gracefully).
Constructor Injection to the rescue
We’ll let ourselves be inspired by many modern dependency injection frameworks, and apply constructor injection to the problem. Each service implementation class will have a single property constructor parameter, typed with its own Component Definition interface. This allows us to access the dependencies through that parameter.
The example from before will now look like this:
This doesn’t look too different from the original cake pattern. It even has some improvements: The service implementation is no longer a nested class, meaning easier access to methods not in the Service Definition for testing purposes. Of course, such easy access does mean that undisciplined coders can slightly more easily shoot everyone in the foot by creating spaghetti code. That does seem like a fair tradeoff, however.
What about the component registry?
Again, not too much has changed, except that we move the service implementations into the actual final production registry in order to pass the object reference to the constructors. The test definitions potentially become quite a bit simpler, as we can inherit from only the relevant component and instantiate the relevant services as mocks, instead of needing to mock the entire cake.
Doing that in the original cake pattern would have required us to write quite a bit of extra code. Of course, there are ways around that when using the cake pattern, such as creating a completely mocked test registry and then subclassing that test registry, overriding the one class you want to test — but creating more focused test registries as above are at least as easy.
A note on nested classes, class names and logging
One of the changes we’ve made is to avoid having implementation classes nested within component traits. Apart from making it simpler to access the implementation classes directly for testing, this has positive effects on logging: The canonical names of the implementation classes are now (much) shorter, making logs that include such names much more readable.
What about cycles?
Using inheritance instead of self-types creates another limitation that we haven’t discussed yet: We no longer allow circular dependencies. If module A depends on module B, and module B depends on module A, self-typing is entirely happy — but when using inheritance the compiler will start complaining about cyclical type hierarchies. Losing circular dependencies doesn’t have to be a big deal — indeed, circular dependencies are a known code smell and should be avoided anyway, and preventing them in the first place may be seen as a boon.
On the other hand, most large projects I’ve worked with do have some circular dependencies, and I’m afraid that not allowing them at all will lead to situations where you have to perform large (and therefore expensive) refactorings when it turns out you’ve chosen the wrong modules — which will lead developers in a rush to create hacky solutions that bypass the dependency injection altogether. Yes, in a perfect world, developers aren’t stressed or rushed, and we always have time to fix mistakes even on such a high level as modularization of our code. I’ve yet to work on a project that looks anything like such a perfect world, so I’d prefer to allow circular dependencies despite their inherent issues.
Our new “Kake Pattern” can be easily modified to allow for such dependencies: We remove the inheritance between component definitions (making the type system purists happier as a bonus), and type the constructor parameter with the Component Registry interface. This can be viewed as depending on the entire registry instead of just the components we need. The Component Registry still inherits from all the components in the code (keeping the purists slightly annoyed), but as all inheritance is now in one place, we avoid any problems with cycles.
With these changes, our hello-world example will now look like this:
The component and test registries from before do not change at all in this small example.
This change has both positive and negative sides. Apart from allowing circular dependencies, there are some other consequences:
Less work needed for new dependencies
The change allows every component to easily access any other component listed in the registry, meaning that there is less work involved in introducing a new dependency. Need a module that’s not already used in this module? Just access it through the registry, no ceremony required.
This may have a downside: It could make it easier to add dependencies you wouldn’t need if you put a bit of thought into restructuring your program, creating spaghetti code. You have to consider the level of freedom and responsibility. If you want less of both, keep the inheritance that documents your dependencies, forces you to think a bit more before adding new ones, and prevents you from defining cycles.
Dependencies are no longer documented at component level
The change removes explicit documentation of what components a given component depends on. The dependency is still statically checked, so I think it is possible to live with this, but it’s certainly more work to figure out what the module you’re working on depends on — and what might depend on it.
The modules become slightly more tightly coupled
The change creates a type reference from every component to the component registry, making the system a bit more tightly coupled. The only practical concern I’ve found with regards to this is that every single component must be specified (or mocked) when testing a component, not only the components that are depended on.
To avoid writing a lot of code, this can be handled by using a “TestRegistry” that mocks out everything, and then reintroducing the implementation you want to test in each specific test. This is actually a solution you’d probably be using with the original self-typed cake pattern as well (although we glossed over it in our earlier example), so it’s not a problem with regards to the original cake pattern — but it is a weakness compared to the inheritance-based version.
Compared to the original cake pattern we are actually better off: In the original version, the component initialization is typically done in the Component Registry itself, and the Test Registry would inherit from the Component Registry. That means the compiler can’t tell you if you forgot to replace a module with a mock in the test registry. In our “Kake”, component initialization is still performed in the classes that implement the Component Registry interface, meaning that real implementations won’t “leak” into tests from the production registry.
The best of both worlds
It is possible to mainly write a program in the style that keeps documentation and prevents cycles, and then have only components that actually need circular dependencies use the second style — depending directly on the Component Registry. This is probably the best way to go about it, as it seems to maintain the best of both worlds: Write code as if cyclical dependencies aren’t needed, until they are — then change only the code that actually needs cycles into the second style.
Consider an example where you have four services: One SSN service that deals in matters regarding social security numbers, one Parent-service that deals with matters regarding parents and one Child-service that deals with matters regarding children. Finally, a Family service that deals with families. The Child and Parent services need to depend on one another, and they both depend on the SSN service. The Family service depends on the Child and Parent services. The component interfaces and implementation classes in this case could look like this:
Only the cyclically dependent classes need to depend directly on the registry type. The other services, both above and below the interdependent classes, can use injection typed with their own components just fine.
Conclusion and further work
We have achieved a “Cake for Kotlin”: It is clearly possible to write dependency injection in Kotlin in a way that gives us the advantages of the cake pattern; avoiding the need for frameworks and keeping the compiler’s ability to check that we’ve actually instantiated all our service dependencies. (As a bonus, the same approach should be possible in Java with only minor adaptions, and you could even use it in Scala if you prefer this approach to the original Cake Pattern.)
There are two ways of doing it: One keeps the documentation of dependencies on the component level, and prevents coupling to the component registry, making for somewhat simpler unit testing, but prevents circular dependencies. The other allows for circular dependencies, at the cost of slightly tighter coupling and making the actual dependencies of a module less visible. We have seen that these two styles can be easily combined in the same project, giving you documentation in most places, but greater freedom where you need it.
I have tentative plans for future blog posts on this subject. Possibilities include:
- Presenting a bigger example of code modularized in the style presented in this article, including tests and perhaps integrations with frameworks.
- Looking at compilation performance, which we suspect is a problem with the original Cake Pattern.
- Comparisons with commonly used dependency injection frameworks.
If you have questions or thoughts, or there’s something you’d like to hear about in a future blog post, I would love to hear from you. Feel free to leave a response!
Also read:
