Quality code: Node.js design patterns and dependency management

Hello everyone, and welcome to the Sprkl expert talk with Luciano Mammino, the author of Node.js Design Patterns.

This time we’ll discuss dependency management around Node.js design patterns. In our expert talks, we host a prominent developer in each episode and explore topics that would bring value to the developer community. 

I’m Raz, a software engineer at Sprkl Personal Observability, and I’m the one who came up with those questions in bold. 

Sprkl is a Personal Observability platform that provides personalized feedback on your code changes on your Node.js projects while coding in the IDE. As a result, Sprkl helps ship correct and efficient code while spending less time on debugging, code review, and frustrating rework. (Powered by OpenTelemetry). 

Check out Sprkl on our website and on the VS Code marketplace. It’s really cool stuff. 

Key takeaways 

• Adopt the habit of developing software “lean.” First, start lean and add features to it with your middleware library. Then modulize it into smaller packages with independent features. (You usually see this mostly in monolith applications).

• Keep the core library lean and create a wrapper library for new feature requests. So users can rely on the small core or the heavier wrapper with more dependencies. 

• Use frameworks that help keep “things” small and add functionalities based on your needs. (Express.js if you’re using Node.js). 

• Start with one bulky server, and when it starts getting large, make it independent based on the domain of every part of it. 

• Dependency management differs in Node.js and JS; it’s a controversial topic.

• Developers forget the dependency versus isolation concept. 

• Don’t use a dependency injection container in JavaScript because it’s easy to mock the imports. 

• Consider it a dependency when you have to do external calls, for instance, to a database that might be part of another library. 

• Refrain from limiting yourself to unit testing when managing dependencies. Two pieces of functionalities that are unit-tested correctly might break in different ways when put together.
  
• Unit testing is very complex to do correctly because the behavior might change since you’re dealing with an external system where you can only guess what’s happening. 

• Consider integration and end-to-end tests when getting closer to production. Integration testing covers many issues you could find when testing manually or that later would show up in production. 

The first question is related to object-oriented, functional code, dependency management, and design in general. For example, when you write a new module or a new library (a module can be a single file that does something or a library, a package, or something like that).

What decisions and principles do follow to ensure you’re doing the best work? Of course, every library is different, but we all have some habits that we apply to ensure we deliver high-quality code. What are the best practices for module and library development?

🦁Yeah, it’s a good question. I don’t have a structured process like a checklist of things I want to do. I use more of an iterative process. 

I start with a clear enough use case and try to fully understand the context of the problem I’m trying to solve with this library. 

Then I come up with the bare minimum to write to solve that problem. To clarify your use case: ask yourself a bunch of questions like: 

• Is this going to be a library where I’m going to be the only user? Or is this a library that other people will be using as well? And if they do, in which context will they use it? 

•  If I’m giving this to somebody else and I’m not sure what they will use it for: how flexible should this be? Should I go for a minimal API and clarify that I want to solve only one problem?

• Is there room for being more generic, trying to have a slightly moreover API that allows people to solve a more extensive range of problems? 

Since It’s hard to have strict rules around this issue, depending on the project and the library, you’ll need to evaluate your decisions, and it’s good to have some principles for guidance. Design principles is a better term.

A library is built to be the fastest in a particular thing. So every time you evaluate a change, you need to consider that principle. If the change is going to compromise performance, you’ll probably need to reject that change or invest more time figuring out how to implement the change in a way that will not affect performance. 

Another use case can be: I want this library to be the smallest without any external dependencies. Now, you can evaluate again and ask yourself: 

Is there a new feature request asking me to implement a very complex thing that may require me to import additional libraries? Don’t do it in the same library: create a wrapper library that gives you all the extra bells and whistles. This way, users who need the small core can still rely on the core library. And users that want more features can rely on the heavier wrapper library with more dependencies. 

Can you give an example of creating a lightweight core package, which doesn’t do anything except the middleware functionality?

🦁I worked on the Middy project, a middleware framework for AWS Lambda. Using Node.js is similar to Express.js because you can indicate that all the preconditions and postconditions live somewhere else and are reusable and tested. This way, you can focus on the core business logic and attach these things around it. So it’s like a middleware pattern. 

The first version of Middy was 0.X. It was this extensive monolithic library that the more features we added, the bulkier it got. And in serverless, that was affecting people’s performance because you get longer cold starts. So, when we went for version one, we decided to split this project, still using it as a monorepo but published as a bunch of independent packages. There will be a very lightweight core package, which doesn’t do anything except the middleware functionality. Then all the other packages are specific middlewares that solve the most common problems, but you can only install them only if you have that particular problem. You want to use that specific middleware. 

Develop software “lean.” So, you begin very lean and slowly add features to it with your middleware library. Then you continue adding features until it gets a little bit out of hand when you want to mitigate whatever you can. This way, when you want some part of the functionality but not all of it, you modulize it into smaller packages with independent features that may have some dependencies on each other. 

But the core package is still usable as a highly lean project. That’s why I like that approach: it’s the correct way to work. You usually see this mostly in monolith applications. Now with the microservices, many new orgs tend to go full out on microservices because it’s the thing to do, but you can always start with one bulky server, and when it starts getting large, you can make it independent based on the domain of every part of it. 

Let’s discuss dependency management in design. When you call a function, you’re dependent on another function! Unfortunately, developers forget the dependency versus isolation concepts.

How do you manage dependencies? What steps should we avoid? 

🦁In Node.js and JS, it’s a controversial topic but let me take a step back to explain why I think that. When I was doing PHP, Symfony was one of the frameworks I liked the most. 

Symfony is very object-oriented, and they probably took a lot of ideas from Spring and the Java world. So it’s heavy on dependency injection and dependency injection containers. So you do everything through the dependency injection container. 

And both are excellent ideas, but when I started to do more Node.js and more complex JavaScript in general, I realized that dependency injection is an outstanding principle. But you don’t need a dependency injection container in JavaScript because it’s so easy to mock the imports. Usually, it’s more convenient to mock the imports instead of building an entire dependency injection container and then managing all the configuration in a testing environment rather than a production environment. 

When you have to do external calls, for instance, to a database or even another function that might be part of another library: you need to consider that as a dependency. 

My guiding principle for discovering and understanding dependencies is unit testing. When I realize that testing a particular thing doesn’t make sense and needs to be a mock in my test, I know this is not part of my code; this is a dependency of the particular piece of code I want to test. 

I generally do the unit tests after I write a sketch of code first. And then I start writing the test and refactor: Not a proper TDD.

But with this process, when I start writing the test and refactoring, I need to extrapolate some piece of functionality, and it needs to become a dependency So I can mock it. And my code becomes more flexible, and I can test things more granularly.

It’s an iterative process where you’ll probably get it a little bit right at the beginning if you have experience, but you’ll never get it 100% on the first shot. So this is why I call it an iterative process.  

It’s an excellent metric to test your functionality and see what it depends on because, if you have a logic that doesn’t eventually persist, let’s take the example of a DB as a dependency. What if you can’t always mock the database for all the functionality you try to get? For example, if you have many joins or some functionality requiring a proper API to a database.

What’s your take on it? 

🦁Unit testing is helpful up to a certain extent. It assures that a particular function is correct, i.e., the code you wrote in a function you’re trying to test. But, when you mock that function, you assume that the mock matches the “real thing.” Unfortunately, that is only sometimes the case. Either because it’s very complex to do these tests correctly or because the behavior might change since you’re dealing with an external system where you can only guess what’s happening. Plus, things might change over time. 

I suggest not limiting yourself to unit testing only; they are a good starting point but consider integration and end-to-end tests when you get closer to production. Since you might start seeing behaviors you didn’t anticipate, for example, you can have two pieces of functionalities that are unit-tested correctly. Still, putting them together might expose different behaviors you didn’t anticipate, and they might break in different ways. When you start to create integration and end-to-end tests, it’s more likely that you will surface one of these problems and realize that there is something else that you need to change in your code. 
Integration testing covers many issues you could find when testing manually or later in production. Also, integration testing has different meanings for different teams. For some teams, integration testing can be full API testing, including the database and maybe excluding the other services in the cluster. Others use integration testing as function calls with their dependencies and perhaps a little persistence. It’s about the interpretation of each team. 

We at Sprkl are less keen on API testing. Instead, we’re more interested in testing the dependencies played together, as you previously mentioned: you unit test one component and unit test another successfully, but when you connect both, they sometimes play differently. 

There are lots of jokes, memes, and gifs around the web about unit testing: You open one window, and it works, and when you open the second window, it works, but when you open both together – they smash into each other. I found it on Twitter:

2 unit tests. 0 integration tests

Okay, this is where we will stop now! We have one more part – a short one after this one.

We just launched Sprkl for CI try it for FREE!