Hello everyone, and welcome to the Sprkl tips & tools series. In our series, we talk to prominent developers to explore topics around developer productivity and software complexity.
This time Daniel Beck, a lead software engineer, designer & UX specialist in product development, and consultant, gave us insightful tips on enhancing the productivity of software development teams.
Software development teams are the most crucial for business performance and profitability. Without them, there is no product! Enhancing developer productivity to maximize their potential is critical for leaders in modern software orgs.
In practice, while the details differ across orgs, this generally boils down to one of three general situations:
In an early startup: you probably shouldn’t try setting up too many processes or code architecture; any predictions you make about what you’ll need before you find your product market fit is likely wrong. The risk here is getting too far ahead of yourself and slowing yourself down with too much premature process or technical overhead.
In a large organization: the main challenge is visibility and trust: now you’ve got multiple departments – design, product, engineering, program management, etc. – each with multiple layers of management, all trying to work on the same product. That structure is necessary to organize the large numbers of people involved, but it can lead to a disconnect between decision-makers and actual reality. The best way to mitigate that risk is to, as much as possible, take that decision-making away from the high-level execs who are disconnected from the on-the-ground work.
That middle zone is the challenging one: you do have to throw out a lot of what used to work, which can be very painful for the people who used to work that way. And even worse, at the same time that you need everyone to align on your new processes, you’re also scaling your company and adding lots of new people with new opinions on what those processes should be. And you’re probably adding new management layers and more specialized roles that will influence the shape of things; a bad hire or two in this area can push you off the rails.
The middle zone, in my opinion, is the most interesting phase: it’s where you get the opportunity to make big changes and build for the future. You just keep trying things at a startup until one works; at an established organization, the most interesting decisions have already been made. That middle zone can be a wild ride, but it has the highest opportunity for individuals to have a significant impact in the long term.
There are different kinds of complexity; some of them are unavoidable (algorithmic complexity), some of them are self-inflicted (tech debt or conflicting code styles), some are organizational (business rules evolve faster than code does), some of them are tradeoffs you make to reduce other kinds of complexity.
So the trick isn’t “let’s make our codebase less complex,” it’s identifying which complexity is harmful and which is necessary for the current stage of your company.
The architectural complexity of microservices or Kubernetes is a great example of tradeoff complexity: microservices, Kubernetes, Terraform, et al. are inherently much more complex than their traditional equivalents; that complexity is the price you pay for scalability. I see way too many small or midsize companies jump into complicated solutions too early or just because they think they need to, and end up accepting that architectural complexity before they’re feeling the pain points that would justify it.
I see these consultants offering, e.g., “Kubernetes for startups” packages, which is wrong. If you do not routinely need to spin up and configure new containers, you don’t actually need Kubernetes. That pool of Terraform-configured step functions could probably have been built more quickly and easily as a plain old API. Likewise, if you’re not working at FAANG scale, you probably don’t need FAANG-scale tooling to get your job done.
Git methodologies are a good one. I’m old enough that my first encounter with source control was CVS, in which developers would literally “check out” the file they were working on, blocking anyone else from touching that file until they checked it back in. (That’s one way to prevent merge conflicts!) Subversion improved on this; it had what could charitably be described as “branches,” but it wasn’t until git that a real branch-and-merge strategy was really viable; to work effectively before git, you had to do a lot of communication and keep your changes small and focused.
Engineers who grew up on git tend to forget that it was designed for a specific purpose – managing a large number of contributions from various external sources into the Linux kernel – which doesn’t necessarily match the way a lot of engineering teams work. (In a way, git itself is another architectural complexity tradeoff: you accept the fact that merges are hard in exchange for allowing independent teams to work simultaneously on the same code.)
GitFlow was a fairly elaborate process that evolved to minimize the pain of those merges. However, most engineering orgs aren’t the Linux kernel; they’re not made up of teams of external contributors who have no choice but to work independently and deal with the conflicts in the end. So it’s interesting to me that that trend is now starting to swing the other way, as more and more organizations realize that smaller, more focused commits merged constantly are just a better way to work (for lots of reasons, not just avoiding merge headaches, but that’s a key one).
I’m 100% a convert to trunk-based development after seeing how much it improved our process at my last organization; switching from long-lived branches to quickly-merged code and a simple feature flag system was one of those rare cases of an absolutely unambiguous improvement; the whole problem of code merges just evaporated overnight. Don’t get me wrong; I love git; you couldn’t pay me enough to go back to the bad old days of file locks, but long-lived branches are unnecessary and cause more problems than they solve. I wouldn’t be surprised if some future version of “source control” ends up looking a lot more like real-time collaborative editing than what we’re used to doing today.
Every organization I’ve worked with in the last couple of decades has described itself as using an Agile methodology. None of them mean the same thing by it – the only thing everyone agrees on about Agile is that everybody else is doing it wrong.
I favor as lightweight a process as possible; shorter planning cycles are better than longer ones, fewer routine ceremonies, and as few formalized roles as possible.
A single engineering team does not need a team lead, a product manager, an engineering manager, a technical program manager, a scrum master, a product owner, and more. After a while, it can look like one of those photos of six construction workers standing around waiting for the one guy with the shovel.
You need someone to gather consensus on what you’re building, and you need someone to be responsible for ensuring that that’s what got built. So use as few people as possible to make those happen, and spend your money on people with shovels instead.
Personally, I’m a big fan of test-driven development when I’m building something complicated, less because of the resulting code coverage and more because it’s just easier to think through whatever the problem is one edge case at a time instead of trying to reason about it all at once. Write a test for each case, write code until all the tests pass, and you’re done.
This doesn’t necessarily result in tests that will be particularly useful after the code has been written, though – honestly, integration and regression testing are better for that than unit tests anyway – but it helps guide me into writing more modular, encapsulated .code, which is the secret point of unit tests: code that’s inherently easily testable is also inherently more reliable, comprehensible, and less in need of actual test coverage.
I think the most important and challenging part of coding is not the delivery process, build process, testing, framework, or documentation – it’s in deciding what to deliver in the first place. If you can nail that part, if you can predict accurately enough what your users need, the rest pretty much just follows on its own.
Keeping your releases small and frequent makes this much easier because you don’t have to predict with perfect accuracy: you can try out a small thing, iterate on it if the users like it, or abandon it without too much-sunk cost if they don’t.
Small releases are tremendously challenging for many organizations, especially those not used to continuous delivery. There can be a lot of pressure to stuff too many ideas into every new feature. This leads to a vicious cycle. Overstuffed releases take longer to develop, which increases the pressure to stuff even more things into version n+1 because everyone knows it’s going to be a long time before you’ll get a chance to add anything in version n+2.
I spend much time talking management out of leaning too heavily on productivity metrics: they’re tempting traps because there are so many numbers you can measure and charts you can make and it all looks so much like science that you could be forgiven for forgetting that they don’t actually measure productivity.
We’ve all figured out that counting lines of code, or hours-on-keyboard are useless metrics, so far so good; now we just need to learn the same lesson about code coverage, PR size, TTO, TTM, story points per sprint, and rework percentage. Those things can be meaningful signals, but they’re not productivity measurements.
A long cycle time might indicate any number of different things: maybe the requirements were unclear or are changing, perhaps the team is having to refactor a lot of legacy code before they can progress, maybe the team got walloped by some other unplanned work, maybe there’s a lot of friction somewhere in your development process, maybe the thing the team is working on is just genuinely difficult, maybe there’s disagreement within the team about how to build the thing, maybe somebody’s slacking off and bottlenecking the rest of the team. The point is you just don’t know from the metric itself whether it even is a problem, let alone what to do about it.
(I’d much rather have a team with a slow cycle time because they went back to the product team to clarify the requirements than the team plowed ahead and built the wrong thing very quickly).
Remember that engineers like solving complicated problems. They like writing code and building things. That’s why they’re in this line of work! So give an engineer a meaty problem, the scope and authority to solve that problem, and a development environment that isn’t going to get in their way, and they’re going to be productive every time.
My main goal as an engineering manager is to get out of my team’s way and to do my best to keep the rest of the organization out of their way too. Ideally, dev teams should get feedback at every phase and from the users (either directly or via observed behavior in logs, etc.) rather than internal decision-makers.
“Few layers of indirection between the engineers and the end users as possible” is the most important; the worst case is orgs where the end-user talks to a salesperson who talks to the product team, who talks to a designer who throws a finished workflow over the wall to the engineers. What gets built in those situations has almost nothing to do with what the user wants.
So many companies go through their scrum training with everyone sagely nodding their heads at the idea of including the Product Owner role, getting a real user involved in the planning process — but almost nobody ends up doing it. (Or they miss the point altogether and assign the title to a product manager or, even worse, a program manager.)
It’s super valuable; what developers and designers produce is much better when they really understand how the end user will take advantage of it, compared to “we’re building this because the quarterly plan says we should build it.” (At my last org, we were lucky enough that it was mostly managed service; the hands-on-keyboard users of our product were mostly in-house client partners working on behalf of our enterprise customers, so it was relatively easy to talk to them and get a direct line on their needs).
“I don’t know why we’re building this feature.”
This one is huge. If a team is building something just because they were told to and not because they understand its value, they’re going to be undermotivated (and will likely build the wrong thing, to boot.) The best solution to this is to have as few people as possible between the engineers and the actual users, and to empower your engineering teams to push back on ideas that don’t make sense to them.
“I spent all day in meetings,” “interrupted by questions” “responding to pagerduty tickets.”
This one’s pretty well-known; software engineers need uninterrupted focus time. Make sure they get it. Encourage them to block out segments of the day in their calendars, reduce the number of routine scrum ceremonies and planning sessions you subject them to, and if someone keeps peppering your engineers with questions or one-off requests, empower your engineers to say “no” (and find some less-interruptive way to get that person what they need).
“I spent all day tracking down [a bug / the right config / a particular error message / a stray semicolon].”
Everyone gets hit by the occasional head-scratcher of a bug that takes way longer to sort out than it theoretically should have, and to a point, that’s fine. But if it’s happening routinely, that’s another signal that needs looking at: maybe your code isn’t sending out accurate error messages, making bugfixes take longer than they should; maybe your engineers are working in an environment or context they’re not familiar enough with or that is too bleeding-edge; maybe they’re working inefficiently. (This last one is, ironically, especially common in orgs that put a lot of emphasis on fast delivery: engineers under pressure to churn out code don’t feel empowered to take the time to figure out, say, why their sourcemaps stopped working three months ago, so instead spend time trying to debug compiled production code. You need to give your team enough breathing room to streamline their own processes.)
I’m Daniel; good to meet you — I’ve been working on the web since the very beginning and apparently have formed many strong opinions along the way. The browser used at my first job was NCSA Mosaic. (We were very excited when Netscape came along and introduced innovations such as a background color that wasn’t gray.) It was pretty easy to get into the industry back then; you just had to learn all five HTML tags and were all set; from there, it was just a matter of keeping up with changes as new ideas get introduced and old ones fell out of favor. I even tried the solopreneur thing for a while, during which I learned I really need someone to set external deadlines for me if I’m going to get anything done.
Check out my website.
Sprkl is a Personal Observability platform that provides personalized feedback on code changes while coding in the IDE. We help developers ship correct and efficient code while spending less time on debugging and frustrating rework. Powered by OpenTelemetry, Sprkl instruments every code change and analyzes it upon execution.
Try sprkl for free
Share
Enjoy your reading 19 Min Read
We promise you’ll only get notified
when new content is out
Font 1
Font 1
Font 1
Font 1
Font 1
Font 1
Font - code
Font - code
