Ways to Improve Your Development Process
When it comes to software development there are different levels when it comes to software reliability. The highest echelons would include things like formal proofs while the lowest levels being fire and pray. Note that while I say that the formal proofs would be the high end, I'm not saying everyone should shoot for that. It's time consuming and usually not worth the effort depending on what you're building. However, that doesn't mean that you can't improve your standing on that axis. Let's look at easy ways to do that.
Unit Tests #
For the sake of argument lets assume that you have no testing infrastructure in place at all. Perhaps that's because you don't see the point OR because you simply are new to the concept. As such we'll start with the basic tool: Unit testing. Unit testing is a level of testing where we're trying to make sure individual bits of our code works the way we expect it to. The purpose is just to validate those small bits and not the application as a whole.
What is the point of this? I ran my code and it worked on my machine. Well, the issue is generally as you build software, you tend to go back and modify previously written code. The point of the unit tests is to make sure that both your original pass of the code works and secondly that those changes later didn't break something unexpectedly.
Cool, sounds good so is this a universal good thing to do? Well, there are several benefits that come with unit testing but also some downsides:
Pros
There is some evidence that it actually helps with productivity.
It has almost uniformly been shown to reduce defects in software by a similar amount, 20% to 30%.
Code tends to be more reusable because in order to make unit testing possible, code needs to be modular.
Cons
There are issues maintaining the unit tests over the lifetime of a project.
So while some research has shown the benefits to productivity, many show an increase of 20% to 30% development time. More development time means that the cost to develop the project goes up initially but long-term maintenance of the software takes less resources due to the reduction in defects. So, spend money now or a lot more later.
If the reduction of issues at 3 AM isn't an issue for you and you don't want to spend the time or resources, awesome. The rest of this post and the next couple will be of no use to you. Feel free to jump now. For those that want to find out HOW to do unit tests, a decent starter is a book like Test-Driven Development by Example or if you want a quick overview there are a number of blog posts out there like this one. Note that you don't have to do test driven development. You can always write tests after writing the functionality. Like the above there are pros and cons to the practice. On top of that most research shows very little difference between writing the tests up front vs after the fact. I find it depends on how you build software as to which is better for any individual situation. With that in mind we'll just assume that you have tests and not worry about how you got there when looking at the next step.
Continuous Testing #
So let's assume that you have tests. Awesome but when was the last time you ran them? If you're running them manually then probably not nearly as often as you need to be. The longer you go between writing them and running them, the less useful they are. You write a bunch of code and when you run them you have to potentially go through days worth of code to find the bug that you introduced. Doing this leads people to drop unit testing. What's the way to fix this? Continuous testing.
What is continuous testing? It's testing that occurs automatically after some trigger happens. For many tools, it happens on a build server during the continuous integration build process. However, there's nothing stopping you from using some of them on your own system. Much of the time IDEs will allow you to automatically run tests when you build. Check a box and you're done. Sometimes there are plugins like Continuous Tests to fill the gap. In either instance they allow you to run the tests on your own system at specific times in your development process. And assuming you wrote your tests well enough it shouldn't impact anything. Run in the background and just go about your business.
Should you use the build process integrations or the ones in your IDE? Both. Make it part of your development process so you get feedback while you're still working on the item AND make it part of your final build process. Never push anything to production without that final build testing. But for starting out, let's look at setting it up on your own system. What are the pros and cons?
Pros
Reduces overall wasted time when developing by about 10%.
Developers tend to be more likely to finish the thing that they're working on in less time when using continuous testing tools. Note that the linked study was about students finishing a project in a given amount of time. It's assumed that if given more time the people not using the tool would have also finished the project.
Cons
While continuous testing is only a small portion of it, continuous integration's impact is mixed but generally considered positive.
Waiting for tests to complete takes up about 10% of development time when they're used. But that's if you wait for them to complete and not just continue with developing code. It can also be mitigated by having the system build the software and test in the background, only notifying you when there are issues.
Setting this up is different for each language but usually it's a cheap way to improve your testing. Much of the time being a single check box and you're done. And the bonus is that about 90% of the developers that try it recommend using it. At least based on the literature. Adding it to your continuous integration system as well, which you should also be doing, adds another layer to help you and is usually equally simple to set up.
Anyway, we now have tests and continuously are running those tests. So based on the literature, that means we are getting the improvements in defect reduction with the tests and mitigating some, but not all, of the speed issues with continuous testing. So, what's the next step?
Metric Gathering #
I can hear the collective groan from here. Metrics have a tendency to be used as a weapon against developers, testers, and the like because generally people don't understand them. Metrics are nothing more than indicators that can be used to help you. The key is to use metrics that are shown to have a correlation with helping you identify areas of your code base that you want to spend some extra effort testing.
So what are those metrics? Well to be honest you can get by with a short list of items:
Code cohesion. Note that this hasn't been tested as much as I would like but cohesion seems to have a high correlation with testability of software. Higher cohesion means higher testability. Higher testability means unit testing that code will be a whole lot easier.
Cyclomatic Complexity and Lines of Code. While these have a bad rap for being used as cudgels by management, you can actually use them as decent metrics of where you should focus your time. The higher those numbers, the more you need to test and probably refactor the code.
Coupling. Although this one is not that great of a metric in comparison to the ones above, like the above metrics it can be used to determine areas of your code that can be simplified.
You may be surprised that I didn't mention this one but Code coverage may not be that great of a metric. Why? It turns out that while branch coverage is nice and all, true coverage would need to include every data point that's possible. So, unless you're going to wait 20 years to test every integer value possible for your method, you're probably not going to have true 100% coverage of your code. But generally aiming for 80% code coverage for branches is what you should be aiming for. Less than that and you're probably not testing enough. Over that and you probably won't be finding too many new bugs. So it is still a good metric to have in your arsenal but it's not nearly as important as code cohesion and cyclomatic complexity for determining code quality and your ability to test your code.
Anyway, with the addition of these metrics we know where to focus our time thus reducing development time further. With that we're at 30% reduction in defects and about the same development speed as before. We know where issues are most likely to occur in our code. And it can all be automated costing us nothing.
So can we go beyond that? Heck yeah and I'll talk about that in upcoming posts. But going forward I'll assume that the above is our base line. In future posts I'll talk about how we can super charge this to allow us to build better software.
Items in the Series #
Sign up for More Posts
I have no idea why you'd put yourself through that but if you'd like more posts from me then how about using my RSS feed? Take a look.