To give some context to this post, we've recently inherited a number of projects built by other developers. In each case, we've been asked to finish off work that was incomplete, or to take over because of a relationship breakdown. In every one of those projects cases, there is one constant: a lack of an automated test suite.
In business, relationships usually break down because of a mismatch in expectation versus reality, often compounded by a lack of communication. Testing, communication and expectation are inextricably linked to one another, and over the next few minutes, I'll explain why.
What are tests in a software project?
Testing traditionally was done by putting a person (or people) in front of your product with a documented series of tests. A manual test suite. You'd tick each one off as you went, or mark failures with what went wrong and hand back to the development team to resolve issues.
When the changes were made, you'd do it all over again. You can't just test the one failing test, because the changes might break something elsewhere.
All this testing adds time to a project, and time is money. So if you have one company bidding for some work and they don't do formal testing, they appear to come in a lot cheaper than the one that does.
Why write tests for software?
The benefit of tests in a software project are clear: they reduce the risk of breakage. Breakage happens both in the moment - whilst developing - and later, when adding or changing functionality in the future. The latter is called a regression, and the software world is rife with regressions. Consider how many times you've been involved in a project that involves software (including your website) where you've asked for a change and it's broken something seemingly unrelated. That's a regression.
It also makes upgrades easier. There isn't a piece of software in the world that doesn't use third party libraries or frameworks for at least part of it, and updating those can have an impact on the stability of the system. Having a well defined test suite makes it easier to upgrade that over time, without risking those breakages hitting your users.
The value of having a good coverage of tests won't necessarily become apparent right at the start of a project, at least not to a non-developer. If that's you, you'll realise the value down the road; weeks, perhaps months after the system has been built and been running. You'll ask for a change, and it will just happen, and won't break something else. Or if you find a bug and you report it, a test can be written that proves it (bugs need to be replicated by the developer to be fixable), and then once fixed, it can't happen again. Without tests, that bug can rear its head many times into the future.
So the value proposition for having tests is stability, and that means a robust part of your business operating arsenal. With an unreliable system, people lose faith in the system and find workarounds, and those then take additional time and cost more money. If they trust the system, their requests instead create further benefit to themselves and the business as a whole.
Why your software development project should have automated tests
You can probably see why automating your tests is useful; it saves a huge amount of time manually testing each and every line in your test suite. A test suite of several hundred tests can take a matter of a few minutes to run, and you can run all or part of it whilst developing to make sure those tests don't get anywhere near a real user.
So taking the above statement that testing costs time and money, we can see that automating those tests now means a lot less time is spent actually doing the testing. But someone still has to write the tests. That's developer time, and so 1 hour of development time might add another half an hour in writing tests.
Writing automated tests allows developers to test the really deep parts of the system - the bits where numbers are crunched - to make sure they're rock solid. It also provides documentation for new team members, or heaven forbid, completely new teams. Handover with tests is clean, and it means that the incoming developers can take the project on knowing they have a working base. That means little to no disruption. On the other hand, as we've found with all the projects we've taken on without a test suite, those systems come with their own hidden mysteries. Things that come out of the closet as you start to work on them, and those cost more time and money than writing the tests in the first place.
Writing automated tests also makes for cleaner code. They allow for refactoring (reworking) of code to be cleaner and more maintainable, and also give clarity of purpose up front, meaning a developer will often write less, and better formed code. Again, that's a long term maintainability win, all of which means more time and money saved.
What happens if your software project doesn't have an automated test suite?
Honestly, you're in trouble. Without wishing to put too fine a point on the stick I'm prodding you with, a lack of a test suite screams future problems even if it doesn't show current ones. We've just taken on a project that has cost several tens of thousands of pounds to build over a number of months, and we were told that there were a never ending list of issues that couldn't seem to get fixed.
So a direct outcome of a lack of tests is a lack of trust, and that means a catastrophic failure in the relationship between the business and its development team. That could as well have been an in-house team - the outcome is the same.
More than that, any new development team that takes on the project will be questioning whether they want to weather the risk of the project, and whether the risk should be weathered by the client too. It's a commercial decision at the end of the day, but one both parties will need to make, and not one to be taken lightly.
One alternative is a complete rewrite of the application from the ground up. That's not something I would usually recommend, and there are plenty of articles around the web pointing to it being absolutely the wrong decision. However, if you're dealing with a brand new service, and it's a mess from the start, sometimes you can make some big changes by rewriting. However, in that situation, it's not a rebuild of what is, but a build of something completely new - it's still not a rewrite really, but a completely new product.
The other is to spend time and money investing in the project properly, building out the tests, documentation and cleaning up bad code or architecture. If all the code is bad? Perhaps refer to the first alternative. It's still a colossal waste of time and money to have built something without tests in the first place.
Earlier in the article, I touched on upgrades. With an automated test suite you can test an upgrade of one part of or all of a system's dependencies by performing the upgrade and letting a server run the tests while you continue on with your work. When it's done, you'll know what breaks, what needs fixing, and can form a plan as to how to do that and keep the application running the latest versions of everything it depends on.
What VALUE do tests bring to your project, and to your business?
Testing your systems is critical. You can't run a successful long term software project without them. If you have already started without them, make sure you add them in.
If you haven't started your project yet, make sure it's at the top of the list of questions you ask your prospective developers, along with what their average code coverage is. Code coverage is the amount of the code that is covered by tests, which gives a good indication of how well tested it is.
At Initforthe, we aim for 95% which is very high in the industry. We achieve 85%+ on all our projects and regularly hit 90%+. We write tests for every single bug that a user comes up against and prevent any regressions in the future. And the remainder of the tests cover the vast majority of the code we write to make sure it always works, can be upgraded easily, and you'll have minimal complaints or support requests from your users.
Let's also be very clear, and relate this directly to the image at the top of this post: 346 people died as a direct result of a lack of software testing when two Boeing 737 Max aircraft crashed. Further, the aircraft has been grounded ever since March 2019 causing untold billions of dollars in losses for both Boeing and its customers.
So in conclusion, it's less about whether there is value in testing, and more about what can happen if you don't have tests in your software project. Caveat emptor.
