#032 - The role of a BA is testing
Welcome to Issue #032 of The Forcing Function - your guide to delivering the right outcomes for your projects and your users.
✍️ Insights: If you don’t test your requirements, they will test the gaps your in understanding of them.
🤔 Made me think: Where fact is stranger than fiction with DNA testing.
👨💻 Worth checking out: Testing a bulletproof vest, 100 years ago.
✍️ Insights
If you don’t test your requirements, they will test the gaps in your understanding of them.
It was dead with no signs of life.
Even worse, it meant having to survive a freezing cold shower that morning. A friend thought that it was a feature as he swears by cold showers to reinvigorate yourself in the morning. But there’s a difference willingly wanting to do that versus having it forced upon you with no choice.
If only the engineer has properly tested what he had done the previous day.
In upgrading my electricity meter to a smart meter (which helps see your energy use in real-time), that engineer had installed a smart meter than could only handle the single standard circuit that all homes had. That’s what he tested before he left and that circuit was working. However, my apartment also has an off-peak circuit for the storage water heater and that was now unconnected.
If he had tested that circuit, he’d have quickly realised that he had installed the wrong type of smart meter. And I would’ve avoided not having any hot water. I ended up having to get the energy company to revert me back to a “dumb” meter that did work for both circuits.
It reminded me of being on-site for some weekend disaster recovery testing.
Being far more naive and far less experienced back then, I thought that this was overkill for something that was unlikely to happen. That was until I saw the monitors flicker off. To test the instant failover to the the uninterruptible power supply (UPS) systems, we cut the mains power to check that everything was still running on UPS so users would have 15 minutes to save their work.
At first, I thought that the test had failed because the monitors should’ve stayed on. But, their standby lights were still blinking away so they were powered by the UPS. Imagine our embarrassment and more than a little face-palming, when we worked out what had gone wrong.
None of the desktop machines were plugged into the UPS circuit.
They were on a regular circuit and so when the mains power was cut they immediately shut down, collapsing instantly like puppets with their strings cut. So, anything that the user would’ve been working on would’ve been lost. Exactly what disaster recovery was meant to prevent.
It was a vivid lesson for me about the importance of testing because until you do, you simply can’t have confidence that you know what will happen.
Testing humbles you as a business analyst
As business analysts (BAs) we pride ourselves on both our understanding of the issue that needs to be resolved in our project as well as our attention to detail.
We do the stakeholder meetings, we analyse what requirements are needed from the myriad of scenarios that we come up with, and document everything clearly with visuals and tables for our development team. The more experienced BAs also consider unhappy paths. Those alternate branches where users get lost or where they choose to not to follow the path you’ve carefully laid out for them.
Yet, despite our best efforts, that’s never quite enough to get complete requirements coverage. We remain blind to gaps in our logic. Some wider and more glaring than others.
When I am lucky to work with a professional tester, I’m grateful for their expertise. For, they will usually find that one test permutation that has me face-palming because it’s so bloody obvious when it’s highlighted. That’s on top of the other edge cases I didn’t think about, the dead-ends in my user journeys, and the non-sequiturs in what the user has to do (or doesn’t do) in my process maps.
It’s not just that a tester brings a fresh pair of eyes to what’s required. It’s because their mindset and experience is far more attuned to deeply understanding the different test scenarios, the different test variables, and then all the test permutations that could arise. Not just the few obvious ones but all of them as any one of them could be catastrophically bad for the solution.
More often than not, clients tend to believe that they can adequately do testing themselves and “save” money on the project. They are only partially right, though. They do need to do the user acceptance testing (UAT) to formally sign-off changes for release into production.
Doing the end-to-end testing that precedes the UAT is entirely on a different level.
Unless you’re well versed in doing this day in and day out, the chances are that you’ll miss something critical when you do go live. At which point, you’re firmly on the back foot, desperately trying to fix on the fly. Fire-fighting your way to get your solution working and stable without having to change your whole design.
All the while, behind you, your pile of technical debt exponentially grows ever larger which you’ll clear down soon, right?
Testing is expressing the same requirement in a different way
Think of your favourite song.
Perhaps it’s about unrequited love or your wedding song or something that your Mum sang to you when you were young.
Now, think of a poem about the same topic.
The concept is the same but the way it comes across musically is different to how it’s said in words. Neither is better or worse than the other - they are simply two complementary sides (of many) with their own nuances and lend themselves to particular interpretations. Depending on the context, a song might communicate something that a poem can’t - and vice versa.
That analogous to the relationship between a requirement that we write as BAs and the testing artefacts that a tester writes.
As the BA, we write requirements in a specific way with specific meaning and intent. The developer may interpret those same requirements in a slightly yet substantively different way. When unchallenged, the result is that the requirement doesn’t do what the business expects.
Hence, the age-old argument about whether it’s a defect to be fixed or it’s as per design so the business needs to raise a change request.
By ensuring that there is rigorous and robust testing, the tester enforces discipline on all stakeholders but from a different perspective. Is this what the BA meant by the requirement? Is that what the developer has built? Is the acceptance criteria what the business stakeholder is expecting to test against?
But, I’ve seen BAs rail against testers and become defensive.
It doesn’t have to be this way. Testers are not auditors of the work that a BA does. They aren’t assessing a BA’s professionalism, their competence, or their judgement.
Testers are partners to BAs just as BAs are partners to testers.
We help testers understand our thinking, our reasoning, and our approach so that they have the knowledge to do their job well. In return, they are our guardrails and give us well-founded confidence that we haven’t missed anything significant.
It’s only by working together that we can be a powerful combination that delivers the the right requirements in the right way to the right users.
🤔 Made me think
Where fact is stranger than fiction with test results
It’s not just that you test thoroughly but that you’re testing the right thing in the right way before you act on the results.
An obvious statement you’d think.
In 2017, Louis Côté sent in a sample from of his girlfriend's chihuahua, Snoopy, to a DNA testing company along with a sample from his inner cheek. The results were wrong as they showed that both Snoopy and him had 20 per cent Native American ancestry. Not only that, this was the start of unravelling a tax scam.
Check out the full story (also on archive.org) by the Canadian Broadcasting Corporation.
🧑💻Worth checking out
🔗 Testing a bulletproof vest, 100 years ago | The Atlantic.
Thankfully, doing live testing isn’t as life or death as this photo from 1922 shows where .38 and .45 calibre bullets were shot at close range.
What hasn’t changed is the gravitas, the trust, and the validation you earn when you prove what you’re doing with a difficult test.
You can see more fascinating photos from 100+ years ago in this photo essay by The Atlantic.
🖖Until next Thursday ...
If you enjoyed this newsletter, let me know with the ♥️ button or add your thoughts and questions in the comments. I read every message.
And, if your friends or colleagues might like this newsletter, do consider forwarding it to them.
For now, thank you so much for reading this week's issue of The Forcing Function and I hope that you have a great day.
PS: Thanks to P for reading drafts for me.