kinds of features:
- new app
- new code, new class/function(s)
- new functionality that didn’t exist before
- updated app
- maybe new code, updated old code – same classes/function(s)
- same functionality, just enhanced
- updated system
- system architecture – a system is updated (some component of the larger system)
- bigger than an app, difference here being one of scale and scope
- replaced system
- a large piece of the software ecosystem was replaced with something different
- not an update, this is replacement
- new system
- a large piece is added to the software ecosystem (one that didn’t exist before)
Part of my job is testing systems for quality and functionality after a change occurs. It’s actually harder to define change than you first might think. What was changed? How much? What all does that thing touch that might be affected? A system doesn’t have to be that complicated before those seemingly simple questions start becoming hard to answer.
Also – what is truth? What is ‘correct’? If everything is relative than how do you even know? If your car won’t start then sure, it’s easy to say it’s broken. But how do you test it’s color? It’s sound system quality?
Testing can be a lot of things. It can be writing reams of test cases to try to get into every nook & cranny (but usually if this is the route you’re going it’s more for the optics, that many test cases is never really useful IMHO). I try to think about it like this:
This is one of my favorite X-Men, Archangel. He began with more basic, feathered white wings but those were lost in a battle. New metallic wings were grafted onto his body.
So if I’m part of the mutant QA having a look at him to evaluate how his new wings turned out…what questions do I even begin to ask? Do they…work? Such an open-ended, vague question is almost useless. “Work” as defined by what? Compared to his old wings? Well sure but the new wings also have a list of new attributes and capabilities. He can fly faster, higher. His new wings have new powers his old ones lacked. Also – what about him? He just underwent a major operation to acquire techno-organic wings of an unknown metallic substance grafted right onto his spine. Can he still walk? Go to the bathroom? What about evaluating his quality of life? Those wings look cumbersome, also as a side effect his skin turned blue…should that be included in this analysis? It’s tangential but not necessarily trivial depending on the lens you’re looking through (to him it’s probably a big deal).
Personally – I think it helps to try to break it down by thinking in terms of ‘state’. The state something is in at any given moment is just the sum of it’s current attributes and functionality. What collection of attributes and functionality you select form a data set that describe a state. Archangel has several states you could measure/describe – his wings physical capabilities, his powerset, his flight capacity, his body’s functionality, his mental state. You could compare/contrast those to his previous states prior to the new metallic wings. What has changed? What has remained the same?