There should be three sources of truth in any software system:
1. The requirements doc
2. The tests
3. The source tree
Each one is an alternate description (varying in accuracy) of how the software you are building should work. We are not even talking about QA here, this is all within the realm of a software developer.
One major advantage of using AngularJS for web app development is how easy it is to set up a testing strategy. I'm not posting to talk about how to use angular-mocks.js to unit test with jasmine. Or how to use angular-scenario to do integration tests either. Let's talk about total testing strategy.
Developers often talk about the importance of unit testing in your project, and how after a certain point your project will not scale without them, even if you are working with a masterpiece of a source tree. In an age where modern web applications are servicing billions of requests from all over the world, any serious work you're doing needs to be scalable.
Web tech has brought us into the information age, it's time to take your projects to the next level and handle more information. You are going to need test automation.
Poor testing --> Poor scaling --> Missing the boat
But there is actually more to the story than unit testing, and it's not just about making the decision to have unit tests or not.
- What are the testable "units" in your source tree? It's easy to write code that does not have easily isolated units if you're not trying to keep it that way.
- Which "units" are most critical? In a large enough project, and a restricting timeline, you might not be able to test everything, so prioritizing is important.
- Are your unit tests up to date? Tests should mirror your project requirements, which are usually constantly changing. You need to make sure these tests capture these changes to be sure that your actual source tree captures these changes.
- What percentage of your source tree is tested in unit tests? The larger the better, aim for at least 80%. Unit tests are the fastest tests to run and will prevent many simple (but still show stopper) bugs from popping up in production. The fastest run tests are the most often run tests.
Beyond unit testing, you need to think about end-to-end (e2e) testing. If you have been successfully using unit tests, you have probably organized your code into testable units. But when it comes down to showtime all of those units need to work together. Integration gets complicated. So don't blindly trust the integration code.
e2e testing lets us automate the actual system. Send live input and track what happens. Assert that what was supposed to happen actually happened. This type of testing is almost as even better than having a robot automatically run your app, use it, record the results and give them back to you. It takes longer to run. These tests are harder to write, and even harder to maintain. You are unlikely to even have half of your use cases covered here so you need to carefully think about what kind of tests to write, and how to maintain them.
- How often is this use case repeated by your users? (Example: refreshing inbox vs. changing signature settings in a email app)
- Is the amount of time this test takes to run reasonable? Adding an additional 2-3 minute delay to a testing cycle is bad, developers will stop running them.
- How much time is spent updating this test? The feature you are testing may be immature and constantly changing, forcing you to re-write this complex test all the time, you be the judge if this is actually adding value to your project
This is only scratching the surface, as many books have been written on testing alone, and many more probably need to be written. Once you understand your requirements doc, your tests (unit + e2e) and your source tree, examine all three of these pieces to re-affirm in your mind that your system is doing what it should. (for this release!) Then build it, run the tests and send it to QA. Repeat this process over your release schedule as features are completed.
What do you do to manage your testing strategy? How do you balance unit vs integration tests? What indicators do you use to determine if you need to change your process?