Assure quality of a software product

We are developing an analytical software product where the accuracy of all our data that we generate needs to be assured. However we recently had an incident where we provided wrong data to one of our clients. After careful examination we found a regression that was introduced to our production stacks after a recent release. As a result we almost lost the client and it affected our reputation of a reliable data provider. How can we avoid such incidents in future and assure that the data we are publishing have a good quality?
Submitted by Peter Tulala on Sun, 12/04/2016 - 00:16
1 answer

Improve software development process

I suggest to introduce 3 new policies in your software development process to have a better control over the quality of your product:

1. Better branching model, pull requests and code reviews
Forbid your programmers to direct commiting of new code to development branch, but rather implement a better branching model. Popular choice is to use for example Vincent Driessen's branching model (http://nvie.com/posts/a-successful-git-branching-model/). Require that all changes to the codebase must be done in separate branch and merged to the trunk only after careful line-by-line code review by another programmer. This policy is easy to implement and errors found during code review process are less expensive to fix than errors found in production.
Useful tools: Github

2. Regression testing
Here I’d recommend to introduce 3 layers of testing your code: 1) Unit testing after each commit. This test is usually executed very fast and can detect the most serious bugs. 2) Integration testing to test components and modules of your product. You can choose between top-down or bottom-up strategies and possibly execute part of integration tests only once a day. 3) Functional testing to test your product as a black-box. Executing on daily basis to compare formal specification of your product with the version currently in trunk.
Also consider introducing test-driven development, i.e. programmers must write tests before starting to implement new feature.
Useful tools: Spock or JUnit for unit and integration tests, Selenium for functional tests of web application and some CI server for executing tests

3. Dedicated QA infrastructure
Although good test coverage usually detects most of the regressions, there might still occur errors that were not detected during automated testing. I recommend to create a dedicated QA environment to allow manual testing and prevent such problems. Modify your release management process by introducing a policy that every new feature in current milestone must be first manually tested on staging server before releasing it to production stack.
Useful tools: Amazon AWS

Submitted by Peter Tulala on Sun, 12/04/2016 - 12:51