Improve software deployment process

Software product we are developing have recently grown to the extent that it takes a very long time to build all necessary packages and deploy them to our production infrastructure. Our quality assurance team needs to review all new features implemented in each milestone before we release them so we need very frequent deployments (sometimes several times a day). However, our programmers are currently building all software packages on their local machines and manually deploying them to the staging server over SSH. This costs the company significant amount of man-hours and other resources involved in this process. Is there a better way to improve our deployment process so that the programmers could invest their time in more productive manner? COMPETENCES (based on all 3 challenges): Results orientation Policy development and Planning Managing performance Innovation and reinforcing change
1 answer

Use continuous integration system

I recommend to use some continuous integration (CI) system, which includes the following best practises:

1. maintaining of a code repository
2. automating builds
3. making the build self-testing
4. everyone regularly commits changes
5. (optional) build after each commit
6. pre-production testing
7. everyone can see the latest build
8. deployment automation

Popular CI solutions are for example Jenkins or TeamCity. Both solutions are based on the best practises and provide complete build, test and deployment process automation. Typical CI server consists of several tasks which can be chained together. There are many different plugins available for most of CI systems, supporting all kind of tasks, which makes CI a very powerful tool of every project manager.

CI workflow after your programmer commit a new code could be for example:
1. Trigger quality measurement - automatic analyzing of code for potential errors and inspection of code quality violations (so called "linting").
2. Trigger building of all required packages and executables.
3. Trigger execution of unit tests and some of the simple integration tests.
4. Notify programmer about the result after the job is done. This can be also automated with IDE of a programmer, so he gets errors highlighted directly in the source code on his screen.

Furthermore every night is usually executed functional testing and possibly some of the more time-consuming integration tests. Typical CI configuration also contains tasks for automated deploying your builds to staging or production server.