Nov 7, 2017

A full system test on the other hand covers all business use cases end-to-end.

Unit and integration tests are nice to get a fast feedback but if your application is solely tested by them you’ll get other errors on production (like environment-specific configuration, problems with the server environments or external interfaces, etc.). 

The application should run in the same way during the system test as it would in production. 

That means the system test environment is ideally the same — including server configuration, databases, network, etc. — with the only difference that it’s not the real production server.

During the system tests all external systems are mocked away and ensure that the application communicates with them in a correct way.

An easy way to achieve an identical system for all environments is to use containers. The container which incorporates the application, the application server and the operating system is built once from the CI server and used for all environments. Using this approach highly increases the stability as exactly the same artifacts run in production which have been tested extensively.