tests: Support tests that are expected to fail.
authorJustus Winter <justus@g10code.com>
Thu, 4 May 2017 13:12:49 +0000 (15:12 +0200)
committerJustus Winter <justus@g10code.com>
Thu, 4 May 2017 13:22:52 +0000 (15:22 +0200)
commitd6b46462f8c5c705ffb7cf8af03465a926aa11d3
tree3831110bbeac5ff8d53b0fa779fedfb66657cc52
parenteab0138e3179f247180a639a91570e5ee2c6ad0e
tests: Support tests that are expected to fail.

* tests/gpgscm/tests.scm (test-pool): Rework reporting.  Filter using
the computed test status instead of the return value.  Also print the
new categories 'failed expectedly' and 'passed unexpectedly'.
(test): If a test ends with a bang (!), it is expected to fail.  Adapt
status, status-string, and xml accordingly.
--

Allow tests to be marked as being expected to fail by appending a bang
(!) to the tests name.  If such a test fails, it will not be counted
as failure, but will still be prominently displayed in the report.  If
it succeeds unexpectedly, this is counted as a failure.

Fixes T3134.

GnuPG-bug-id: 3134
Signed-off-by: Justus Winter <justus@g10code.com>
tests/gpgscm/tests.scm