Test Case Design
I just got burned by a test case that gave a false positive when it was executed. The verification step was to look for the existance of a secret string in the post-processed log files. I looked in the post-processed logs and and didn't find any evidence of the string thus the test passed.
But, there was a defect and not all log files were instrumented correctly so in a few cases the secret strings were not actually get stripped out of the post-processed logs.
A better test design would be to locate examples of each type of secret string in the raw log files. Then verify that each of these were stripped out by the post-processing. In other words, prove that there is something to be removed before believing that it was removed (and not just missing in the first place.)