There ought to be a good set of FOSS unit tests for those who dare implement their own crypto. For instance, you let it hook in to your PRNG, and it'll tell you if the output is random-looking enough.
It wouldn't be a panacea for bad crypto, and it does create a risk of people thinking "oh, it passed all of the tests, it must be secure," while still implementing it overall incorrectly. But I still think it would mitigate these "foolish/easy" errors and allows devs to focus on proper overall implementation.
The problem is that it's really hard to write unit tests to determine if you've gotten the crypto right. For one, the unit tests would have to have a standardized interface to your cryptographic protocol. But if you're using a standard cryptographic protocol, why would you be rolling your own implementation instead of using an existing, well tested and vetted one?
Second, it's quite difficult, in an automated way, to look for the kinds of security vulnerabilities you would need to audit. Timing attacks may show up in different places for different software. Poorly seeded entropy is quite hard to detect; Debian systems were generating keys with only about 16 bits of entropy for years, which is enough that it's hard to detect in an automated way unless you are really looking for that specific breakage but is a devastating security hole. You may have bugs that are not crypto related at all but simple buffer overflows or the like. And so on.
You could probably automate some of it. But on the whole, you need many good, experienced people regularly auditing the code. And it's a lot easier to get that by using an existing, widely used and widely audited library than by rolling your own.
However these are just statistical tests; they tell you nothing about how much actual entropy you're actually getting (cf. Debian). That said, it probably would have caught the Cryptocat bias.
Not only the number of bits in the seed, but also their unpredictability.
There are many things that can go wrong. Some examples:
- the seed is timer-based and has far less entropy (information) than its bit size
- the seed is OK, but the generator is somehow weak (e.g., only made with linear components, or throws away parts of the seed, or outputs too much of its internal state)
- Bugs, bugs everywhere (cute example: an RC4-based generator that uses the XOR trick to swap elements)
No standard statistical test would catch those (it would catch the latter, actually), even though the security of the generator would be gravely diminished.
You can't unit test crypto. There are tests you can run to check for PRNGs being obviously broken, but that doesn't help you: the point of the article is that crypto has to be 100% right or it's 100% broken. It doesn't matter much to an attacker whether a flaw is "foolish/easy" or not: he can exploit you either way.
Also, a lot of the time the vulnerabilities are not in primitives like PRNGs, but in how application programmers combine the primitives in their application, and there's no way to unit test the security of that. The way to mitigate this is by developing and using high level library interfaces that make it very difficult to screw up.
There exist test suites which look for some typical problems. The http://en.wikipedia.org/wiki/Diehard_tests were well-known long ago when I last thought seriously about randomness, and a few minutes with wikipedia told me about http://www.phy.duke.edu/~rgb/General/dieharder.php too. But your dream of software that can truly automatically "tell you if the output is random-looking enough [for crypto]" is deep unrealistic, akin to the deep confusedness sometimes exhibited by business majors dreaming up copy protection features. Merely defining the notion of "random enough for crypto" is a cornucopia of interesting technical problems, and the useful definitions naturally tend to involve the boundaries of complexity classes and thus a bunch of open conjectures, and it gets worse from there. (Things like: Even if you could prove the conjecture that this complexity class is good enough, how are you going to test for nonmembership in it faster than your adversary can break it? Most people are not very interested in crypto which can be broken by an adversary in a month of CPU time, and most people are not interested in tests that take a month of CPU time to run.)
Unfortunately, it's really really difficult to test for randomness. We can test for particular patterns, but nowhere near what a determined human with access to the source code can find.
At the moment, the only "correct" test I know of is hilariously impractical: enumerate all possible programs up to a reasonable length, run those programs against output from the random number generator for a few minutes each, and see if they're better than chance at recovering the seed you used.
The salt is different for EVERY secret. When you hash/store a new password, you randomly generate a salt for THAT password, and you store each salt & password together.
Because each salt is ~unique, the attacker must rebuild a separate dictionary for every single stored secret. That's functionally the same as having no dictionary, at all.
Don't abuse the text field in the submission form to add commentary to links. The text field is for starting discussions. If you're submitting a link, put it in the url field. If you want to add initial commentary on the link, write a blog post about it and submit that instead.
Not to mention users like me, who opened it in a background tab, never even saw the video playing. I wouldn't have known it was anything but a blurry image if I hadn't seen this comment.
It wouldn't be a panacea for bad crypto, and it does create a risk of people thinking "oh, it passed all of the tests, it must be secure," while still implementing it overall incorrectly. But I still think it would mitigate these "foolish/easy" errors and allows devs to focus on proper overall implementation.
Or does something like this already exist?