Split off private_transform function in RSA.

This change extracts two, common parts of RSA_decrypt and RSA_sign into
a function called |private_transform|. It also allows this to be
overridden in a method, which is convenient for opaque keys that only
expose the raw RSA transform as it means that the padding code from
BoringSSL can be easily reimplemented.

One significant change here is that short RSA ciphertexts will no longer
be accepted. I think this is correct and OpenSSL has a comment about PGP
mistakenly stripping leading zeros. However, these is the possibility
that it could break something.

Change-Id: I258c5cbbf21314cc9b6e8d2a2b898fd9a440cd40
Reviewed-on: https://boringssl-review.googlesource.com/1554
Reviewed-by: Adam Langley <agl@google.com>
5 files changed