[Solved] Bad Data exception during RSA decryption

by


Among other problems you are assuming that the ciphertext can be converted to UTF-8 meaningfully, and that isn’t guaranteed.

If you want to encrypt text and transmit/store the encrypted contents as text you need to follow this pattern:

Encrypt(textIn => textOut):

  • Convert textIn to bytesIn via some encoding. UTF-8 is pretty good.
  • Encrypt bytesIn to bytesOut.
  • Apply a lossless encoding to bytesOut. Base64 is almost always the best choice for this, and is the best one that’s built in to .NET. (textOut = Convert.ToBase64String(bytesOut)).
  • Return textOut.

Decrypt(textIn => textOut):

  • Apply the inverse transform from Encrypt to turn textIn to bytesIn. So likely Convert.FromBase64String.
  • Decrypt bytesIn to bytesOut.
  • Apply a text encoding transform to turn bytesOut to textOut. There’s nothing that really says that this needs to be the same as the one used in Encrypt, but it probably makes sense to do so.

The problem with using UTF-8 (or ASCII or UCS-2, etc) for a textual representation of encrypted data is that the encrypted data can legitimately contain a byte whose value is 0x00 (or a control code, linebreak, etc). While .NET, for the most part, happily transports embedded null characters in a string it definitely can be a large source of confusion (and not all languages/functions handle it the same).

Other fun characters:

  • What does your language/function do for an embedded 0x7F (Delete)?
  • If you store the string in a single-line text field, what happens to any CR or LF characters? What does your program do on Windows if it gets CR without LF, or LF without CR?
  • A Unix command-line with 0x13 (^S) may appear to hang, but it’s just waiting on 0x11 (^Q).

What encodings to use for storing ciphertext? Well, anything that safely packs control characters.

1

solved Bad Data exception during RSA decryption