Tuesday, April 7, 2015

openssl tool gives other output that library


I've encrypted some data by command-line tool openssl with symmetric cipher and can't decrypt by some another util (or even from code using openssl library).

or vice versa:

I've encrypted some data with some crypto library or some tool (not openssl) with symmetric cipher and can't decrypt by openssl command-line tool.

So, in short - why you can encrypt-decrypt (by symmetric cipher) data only by openssl tool? Why another tools failed, when trying interact with openssl command-line tool by encrypted data?

I've dealt with blowfish algorithm in cfb mode, but I think this experience can be useful also with another simmetric cipher in openssl.

to pass lyrics and just get solution - just scroll to "problem solving in a short".

light introduction in blowfish.

Blowfish - it's symmetric block cypher. Encrypt blocks 64 bits (8 bytes). [link]
To encrypt data which size diff from 8 bytes used different 'modes': ECB, CBC, CFB, OFB, CTR. Other words, 'mode' transforms block cipher to stream cipher. [link]

So, to encode with blowfish something, we need to have:
  • text
  • key
  • mode
and one more thing - initialization vector (or just 'iv'). [link]
To decrypt encrypted data you must have the same 'iv' that was used for encryption. You can consider it as second key. Or just number, which must be synchronized between Alice and Bob.

illustration of a problem (a bit of LOL):

lets try encrypt this:
  • text: my_lovely_text
  • key: mypass
  • algotithm: blowfish
  • mode: cfb
  • iv: filled by binary zeros

openssl tool 
(I know that's key usage is wrong, bcs of described further nuances. But for illustration this naive logic works fine):
    openssl.exe enc -bf-cfb -in file.txt -out file.bf -nosalt
    (it asked you password, you entered 'mypass' and it gives data):
        58 3C 1A 88-BC FF 2E 02-5C EF EA 1F-15 C9

    okay... but we forgot to set 'iv'. Who knows which 'iv' is default in openssl. Let's try one more time.
    openssl.exe enc -iv 0 -bf-cfb -in file.txt -out file.bf -nosalt
    (it asked you password, you entered 'mypass' and it gives data):
        95 3F 04 E5-FD F7 75 4E-B8 9E 39 AA-E1 2F

okay. lets see what another sources give us on the same data with the same parameters:
(I just took it from 1st page of google on request 'online encrypt blowfish cfb')

    (it doesn't have 'iv' field - so, it's some 'defaulted' value)
    (set "Encode the output using" to "HEXA"):

    here are "Init.vector" generated by password. Let's try both - default and zero iv.
    iv: e7 27 d1 46 4a e1 24 36
    result: d2    70    78    f8    0c    81    d1    9e    6b    4a    57    15    4e    33

    okay, let's try zero iv
    iv: 00 00 00 00 00 00 00 00
    result: 3c    38    ba    2e    46    c4    2a    55    38    07    1f    de    89    e3
    hmmm. And here are exists some blowfish-compat case - let's try it.

    iv: e7 27 d1 46 4a e1 24 36
    result: 04    8a    82    b6    02    db    e3    a0    38    70    87    64    3c    e7

    okay. that was default. let's try zero iv.
    iv: 00 00 00 00 00 00 00 00
    result: 3b    3e    54    6a    51    4c    8c    a7    98    0b    0f    37    9f    c4

    here are no 'iv' field - so, it must be some defaulted. Enter our data and get:
        BLOWFISH:    2cd10b1ad66f742c7db38fbab105
    okay. Buy let's try one more time (just to be sure)
        BLOWFISH:    94408e95ac0d6eac2aa3f9f61a80
    hmm... one more time?
        BLOWFISH:    4c1e679fa0fcd8e4c8e45a411462
        BLOWFISH:    c2252b7a79193bd512f1e4ff2476
    looks like it returns random numbers.

    (it's length 16 bytes - it's something new, because earlier every tool gives on my text, which length is 14 bytes - output 14 bytes, and it gave 16 bytes. Must be padding)

and let's try some script languages.

python crypto

from Crypto.Cipher import Blowfish
c = Blowfish.new('mypass', Blowfish.MODE_CFB, "\0"*8)
cyphertext = c.encrypt('my_lovely_text')
print (cyphertext.encode('hex'))     3c38ba2e46c42a5538071fde89e3

    WHOA - the first match! http://blowfish.online-domain-tools.com/ gave the same.

php mcrypt

$e = mcrypt_encrypt(MCRYPT_BLOWFISH, "mypass", "my_lovely_text", MCRYPT_MODE_CFB, "\0\0\0\0\0\0\0\0");
echo "$e";

    3C 38 BA 2E-46 C4 2A 55-38 07 1F DE-89 E3
    Wow - yet another match.

perl openssl

use Crypt::OpenSSL::Blowfish::CFB64;
my $crypt = Crypt::OpenSSL::Blowfish::CFB64->new('mypass', pack( C8 => 0,0,0,0,0,0,0,0 ));
my $binary_data = $crypt->encrypt("my_lovely_text");
print $binary_data;

    3C 49 42 3A-0F 4F 65 50-67 29 74 36-31 37
    looks like yet another variant %)

c (well, actually a bit of c++, but who cares) openssl

#include <stdlib.h>
#include <string.h>
#include <openssl/blowfish.h>
#include <fstream>

int main(int argc, char** argv)
    unsigned char out_buf[1024];

    const unsigned char plaintext[] = "my_lovely_text";
    int plaintext_len = strlen((const char*)plaintext);
    const unsigned char key[] = "mypass";

    int counter = 0;
    unsigned char iv[8];
    BF_KEY keyStruct;

    memset(iv, 0, 8);
    BF_set_key(&keyStruct, strlen((const char*)key), key);

    std::ofstream o("file.bf", std::ios::binary);
    o.write((char*)out_buf, plaintext_len);

    3C 49 42 3A-0F 4F 65 50-67 29 74 36-31 37
    God bless at least openssl library on different languages gives the same results)

So. Let's make summary table: (I've marked matched data by colors)
583C1A88BCFF2E025CEFEA1F15C9openssl tooldefault
953F04E5FDF7754EB89E39AAE12Fopenssl toolexplicit set 'iv'=0
302416d7806565d1a15e6b58666etools4noobsdefault (the only option here)
3c38ba2e46c42a5538071fde89e3blowfish.online-domain-toolsexplicit set 'iv'=0
3b3e546a514c8ca7980b0f379fc4blowfish-compat.online-domain-toolsexplicit set 'iv'=0
2cd10b1ad66f742c7db38fbab105bierkandt.orgdefault (the only option here)
94408e95ac0d6eac2aa3f9f61a80bierkandt.orgdefault (the only option here)
4c1e679fa0fcd8e4c8e45a411462bierkandt.orgdefault (the only option here)
c2252b7a79193bd512f1e4ff2476bierkandt.orgdefault (the only option here)

2e36059b6aa5482ffbe5fc3a7d0c3662sladex.org/blowfish.jsdefault (the only option here)
3c38ba2e46c42a5538071fde89e3python cryptoexplicit set 'iv'=0
3C38BA2E46C42A5538071FDE89E3php mcryptexplicit set 'iv'=0
3C49423A0F4F6550672974363137perl OpenSSL::Blowfish::CFB64explicit set 'iv'=0
3C49423A0F4F6550672974363137c/c++ OpenSSLexplicit set 'iv'=0

And. It looks like everybody has own version of blowfish cfb encryption. And only openssl & mcrypt gives the same results on different languages. Somebody even has own padding.
And everything is ok, till you don't need communicate between languages & libraries.
And also interesting part - openssl tool gives result, which differs from openssl lib. Let's see that.


Okay, I don't want consider every tool - why every tool gives another result. But I will explain about why at least openssl tool and openssl library give different results.

This part I understood, when sit whole day in olly debugger and debugged openssl. Openssl - it's something like framework, where you can easily make crypto-chains of algorithms, and it seems a lot of work with connection parts happened during compilance. So, on static sources I couldn't understood how called blowfish from 'main' function. And here are some extracting of that experience.

First of all - here are exists key for openssl tool '-p':
> openssl.exe enc -p -bf-cfb -in file.txt -out file.bf -nosalt
(enter 'mypass' when it asked)
    iv =A10CE9C4682486F8

okay. let's repeat it:
    iv =A10CE9C4682486F8

okay. at least it stable) let's repeat and enter 'mypass2' as password
    iv =836291E15B739ECD

And we see - not only 'key' != 'pass', here are also 'iv' produced by password.

So, the first big think to understanding - is: "Even if you are using -nosalt param - key isn't equal pass - key is md5(pass)"

Okay, let's try enter 'mypass' as key. Here are option '-K' special for that. Well, it accepts key as hex, so
    string 'mypass' in hex - "6D7970617373"
> openssl.exe enc -p -bf-cfb -in file.txt -out file.bf -nosalt -K 6D7970617373 -iv 0
    iv =0000000000000000
The point is - aligning of key. In console tool openssl key always aligned to 16 bytes, and olly debugger accepted these conclusions. Mcrypt & OpenSSL library - both works fine with any length of key.

In documentation (https://www.openssl.org/docs/apps/enc.html) the only phrase about that:
    "Blowfish and RC5 algorithms use a 128 bit key"

    - key length always 16 bytes. If you have less - it completed it by zeros to 16 length.
    - key = md5(pass)
    - iv - derives from pass

problem solving in a short

  • use key '-p' to ensure you have key & iv values which you entered
  • key != pass. Without salt (option '-nosalt') key=md5(pass).
  • length of key for command line tool is ALWAYS 16 bytes
  • how 'iv' generated from pass - I don't know.

let's make friends - openssl tool & openssl lib

What do we know for now. To make friends openssl tool & openssl lib we need:
  • take key length 16 bytes
  • not mix up key & pass
  • add '-p' for ensuring we have write data
  • turn off salt
openssl tool    
> openssl.exe enc -bf-cfb -in file.txt -out file.bf -nosalt -K ABABABABABABABABABABABABABABABAB -iv 0 -p
iv =0000000000000000
file.bf content: 63 C0 0D 29-6D F8 8D 4A-94 81 81 88-41 8C

openssl lib from perl
use Crypt::OpenSSL::Blowfish::CFB64;
my $crypt = Crypt::OpenSSL::Blowfish::CFB64->new("\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB", pack( C8 => 0,0,0,0,0,0,0,0 ));
my $binary_data = $crypt->encrypt("my_lovely_text");
print $binary_data;

file.bf content: 63 C0 0D 29-6D F8 8D 4A-94 81 81 88-41 8C

So. At least here are clear - what to do openssl tool. Of course, we still have 'summary table', which shows that everybody has own understanding of blowfish cfb - but in the next note I will make mcrypt & openssl compatible.


If you need encrypt/decrypt data only using openssl - then it's perfect.
If you need interact with something else - well, here are nuances, one of them - key_length always 16 bytes.

useful sources & links

actually so-called 'blowfish-compat' - it's only mcrypt thing and it's about endianing. [link]

No comments:

Post a Comment