notes of yara 3.4.0 compiling

tools & version:

• windows 8.1
• visual studio 2013
• yara library 3.3.0 or 3.2.0

foreword

Just notes of yara compiling process on windows with visual studio. I will compile without CUCKOO support - bcs I don't need this & don't want to compile yet another library (jansson64.lib)

action

1) unpack archive

2) open solution: .\yara-3.3.0\windows\yara\yara.sln

3) open 'utils.h' -> replace '#define YR_API EXTERNC __declspec(dllexport)' to '#define YR_API EXTERNC' (bcs I don't like exported symbols in my exe files, and link I wanna statically)

4) choose platform & mode

5) set runtime library for all projects (yara & yarac & libyara):
    properties -> c/c++ -> code generation -> runtime library -> /MTd for debug or /MT for release
    (you can select several projects in time - using 'ctrl'+left_mouse_button_click)

6) add to "Preprocessor Definitions" of all projects
    (Properties -> Configuration Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions)
    lines to not conflict with mysql c connector, for example:
strlcpy=libyara_internal_strlcpy
strlcat=libyara_internal_strlcat

7) Go to libyara properties:
    Properties -> Configuration Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions
    and delete CUCKOO from this list.
Then go to libyara properties:
    Properties -> Configuration Properties -> Librarian -> General -> Additional Dependencies
    and delete jansson64.lib from this list.

7) Here are you must choose - you want to compile it with openssl or without.
Why do you need openssl in yara library:
    - Generate an import hash: https://www.mandiant.com/blog/tracking-malware-import-hashing/ (uses define HAVE_LIBCRYPTO)
    - PE module of yara can extract some info from pe digital signature certificate. (uses define HAVE_LIBCRYPTO)
#if defined(HAVE_LIBCRYPTO)
begin_struct_array("signatures");
  declare_string("issuer");
  declare_string("subject");
  declare_integer("version");
  declare_string("algorithm");
  declare_string("serial");
  declare_integer("not_before");
  declare_integer("not_after");
  declare_function("valid_on", "i", "i", valid_on);
  end_struct_array("signatures");
declare_integer("number_of_signatures");
#endif   
    - HASH module of yara can calc provide you cryptographic hash functions: md5, sha1, sha256, checksum32 (uses define HASH, appeared in 3.3.0 version)
   
If you need some of this functionality - you need to build openssl & you need add for all projects:
    - Additional library directory
    - library file of openssl (libeay32.lib on my pc)

If you don't need this functionality
    - delete HAVE_LIBCRYPTO & HASH from "Preprocessor Definitions" of libyara, and insert HAVE_TIMEGM line - else you get undefined type 'tm'.
        Properties -> Configuration Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions
    - delete libeay64.lib from
        Properties -> Configuration Properties -> Librarian -> General -> Additional Dependencies

After that everything will compiles fine.

Possible errors for googling people

warnings:

---------------------------------------------------------------------------------------------
 in x86 (32-bits mode) you will get next warnings:
    3>args.obj : warning LNK4075: ignoring '/EDITANDCONTINUE' due to '/OPT:LBR' specification
    2>args.obj : warning LNK4075: ignoring '/EDITANDCONTINUE' due to '/OPT:LBR' specification

---------------------------------------------------------------------------------------------
1>D:\blablabla\yara-3.3.0\libyara\include\yara/object.h(23): warning C4005: 'INFINITY' : macro redefinition
1>          C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\math.h(102) : see previous definition of 'INFINITY'
1>D:\blablabla\yara-3.3.0\libyara\include\yara/object.h(24): warning C4005: 'NAN' : macro redefinition
1>          C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\math.h(106) : see previous definition of 'NAN'

---------------------------------------------------------------------------------------------
1>libeay32.lib(fips_ers.obj) : warning LNK4221: This object file does not define any previously undefined public symbols, so it will not be used by any link operation that consumes this library
1>libeay32.lib(ecp_nistp224.obj) : warning LNK4221: This object file does not define any previously undefined public symbols, so it will not be used by any link operation that consumes this library
1>libeay32.lib(ecp_nistp256.obj) : warning LNK4221: This object file does not define any previously undefined public symbols, so it will not be used by any link operation that consumes this library
1>libeay32.lib(ecp_nistp521.obj) : warning LNK4221: This object file does not define any previously undefined public symbols, so it will not be used by any link operation that consumes this library
1>libeay32.lib(ecp_nistputil.obj) : warning LNK4221: This object file does not define any previously undefined public symbols, so it will not be used by any link operation that consumes this library

---------------------------------------------------------------------------------------------
what to do - ignore them


if you didn't set HAVE_TIMEGM

---------------------------------------------------------------------------------------------
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(23): error C2027: use of undefined type 'tm'
1>          d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(14) : see declaration of 'tm'
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(23): error C2227: left of '->tm_year' must point to class/struct/union/generic type
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(26): error C2027: use of undefined type 'tm'
1>          d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(14) : see declaration of 'tm'
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(26): error C2227: left of '->tm_mon' must point to class/struct/union/generic type
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(27): error C2027: use of undefined type 'tm'
1>          d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(14) : see declaration of 'tm'
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(27): error C2227: left of '->tm_year' must point to class/struct/union/generic type
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(29): error C2027: use of undefined type 'tm'
1>          d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(14) : see declaration of 'tm'
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(29): error C2227: left of '->tm_mday' must point to class/struct/union/generic type
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(31): error C2027: use of undefined type 'tm'
1>          d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(14) : see declaration of 'tm'
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(31): error C2227: left of '->tm_hour' must point to class/struct/union/generic type
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(33): error C2027: use of undefined type 'tm'
1>          d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(14) : see declaration of 'tm'
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(33): error C2227: left of '->tm_min' must point to class/struct/union/generic type
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(35): error C2027: use of undefined type 'tm'
1>          d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(14) : see declaration of 'tm'
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(35): error C2227: left of '->tm_sec' must point to class/struct/union/generic type

---------------------------------------------------------------------------------------------
what to do - add HAVE_TIMEGM

blowfish in cfb mode in openssl & mcrypt

foreword

Some time ago I've needed to make client for one client-server protocol, where one step included blowfish-cfb encryption/decryption of data. Server side was written in PHP and used mcrypt library for encryption (need to note - mcrypt died in 2003). On server side it looked like that:
    $result = mcrypt_decrypt(MCRYPT_BLOWFISH, $key, $data, MCRYPT_MODE_CFB);
    $result = mcrypt_encrypt(MCRYPT_BLOWFISH, $key, $data, MCRYPT_MODE_CFB);
and that was bad code, bcs there are no 5th parameter. It set indirectly to \0\0\0\0\0\0\0\0. and it worked only before PHP 5.6 - after it won't work. In documentation written: "Invalid key and iv sizes are no longer accepted. mcrypt_encrypt() will now throw a warning and return FALSE if the inputs are invalid. Previously keys and IVs were padded with '\0' bytes to the next valid size." but this is not a problem - it's just little sad moment.

I've used c++ and OpenSSL - so, I've found description such functional in documentation (https://www.openssl.org/docs/crypto/blowfish.html):
    void BF_cfb64_encrypt(const unsigned char *in, unsigned char *out,
        long length, BF_KEY *schedule, unsigned char *ivec, int *num,
        int enc);
    "BF_cfb64_encrypt() is the CFB mode for Blowfish with 64 bit feedback."
    BF_cfb64_encrypt was the only function in openssl, which matched what I searched - blowfish & cfb

I've made encryption/decryption function using OpenSSL and found a surprise - the same data, encrypted with the same key & same 'iv' after c++&openssl and after php&mcrypt was completely different. Well, not completely - only the first byte was the same. So, it was very strange: in both cases I had some classical algothitm - blowfish - which exists more than 20 years; and some standard mode - cfb. I've spent a lot of time (something like 3 days) for understanding - I've loaded php script into php interpretator, and opened this interpretator in ollydbg, and debugged debugged debugged .

Openssl and mcrypt - both of these libraries - it's like framework, where you can connect primitives, but with some difference:

• in mcrypt you have privitive of CFB and block cipher Blowfish - and they somehow connected during compiling & runtime - so, in sources you can't understand how it connected.
• in openssl you have BF_cfb64_encrypt, but ciphers can be connected into chains - and it happens also during compiling & runtime.

so, for this case I debugged in ollydbg only mcrypt (as part of php interpreter), but such modular architecture in openssl forced me debug it into previous note.

I wanted to write only this note, but during this research appeared 3 more notes.
And I've read a lot of discussions of this problem - and didn't find one simple answer. So, that's why I decided to write of this problem.


problem

When I encrypt data with blowfish in cfb mode on openssl - I can't decrypt on mcrypt correctly.
and vice verca:
When I encrypt data with mcrypt in cfb mode on openssl - I can't decrypt on openssl correctly.

It looked like these libraries incompatible, despite they implement the same.

This problem affects a lot of languages (I've seen bindings to: c/c++, perl, python, php, ruby, rust, nodejs, java, even haskell, lisp, lua, go, rust and some languages which I've not heard - nim, racket), because openssl & mcrypt - ones of the most famous crypto libraries, and any wide-spreaded programming language have bindings/wrappers/modules to these libraries. And if someone writes it's own implementation of blowfish-cfb - this person will likely check is it work correct, in comparison with one of these libraries. Such examples:

• C# bouncycastle library (behave as openssl)
• python crypto (behave as mcrypt by default)

Last commit to mcrypt project was in 2003 [link]. You shouldn't use it in your new projects.

to pass lyrics and just get solution - just scroll to "problem solve in a short".


illustration of a problem

php mcrypt
#--------------------------------------------------------------
<?php
    $e = mcrypt_encrypt(MCRYPT_BLOWFISH, "mypass", "my_lovely_text", MCRYPT_MODE_CFB, "\0\0\0\0\0\0\0\0");
    echo "$e";
?>
#--------------------------------------------------------------
    3C 38 BA 2E-46 C4 2A 55-38 07 1F DE-89 E3

perl openssl
#--------------------------------------------------------------
use Crypt::OpenSSL::Blowfish::CFB64;
my $crypt = Crypt::OpenSSL::Blowfish::CFB64->new('mypass', pack( C8 => 0,0,0,0,0,0,0,0 ));
my $binary_data = $crypt->encrypt("my_lovely_text");
print $binary_data;
#--------------------------------------------------------------
    3C 49 42 3A-0F 4F 65 50-67 29 74 36-31 37

C/C++ openssl:
//------------------------------------------------------------
#include <string.h>
#include <openssl/blowfish.h>

int main(int argc, char** argv)
{
    const unsigned char plaintext[] = "my_lovely_text";
    const unsigned char key[] = "mypass";

    unsigned char out_buf[1024];

    int counter = 0;
    unsigned char iv[8];
    BF_KEY keyStruct;

    memset(iv, 0, 8);
    BF_set_key(&keyStruct, strlen((const char*)key), key);
    BF_cfb64_encrypt(
        plaintext,
        out_buf,
        strlen((const char*)plaintext),
        &keyStruct,
        iv,
        &counter,
        BF_ENCRYPT
        );
   
    std::ofstream o("file.bf", std::ios::binary);
    o.write((char*)out_buf, plaintext_len);
    o.close();
   
    return 0;
}
//------------------------------------------------------------
    3c 49 42 3a 0f 4f 65 50 67 29 74 36 31 37


same_problems

I've found in the Internet several similar problems:
    0) https://github.com/tugrul/node-mcrypt
        The author writes:
        "There is already OpenSSL extension bundled with Node.js but there are something wrong with some traditional encryption algorithms on OpenSSL"
        Someone tired from this discrepancy and made his own wrapper on mcrypt.
        This is not looks like answer for question 'why?'.
    1) here are some Russian guy with such problem
        http://phpclub.ru/talk/threads/delphi-php-mcrypt-blowfish.53690/
        http://www.sql.ru/forum/544138/delphi-php-mcrypt-blowfish
        Nodoby proposed good hypothesis.
    2) here are also some Russian guy with such problem
        http://ru-root.livejournal.com/2759003.html
        and solution he found - use mcrypt instead of openssl :D
        But he noticed that the 1st byte is the same. When I dealt with that - I didn't notice that.
        And actually he was the most closest to the solving problem.
        And in comments there was right answer, but I missed it when googled answer.
    3) here are encrypt data with php and trying decrypt with nodejs, also check if everything is correct with openssl console tool (this theme I've mentioned here)
        http://stackoverflow.com/questions/8895370/decrypting-mcrypt-encoded-text-with-node-js
       
so, everybody adviced:
    - use some padding
    - use iv
    - use for decryption the same library which encrypted data
and nobody seen the reason - why it happens? :-/


lyrics

to make long story short - I will pass my adventures with olly debugger & just leave here source of

• openssl blowfish encryption in cfb mode, and 
• generic cfb algorithm - from _mcrypt, where _mcrypt_block_encrypt will be blowfish block encryption.

// openssl blowfish cfb
// openssl-1.0.1j\crypto\bf\bf_cfb64.c

#define BF_ENCRYPT      1
#define BF_DECRYPT      0

void BF_cfb64_encrypt(const unsigned char *in, unsigned char *out, long length,
         const BF_KEY *schedule, unsigned char *ivec, int *num, int encrypt)
    {
    register BF_LONG v0,v1,t;
    register int n= *num;
    register long l=length;
    BF_LONG ti[2];
    unsigned char *iv,c,cc;

    iv=(unsigned char *)ivec;
    if (encrypt)
        {
        while (l--)
            {
            if (n == 0)
                {
                n2l(iv,v0); ti[0]=v0;
                n2l(iv,v1); ti[1]=v1;
                BF_encrypt((BF_LONG *)ti,schedule);
                iv=(unsigned char *)ivec;
                t=ti[0]; l2n(t,iv);
                t=ti[1]; l2n(t,iv);
                iv=(unsigned char *)ivec;
                }
            c= *(in++)^iv[n];
            *(out++)=c;
            iv[n]=c;
            n=(n+1)&0x07;
            }
        }
    else
        {
        while (l--)
            {
            if (n == 0)
                {
                n2l(iv,v0); ti[0]=v0;
                n2l(iv,v1); ti[1]=v1;
                BF_encrypt((BF_LONG *)ti,schedule);
                iv=(unsigned char *)ivec;
                t=ti[0]; l2n(t,iv);
                t=ti[1]; l2n(t,iv);
                iv=(unsigned char *)ivec;
                }
            cc= *(in++);
            c=iv[n];
            iv[n]=cc;
            *(out++)=c^cc;
            n=(n+1)&0x07;
            }
        }
    v0=v1=ti[0]=ti[1]=t=c=cc=0;
    *num=n;
    }









// mcrypt blowfish cfb
// mcrypt\modules\modes\cfb.c

int _mcrypt( CFB_BUFFER* buf, void *plaintext, int len, int blocksize, void* akey, void (*func)(void*,void*), void (*func2)(void*,void*))
{                /* plaintext is 1 byte (8bit cfb) */
    char *plain = plaintext;
    int i, j;
    void (*_mcrypt_block_encrypt) (void *, void *);

    _mcrypt_block_encrypt = func;

    for (j = 0; j < len; j++) {

        memcpy(buf->enc_s_register, buf->s_register, blocksize);

        _mcrypt_block_encrypt(akey, buf->enc_s_register);

        plain[j] ^= buf->enc_s_register[0];

/* Shift the register */
        for (i = 0; i < (blocksize - 1); i++)
            buf->s_register[i] = buf->s_register[i + 1];

        buf->s_register[blocksize - 1] = plain[j];
    }

    return 0;

}




so, the problem is - it's two different algorithms!
mcrypt xor 8-bytes block and then move window to 1 byte.
openssl xor 8-bytes block and then move window to 8 byte.

and wikipedia says (http://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Cipher_Feedback_.28CFB.29)
    "This way of proceeding is known as CFB-8 or CFB-1 (according to the size of the shifting)"
As I see in source code - it often calls cfb64 and cfb8 (instead of cfb-8 and cfb-1), but the point is correct - cfb can be different.
And now, looking in sources, it's clear answer for question - "why the first byte always the same?".


solving

I already had openssl - built, included, proven. So, I've just made my own implementation of cfb8 on c++. It's mcrypt-compatible algorithm (I've tested it, of course), which imports BF_encrypt from openssl library.

#include <openssl/blowfish.h>
#include <vector>

#define endian_swap32(x) ((x << 24) | (x >> 24) | ((x&0xff00)<<8) | ((x&0xff0000)>>8))

void BlowfishEncryptCFB8(
    const std::vector<unsigned char> & in_buf,
    const std::vector<unsigned char> & key,
    const std::vector<unsigned char> & iv,
    std::vector<unsigned char> & out_buf
    )
{
    out_buf = in_buf;

    std::vector<unsigned char> internal_key(sizeof(BF_KEY));
    BF_KEY *key_ = (BF_KEY *)internal_key.data();

    BF_set_key(key_, key.size(), (const unsigned char*)key.data());

    std::vector<unsigned char> enc_s_register(8), s_register;
    s_register = iv;

    for (size_t i = 0; i < in_buf.size(); i++)
    {
        enc_s_register = s_register;

        __int32 * lp_enc_s_register = (__int32 *)enc_s_register.data();
        *lp_enc_s_register = endian_swap32(*lp_enc_s_register);
        *(lp_enc_s_register+1) = endian_swap32(*(lp_enc_s_register+1));

        BF_encrypt((BF_LONG *) enc_s_register.data(), key_);
           
        *lp_enc_s_register = endian_swap32(*lp_enc_s_register);
        *(lp_enc_s_register+1) = endian_swap32(*(lp_enc_s_register+1));

        out_buf[i] ^= enc_s_register[0];

        for (size_t j = 0; j < 7; j++)
            s_register[j] = s_register[j+1];
        s_register[7] = out_buf[i];
    }
}

void BlowfishEncryptCFB8(
    const std::vector<unsigned char> & in_buf,
    const std::vector<unsigned char> & key,
    const std::vector<unsigned char> & iv,
    std::vector<unsigned char> & out_buf
    )
{
    out_buf = in_buf;

    std::vector<unsigned char> internal_key(sizeof(BF_KEY));
    BF_KEY *key_ = (BF_KEY *)internal_key.data();

    BF_set_key(key_, key.size(), (const unsigned char*)key.data());

    std::vector<unsigned char> enc_s_register(8), s_register;
    s_register = iv;

    for (size_t i = 0; i < in_buf.size(); i++)
    {
        enc_s_register = s_register;

        __int32 * lp_enc_s_register = (__int32 *)enc_s_register.data();
        *lp_enc_s_register = endian_swap32(*lp_enc_s_register);
        *(lp_enc_s_register + 1) = endian_swap32(*(lp_enc_s_register + 1));

        BF_encrypt((BF_LONG *)enc_s_register.data(), key_);

        *lp_enc_s_register = endian_swap32(*lp_enc_s_register);
        *(lp_enc_s_register + 1) = endian_swap32(*(lp_enc_s_register + 1));

        unsigned char saved_byte = out_buf[i];
        out_buf[i] ^= enc_s_register[0];

        for (size_t j = 0; j < 7; j++)
            s_register[j] = s_register[j + 1];
        s_register[7] = saved_byte;
    }
}


problem solving in a short

• Realize that CFB mode - it's not some concrete action - it's common name for idea use previous ciphertext as key for next block.
• Realize that OpenSSL blowfish cfb - it's cfb64, and uses 64-bits feedback (that's why in sources this function called BF_cfb64_encrypt)
• Realize that mcrypt blowfish cfb - it's cfb8, and uses 8-bits feedback.
• determie which cfb mode do you have - for example, you can encode 
• key: mypass
• text: my_lovely_text
• iv: 0
• blowfish in cfb mode:
• 3C 38 BA 2E-46 C4 2A 55-38 07 1F DE-89 E3    cfb-8 (mcrypt-like)
• 3C 49 42 3A-0F 4F 65 50-67 29 74 36-31 37    cfb-64 (openssl-like)
• if you are using something based on mcrypt and want openssl-compatibility:
• search something like 'ncfb' mode - it must be openssl-compatible cfb mode.
• else you will have simple choice:
• find implementation of mode which you need
• make your own implementation of cfb - block blowfish you should already have.

useful sources & links

After I've found answer by myself, I've found useful these links:

• http://ru-root.livejournal.com/2759003.html?thread=41331035#t41331035
• http://metaclass.livejournal.com/889791.html
• https://bugs.php.net/bug.php?id=51146
• http://stackoverflow.com/questions/4052594/incorrect-key-size-in-php-mcrypt-when-porting-cryptopp-aes-encryption-to-phps-m/4054017#4054017
• http://thefsb.tumblr.com/post/111035508040/porting-php-code-from-mcrypt-to-openssl

final notes

Aes has cfb8 and cfb1 modes - https://www.openssl.org/docs/apps/enc.html
    Interesting - why blowfish doesn't have such choice, I wonder?

here are comparison openssl with mcrypt:
    http://www.synet.sk/php/en/320-benchmarking-symmetric-cyphers-openssl-vs-mcrypt-in-php
    Conclusions from this link: "OpenSSL offers significantly faster algorithms. This is the right choice for heavy traffic sites."
    well, on blowfish-cfb it just makes in 8 time less work :-D

"The source code in CSV was last updated in October 2003. The only file changed since then is the AUTHORS which was updated in 2007. There are 35 open bugs against mcrypt in SourceForge the oldest dating from 2003"
    http://thefsb.tumblr.com/post/110639027905/custodians-of-php-vote-to-keep-a-crypto-lib

openssl tool gives other output that library

problem

I've encrypted some data by command-line tool openssl with symmetric cipher and can't decrypt by some another util (or even from code using openssl library).

or vice versa:

I've encrypted some data with some crypto library or some tool (not openssl) with symmetric cipher and can't decrypt by openssl command-line tool.

So, in short - why you can encrypt-decrypt (by symmetric cipher) data only by openssl tool? Why another tools failed, when trying interact with openssl command-line tool by encrypted data?

I've dealt with blowfish algorithm in cfb mode, but I think this experience can be useful also with another simmetric cipher in openssl.

to pass lyrics and just get solution - just scroll to "problem solving in a short".


light introduction in blowfish.

Blowfish - it's symmetric block cypher. Encrypt blocks 64 bits (8 bytes). [link]
To encrypt data which size diff from 8 bytes used different 'modes': ECB, CBC, CFB, OFB, CTR. Other words, 'mode' transforms block cipher to stream cipher. [link]

So, to encode with blowfish something, we need to have:

• text
• key
• mode

and one more thing - initialization vector (or just 'iv'). [link]
To decrypt encrypted data you must have the same 'iv' that was used for encryption. You can consider it as second key. Or just number, which must be synchronized between Alice and Bob.


illustration of a problem (a bit of LOL):

lets try encrypt this:

• text: my_lovely_text
• key: mypass
• algotithm: blowfish
• mode: cfb
• iv: filled by binary zeros

openssl tool 
(I know that's key usage is wrong, bcs of described further nuances. But for illustration this naive logic works fine):
    openssl.exe enc -bf-cfb -in file.txt -out file.bf -nosalt
    (it asked you password, you entered 'mypass' and it gives data):
        58 3C 1A 88-BC FF 2E 02-5C EF EA 1F-15 C9

    okay... but we forgot to set 'iv'. Who knows which 'iv' is default in openssl. Let's try one more time.
    openssl.exe enc -iv 0 -bf-cfb -in file.txt -out file.bf -nosalt
    (it asked you password, you entered 'mypass' and it gives data):
        95 3F 04 E5-FD F7 75 4E-B8 9E 39 AA-E1 2F

okay. lets see what another sources give us on the same data with the same parameters:
(I just took it from 1st page of google on request 'online encrypt blowfish cfb')

http://www.tools4noobs.com/online_tools/encrypt/
    (it doesn't have 'iv' field - so, it's some 'defaulted' value)
    (set "Encode the output using" to "HEXA"):
        302416d7806565d1a15e6b58666e

http://blowfish.online-domain-tools.com/
    here are "Init.vector" generated by password. Let's try both - default and zero iv.
    iv: e7 27 d1 46 4a e1 24 36
    result: d2    70    78    f8    0c    81    d1    9e    6b    4a    57    15    4e    33

    okay, let's try zero iv
    iv: 00 00 00 00 00 00 00 00
    result: 3c    38    ba    2e    46    c4    2a    55    38    07    1f    de    89    e3
    hmmm. And here are exists some blowfish-compat case - let's try it.

http://blowfish-compat.online-domain-tools.com/
    iv: e7 27 d1 46 4a e1 24 36
    result: 04    8a    82    b6    02    db    e3    a0    38    70    87    64    3c    e7

    okay. that was default. let's try zero iv.
    iv: 00 00 00 00 00 00 00 00
    result: 3b    3e    54    6a    51    4c    8c    a7    98    0b    0f    37    9f    c4

http://www.bierkandt.org/encryption/symmetric_encryption.php
    here are no 'iv' field - so, it must be some defaulted. Enter our data and get:
        BLOWFISH:    2cd10b1ad66f742c7db38fbab105
    okay. Buy let's try one more time (just to be sure)
        BLOWFISH:    94408e95ac0d6eac2aa3f9f61a80
    hmm... one more time?
        BLOWFISH:    4c1e679fa0fcd8e4c8e45a411462
    hmmm....
        BLOWFISH:    c2252b7a79193bd512f1e4ff2476
    looks like it returns random numbers.

http://sladex.org/blowfish.js/
    2e36059b6aa5482ffbe5fc3a7d0c3662
    (it's length 16 bytes - it's something new, because earlier every tool gives on my text, which length is 14 bytes - output 14 bytes, and it gave 16 bytes. Must be padding)


and let's try some script languages.


python crypto


from Crypto.Cipher import Blowfish
c = Blowfish.new('mypass', Blowfish.MODE_CFB, "\0"*8)
cyphertext = c.encrypt('my_lovely_text')
print (cyphertext.encode('hex'))     3c38ba2e46c42a5538071fde89e3

    WHOA - the first match! http://blowfish.online-domain-tools.com/ gave the same.


php mcrypt

<?php
$e = mcrypt_encrypt(MCRYPT_BLOWFISH, "mypass", "my_lovely_text", MCRYPT_MODE_CFB, "\0\0\0\0\0\0\0\0");
echo "$e";
?>

    3C 38 BA 2E-46 C4 2A 55-38 07 1F DE-89 E3
    Wow - yet another match.

perl openssl

use Crypt::OpenSSL::Blowfish::CFB64;
my $crypt = Crypt::OpenSSL::Blowfish::CFB64->new('mypass', pack( C8 => 0,0,0,0,0,0,0,0 ));
my $binary_data = $crypt->encrypt("my_lovely_text");
print $binary_data;

    3C 49 42 3A-0F 4F 65 50-67 29 74 36-31 37
    looks like yet another variant %)

c (well, actually a bit of c++, but who cares) openssl

#--------------------------------------------------------------
#include <stdlib.h>
#include <string.h>
#include <openssl/blowfish.h>
#include <fstream>

int main(int argc, char** argv)
{
    unsigned char out_buf[1024];

    const unsigned char plaintext[] = "my_lovely_text";
    int plaintext_len = strlen((const char*)plaintext);
    const unsigned char key[] = "mypass";

    int counter = 0;
    unsigned char iv[8];
    BF_KEY keyStruct;

    memset(iv, 0, 8);
    BF_set_key(&keyStruct, strlen((const char*)key), key);
    BF_cfb64_encrypt(
        plaintext,
        out_buf,
        plaintext_len,
        &keyStruct,
        iv,
        &counter,
        BF_ENCRYPT
        );

    std::ofstream o("file.bf", std::ios::binary);
    o.write((char*)out_buf, plaintext_len);
    o.close();
}
#--------------------------------------------------------------

    3C 49 42 3A-0F 4F 65 50-67 29 74 36-31 37
    God bless at least openssl library on different languages gives the same results)


So. Let's make summary table: (I've marked matched data by colors)

encrypted_data	tool	iv
583C1A88BCFF2E025CEFEA1F15C9	openssl tool	default
953F04E5FDF7754EB89E39AAE12F	openssl tool	explicit set 'iv'=0
302416d7806565d1a15e6b58666e	tools4noobs	default (the only option here)
d27078f80c81d19e6b4a57154e33	blowfish.online-domain-tools	default
3c38ba2e46c42a5538071fde89e3	blowfish.online-domain-tools	explicit set 'iv'=0
048a82b602dbe3a0387087643ce7	blowfish-compat.online-domain-tools	default
3b3e546a514c8ca7980b0f379fc4	blowfish-compat.online-domain-tools	explicit set 'iv'=0
2cd10b1ad66f742c7db38fbab105	bierkandt.org	default (the only option here)
94408e95ac0d6eac2aa3f9f61a80	bierkandt.org	default (the only option here)
4c1e679fa0fcd8e4c8e45a411462	bierkandt.org	default (the only option here)
c2252b7a79193bd512f1e4ff2476	bierkandt.org	default (the only option here)
...
2e36059b6aa5482ffbe5fc3a7d0c3662	sladex.org/blowfish.js	default (the only option here)
3c38ba2e46c42a5538071fde89e3	python crypto	explicit set 'iv'=0
3C38BA2E46C42A5538071FDE89E3	php mcrypt	explicit set 'iv'=0
3C49423A0F4F6550672974363137	perl OpenSSL::Blowfish::CFB64	explicit set 'iv'=0
3C49423A0F4F6550672974363137	c/c++ OpenSSL	explicit set 'iv'=0

And. It looks like everybody has own version of blowfish cfb encryption. And only openssl & mcrypt gives the same results on different languages. Somebody even has own padding.
And everything is ok, till you don't need communicate between languages & libraries.
And also interesting part - openssl tool gives result, which differs from openssl lib. Let's see that.


lyrics

Okay, I don't want consider every tool - why every tool gives another result. But I will explain about why at least openssl tool and openssl library give different results.

This part I understood, when sit whole day in olly debugger and debugged openssl. Openssl - it's something like framework, where you can easily make crypto-chains of algorithms, and it seems a lot of work with connection parts happened during compilance. So, on static sources I couldn't understood how called blowfish from 'main' function. And here are some extracting of that experience.

First of all - here are exists key for openssl tool '-p':
> openssl.exe enc -p -bf-cfb -in file.txt -out file.bf -nosalt
(enter 'mypass' when it asked)
    key=A029D0DF84EB5549C641E04A9EF389E5
    iv =A10CE9C4682486F8

okay. let's repeat it:
    key=A029D0DF84EB5549C641E04A9EF389E5
    iv =A10CE9C4682486F8

okay. at least it stable) let's repeat and enter 'mypass2' as password
    key=29E80F01374C71764422B94532A4B336
    iv =836291E15B739ECD

And we see - not only 'key' != 'pass', here are also 'iv' produced by password.

So, the first big think to understanding - is: "Even if you are using -nosalt param - key isn't equal pass - key is md5(pass)"

Okay, let's try enter 'mypass' as key. Here are option '-K' special for that. Well, it accepts key as hex, so
    string 'mypass' in hex - "6D7970617373"
and
> openssl.exe enc -p -bf-cfb -in file.txt -out file.bf -nosalt -K 6D7970617373 -iv 0
    key=6D797061737300000000000000000000
    iv =0000000000000000
The point is - aligning of key. In console tool openssl key always aligned to 16 bytes, and olly debugger accepted these conclusions. Mcrypt & OpenSSL library - both works fine with any length of key.

In documentation (https://www.openssl.org/docs/apps/enc.html) the only phrase about that:
    "Blowfish and RC5 algorithms use a 128 bit key"

So:
    - key length always 16 bytes. If you have less - it completed it by zeros to 16 length.
    - key = md5(pass)
    - iv - derives from pass


problem solving in a short

• use key '-p' to ensure you have key & iv values which you entered
• key != pass. Without salt (option '-nosalt') key=md5(pass).
• length of key for command line tool is ALWAYS 16 bytes
• how 'iv' generated from pass - I don't know.

let's make friends - openssl tool & openssl lib

What do we know for now. To make friends openssl tool & openssl lib we need:

• take key length 16 bytes
• not mix up key & pass
• add '-p' for ensuring we have write data
• turn off salt

openssl tool    
> openssl.exe enc -bf-cfb -in file.txt -out file.bf -nosalt -K ABABABABABABABABABABABABABABABAB -iv 0 -p
key=ABABABABABABABABABABABABABABABAB
iv =0000000000000000
file.bf content: 63 C0 0D 29-6D F8 8D 4A-94 81 81 88-41 8C

openssl lib from perl
use Crypt::OpenSSL::Blowfish::CFB64;
my $crypt = Crypt::OpenSSL::Blowfish::CFB64->new("\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB\xAB", pack( C8 => 0,0,0,0,0,0,0,0 ));
my $binary_data = $crypt->encrypt("my_lovely_text");
print $binary_data;
file.bf content: 63 C0 0D 29-6D F8 8D 4A-94 81 81 88-41 8C

So. At least here are clear - what to do openssl tool. Of course, we still have 'summary table', which shows that everybody has own understanding of blowfish cfb - but in the next note I will make mcrypt & openssl compatible.



conclusion

If you need encrypt/decrypt data only using openssl - then it's perfect.
If you need interact with something else - well, here are nuances, one of them - key_length always 16 bytes.


useful sources & links

actually so-called 'blowfish-compat' - it's only mcrypt thing and it's about endianing. [link]

add openssl modules to perl on windows

problem

Some perl cpan modules required openssl installed for your pc. Here are some of them:

• Crypt::OpenSSL::AES
• Crypt::OpenSSL::Bignum
• Crypt::OpenSSL::Bignum::CTX
• Crypt::OpenSSL::Blowfish
• Crypt::OpenSSL::Blowfish::CFB64
• Crypt::OpenSSL::CA
• Crypt::OpenSSL::CA::AlphabetSoup
• Crypt::OpenSSL::CA::Inline::C
• Crypt::OpenSSL::CA::Resources
• Crypt::OpenSSL::Cloner
• Crypt::OpenSSL::Common
• Crypt::OpenSSL::DSA
• Crypt::OpenSSL::DSA::Signature
• Crypt::OpenSSL::EC
• Crypt::OpenSSL::ECDSA
• Crypt::OpenSSL::PBKDF2
• Crypt::OpenSSL::PKCS10
• Crypt::OpenSSL::PKCS12
• Crypt::OpenSSL::Random
• Crypt::OpenSSL::RC4
• Crypt::OpenSSL::RSA
• Crypt::OpenSSL::SMIME
• Crypt::OpenSSL::VerifyX509
• Crypt::OpenSSL::X509
• Crypt::SSLeay

That happens bcs they are just warppers over openssl functions.
When you are trying install such module - you get source code, which includes some header from openssl library & trying link with openssl library on your pc.
on unix-like systems this problem seems solves quickly - any of next commands:

• sudo yum install perl-Crypt-SSLeay
• sudo yum install openssl-devel
• sudo apt-get install libssl-dev

that's so simple on unix-like systems because in unix you have some standard paths to headers & libraries. So, when you need to find library - you know where to search. On Windows, users don't have such cool thing - that's source of this problem.
And I didn't find any solutions of such problem in the Internet - so, solved & writed it here. Also I'm not pretend on some knowledge perl or it's package managers or something like that - I'm just using perl for solving my tasks, encounted problem and solved it.
In this note I will describe common principles of solving such problems - so, I think this experience can be expanded to any such problem.
to pass lyrics and just get solution - just scroll to "problem solving in a short".


tools & versions

I will describe on example of ActivePerl (and 64 and 32 bit version). Don't know what about strawberry perl, for example - but I think common idea will be the same.
I have ActivePerl v5.18.4

• installed in C:\Perl64 for x86-64 version.
• installed in C:\Perl for x86 version.

(I didn't install both of them in time - it must lead to confusion, but I've checked this approach on both versions - so, when you will see action for two versions - x86 and x86-64 - just choose your one)

Also you need to build openssl with mingw (for x86-64 target platforms - with mingw64).
How to do that - I described in note earlier - [link].
Why exactly mingw - bcs it's much simpler than add VisualStudio-compiled openssl. In the end of this note I will say how to add openssl, compiled in Visual Studio - but I don't like that.


Crypt::SSLeay

Module Crypt::SSLeay has easier solution that other mentioned, bcs it has some extra opportunities for installing. It has way to know where this libraries located ([link]):
> set OPENSSL_INCLUDE=...
> set OPENSSL_LIB=...
> cpanm Crypt::SSLeay
So, if you need only "Crypt::SSLeay" - you can just use it.
But this is specific only for this module - so, it's not universal method.


lyrics

What happened when you are trying install such module - for example 'cpan install Crypt::OpenSSL::Blowfish'? Well, first of all - if it's your fist 'cpan install' when you need compile stuff - it will bring you tools for compiling code (with mingw). You will see something like that:
--------------------------------------------------------------------------------------------------------
It looks like you don't have a C compiler and make utility installed.  Trying
to install dmake and the MinGW gcc compiler using the Perl Package Manager.
This may take a a few minutes...

Syncing site PPM database with .packlists...done
Downloading MinGW-4.6.3...done
Downloading dmake-4.11.20080107...done
Unpacking MinGW-4.6.3...done
Unpacking dmake-4.11.20080107...done
Generating HTML for MinGW-4.6.3...done
Generating HTML for dmake-4.11.20080107...done
Updating files in site area...
3697 files installed
--------------------------------------------------------------------------------------------------------
This utils will be located here:
    C:\Perl\cpan\build
or
    C:\Perl64\cpan\build

Then it will create directory in your cpan build directory
    C:\Perl\cpan\build
or
    C:\Perl64\cpan\build
in format:
    %package_name%-%version%-%six_looked_random_symbols%
(also it put .yml file with the same name, but for describing task it doesn't matter)

and here are extracted package archive and in this directory will be building process.
for example in 'Crypt-OpenSSL-RSA-0.28-gB_yPv'

here are located Makefile.PL file. For Crypt::OpenSSL this script uses module ExtUtils::MakeMaker. This 'Makefile.PL' script generates 'Makefile'. And here are the key thing - this script searches libraries and generates make file to linking with libraries, present on your pc!
When it can't find library it prints to err console stream messages like that:
...
Warning (mostly harmless): No library found for -lcrypto
Warning (mostly harmless): No library found for -lmoldname
...

Then, after Makefile.PL generated makefile, called 'dmake' util, which will try compile stuff accordingly to information in 'makefile' (on example installing Crypt::OpenSSL::Blowfish)
Blowfish.xs:5:30: fatal error: openssl/blowfish.h: No such file or directory
or stuff like that

So, the first step to solve this problem - copy your openssl includes directory to place where perl will find it:
    C:\Perl\site\lib\auto\MinGW\i686-w64-mingw32\include
or
    C:\Perl64\site\lib\auto\MinGW\x86_64-w64-mingw32\include

Then if you will try again 'cpan install Crypt::OpenSSL::Blowfish', it will find includes, but not find libraries:
Warning (mostly harmless): No library found for -lcrypto
...
Blowfish.o:Blowfish.c:(.text+0x125): undefined reference to `BF_encrypt'
Blowfish.o:Blowfish.c:(.text+0x151): undefined reference to `BF_decrypt'
Blowfish.o:Blowfish.c:(.text+0x2c3): undefined reference to `BF_set_key'

so, you need to put libraries where ExtUtils::MakeMaker will find it. Where ExtUtils::MakeMaker searches this files - you can understand by 'Process Monitor' tool [link]. Just turn on logging all events, run 'cpan install Crypt::OpenSSL::Blowfish', wait error message, save log to csv file and then grep by that. So, finally you will get list of locations where it searches libraries (part of log, grepped by 'ssl'):
        "C:\Perl64\cpan\build\Crypt-OpenSSL-RSA-0.28-XdSoEu\-lssl32"
        "C:\Perl64\cpan\build\Crypt-OpenSSL-RSA-0.28-XdSoEu\libssl32.a"
        "C:\MinGW\x86_64-w64-mingw32\lib\libssl32.a"
        "C:\Perl64\lib\CORE\libssl32.a"
        "C:\Perl64\cpan\build\Crypt-OpenSSL-RSA-0.28-XdSoEu\libssl32.dll.a"
        "C:\MinGW\x86_64-w64-mingw32\lib\libssl32.dll.a"
        "C:\Perl64\lib\CORE\libssl32.dll.a"
        "C:\Perl64\cpan\build\Crypt-OpenSSL-RSA-0.28-XdSoEu\libssl32.lib"
        "C:\MinGW\x86_64-w64-mingw32\lib\libssl32.lib"
        "C:\Perl64\lib\CORE\libssl32.lib"
        "C:\Perl64\cpan\build\Crypt-OpenSSL-RSA-0.28-XdSoEu\ssl32.a"
        "C:\MinGW\x86_64-w64-mingw32\lib\ssl32.a"
        "C:\Perl64\lib\CORE\ssl32.a"
        "C:\Perl64\cpan\build\Crypt-OpenSSL-RSA-0.28-XdSoEu\ssl32.dll.a"
        "C:\MinGW\x86_64-w64-mingw32\lib\ssl32.dll.a"
        "C:\Perl64\lib\CORE\ssl32.dll.a"
        "C:\Perl64\cpan\build\Crypt-OpenSSL-RSA-0.28-XdSoEu\ssl32.lib"
as you can see - it searches libraries by names, which differs from that we got after compilcation. And also you can find "C:\Perl64\lib\CORE" ("C:\Perl64\lib\CORE" for x86-64) the best place to putting your libraries. I put it on these paths with these names:
    libssl.a    =>    C:\Perl64\lib\CORE\ssl32.a
    libcrypto.a    =>    C:\Perl64\lib\CORE\eay32.a
    libcrypto.a    =>    C:\Perl64\lib\CORE\libcrypto.a
+ copy this file (somewhy it searcher stuff like GetDeviceCaps):
    C:\Perl64\lib\CORE\libgdi32.a => C:\Perl64\lib\CORE\libgdi32.a
that's it.


dmake case

What if you have similar case, but it's more difficult?
For example - you have some dir with files in your C:\Perl\cpan\build or C:\Perl64\cpan\build directory - which stay here after unsuccessful installation of some cpan module. And you found way to fix makefile to build process finished correctly.
Then, to finish installation exec next commands:
> dmake
> dmake install


visual studio

Here are way to build with VisualStudio-compiled openssl.
First, when you put your lib files and try install some module, you will get:
        C:\Perl64\lib\CORE\libcrypto.a(tmp32/bf_cfb64.obj):(.text$mn+0x1e): undefined reference to `__chkstk'
        C:\Perl64\lib\CORE\libcrypto.a(tmp32/bf_skey.obj):(.text$mn+0x11): undefined reference to `__chkstk'
        C:\Perl64\lib\CORE\libcrypto.a(tmp32/bf_enc.obj):(.text$mn+0x1a): undefined reference to `__chkstk'
so, you need take
    C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\lib\chkstk.obj
for x86 or
    C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\lib\amd64\chkstk.obj
for x86-64.
put it somewhere in accessible place, for example
    C:\Perl64\lib\CORE\chkstk.a
and fix in makefile (in 'C:\Perl64\cpan\build'):

    LDLOADLIBS = C:\Perl64\lib\CORE\libcrypto.a C:\Perl64\lib\CORE\libgdi32.a
to
    LDLOADLIBS = C:\Perl64\lib\CORE\libcrypto.a C:\Perl64\lib\CORE\libgdi32.a C:\Perl64\lib\CORE\chkstk.a

then
> dmake
> dmake install
That's it. But I think way with mingw-compiled library is much simpler.


problem solving in a short

1. put includes from your mingw-compiled lib to:

    C:\Perl\site\lib\auto\MinGW\i686-w64-mingw32\include
or
    C:\Perl64\site\lib\auto\MinGW\x86_64-w64-mingw32\include

2. put your mingw-compiled lib to
    libssl.a    =>    C:\Perl64\lib\CORE\ssl32.a
    libcrypto.a    =>    C:\Perl64\lib\CORE\eay32.a
    libcrypto.a    =>    C:\Perl64\lib\CORE\libcrypto.a
    + copy
    C:\Perl64\lib\CORE\libgdi32.a => C:\Perl64\lib\CORE\libgdi32.a


useful sources & links
here are looks like another solution (but as I see - it's for strawberry perl) - [link]


APPENDIX - possible errors (for googling people):

main problem illustration - on example of 'cpan install Crypt::OpenSSL::SMIME'
------------------------------------------------------------------------------------------------------
Warning (mostly harmless): No library found for -lcrypto
Warning (mostly harmless): No library found for -lmoldname
Warning (mostly harmless): No library found for -lkernel32
Warning (mostly harmless): No library found for -luser32
Warning (mostly harmless): No library found for -lgdi32
Warning (mostly harmless): No library found for -lwinspool
Warning (mostly harmless): No library found for -lcomdlg32
Warning (mostly harmless): No library found for -ladvapi32
Warning (mostly harmless): No library found for -lshell32
Warning (mostly harmless): No library found for -lole32
Warning (mostly harmless): No library found for -loleaut32
Warning (mostly harmless): No library found for -lnetapi32
Warning (mostly harmless): No library found for -luuid
Warning (mostly harmless): No library found for -lws2_32
Warning (mostly harmless): No library found for -lmpr
Warning (mostly harmless): No library found for -lwinmm
Warning (mostly harmless): No library found for -lversion
Warning (mostly harmless): No library found for -lodbc32
Warning (mostly harmless): No library found for -lodbccp32
Warning (mostly harmless): No library found for -lcomctl32
Please specify prototyping behavior for SMIME.xs (see perlxs manual)
SMIME.xs: In function 'XS_Crypt__OpenSSL__SMIME_new':
SMIME.xs:232:3: warning: passing argument 3 of 'Perl_sv_2pv_flags' from incompatible pointer type [enabled by default]
C:\Perl64\lib\CORE/proto.h:3835:21: note: expected 'STRLEN * const' but argument is of type 'int *'
SMIME.xs:233:3: warning: passing argument 3 of 'Perl_sv_2pv_flags' from incompatible pointer type [enabled by default]
C:\Perl64\lib\CORE/proto.h:3835:21: note: expected 'STRLEN * const' but argument is of type 'int *'
SMIME.xs:234:3: warning: passing argument 3 of 'Perl_sv_2pv_flags' from incompatible pointer type [enabled by default]
C:\Perl64\lib\CORE/proto.h:3835:21: note: expected 'STRLEN * const' but argument is of type 'int *'
SMIME.xs:235:3: warning: passing argument 3 of 'Perl_sv_2pv_flags' from incompatible pointer type [enabled by default]
C:\Perl64\lib\CORE/proto.h:3835:21: note: expected 'STRLEN * const' but argument is of type 'int *'
SMIME.xs:236:3: warning: passing argument 3 of 'Perl_sv_2pv_flags' from incompatible pointer type [enabled by default]
C:\Perl64\lib\CORE/proto.h:3835:21: note: expected 'STRLEN * const' but argument is of type 'int *'
SMIME.xs:237:3: warning: passing argument 3 of 'Perl_sv_2pv_flags' from incompatible pointer type [enabled by default]
C:\Perl64\lib\CORE/proto.h:3835:21: note: expected 'STRLEN * const' but argument is of type 'int *'
SMIME.xs:238:3: warning: passing argument 3 of 'Perl_sv_2pv_flags' from incompatible pointer type [enabled by default]
C:\Perl64\lib\CORE/proto.h:3835:21: note: expected 'STRLEN * const' but argument is of type 'int *'
SMIME.xs:245:18: warning: assignment discards 'const' qualifier from pointer target type [enabled by default]
SMIME.o:SMIME.c:(.text+0x1a1): undefined reference to `BIO_new_file'
SMIME.o:SMIME.c:(.text+0x514): undefined reference to `X509_free'
SMIME.o:SMIME.c:(.text+0x520): undefined reference to `EVP_PKEY_free'
SMIME.o:SMIME.c:(.text+0x529): undefined reference to `X509_STORE_free'
SMIME.o:SMIME.c:(.text+0x535): undefined reference to `BIO_free'
SMIME.o:SMIME.c:(.text+0x61c): undefined reference to `X509_STORE_new'
SMIME.o:SMIME.c:(.text+0x62d): undefined reference to `X509_LOOKUP_file'
SMIME.o:SMIME.c:(.text+0x638): undefined reference to `X509_STORE_add_lookup'
SMIME.o:SMIME.c:(.text+0x660): undefined reference to `X509_LOOKUP_ctrl'
SMIME.o:SMIME.c:(.text+0x669): undefined reference to `X509_LOOKUP_hash_dir'
SMIME.o:SMIME.c:(.text+0x674): undefined reference to `X509_STORE_add_lookup'
SMIME.o:SMIME.c:(.text+0x693): undefined reference to `X509_LOOKUP_ctrl'
SMIME.o:SMIME.c:(.text+0x698): undefined reference to `ERR_clear_error'
SMIME.o:SMIME.c:(.text+0x6e7): undefined reference to `X509_STORE_free'
SMIME.o:SMIME.c:(.text+0x72e): undefined reference to `BIO_s_file'
SMIME.o:SMIME.c:(.text+0x736): undefined reference to `BIO_new'
SMIME.o:SMIME.c:(.text+0x754): undefined reference to `BIO_ctrl'
SMIME.o:SMIME.c:(.text+0x7b4): undefined reference to `BIO_free'
SMIME.o:SMIME.c:(.text+0x7d6): undefined reference to `d2i_PrivateKey_bio'
SMIME.o:SMIME.c:(.text+0x876): undefined reference to `d2i_PKCS12_bio'
SMIME.o:SMIME.c:(.text+0x895): undefined reference to `PKCS12_parse'
SMIME.o:SMIME.c:(.text+0x89d): undefined reference to `PKCS12_free'
SMIME.o:SMIME.c:(.text+0x905): undefined reference to `PEM_read_bio_PrivateKey'
SMIME.o:SMIME.c:(.text+0x938): undefined reference to `BIO_s_file'
SMIME.o:SMIME.c:(.text+0x940): undefined reference to `BIO_new'
SMIME.o:SMIME.c:(.text+0x967): undefined reference to `BIO_ctrl'
SMIME.o:SMIME.c:(.text+0x9d2): undefined reference to `BIO_free'
SMIME.o:SMIME.c:(.text+0x9ea): undefined reference to `d2i_PKCS12_bio'
SMIME.o:SMIME.c:(.text+0xa08): undefined reference to `PKCS12_parse'
SMIME.o:SMIME.c:(.text+0xa10): undefined reference to `PKCS12_free'
SMIME.o:SMIME.c:(.text+0xa4d): undefined reference to `BIO_ctrl'
SMIME.o:SMIME.c:(.text+0xa6c): undefined reference to `PEM_read_bio_X509_AUX'
SMIME.o:SMIME.c:(.text+0xd7b): undefined reference to `BIO_ctrl'
SMIME.o:SMIME.c:(.text+0xd9c): undefined reference to `PKCS7_sign'
SMIME.o:SMIME.c:(.text+0xdba): undefined reference to `BIO_ctrl'
SMIME.o:SMIME.c:(.text+0xdd5): undefined reference to `BIO_new_file'
SMIME.o:SMIME.c:(.text+0xdf7): undefined reference to `BIO_printf'
SMIME.o:SMIME.c:(.text+0xe0d): undefined reference to `BIO_printf'
SMIME.o:SMIME.c:(.text+0xe23): undefined reference to `BIO_printf'
SMIME.o:SMIME.c:(.text+0xe41): undefined reference to `SMIME_write_PKCS7'
SMIME.o:SMIME.c:(.text+0xe4d): undefined reference to `PKCS7_free'
SMIME.o:SMIME.c:(.text+0xe52): undefined reference to `sk_new_null'
SMIME.o:SMIME.c:(.text+0xe70): undefined reference to `BIO_ctrl'
SMIME.o:SMIME.c:(.text+0xe9e): undefined reference to `sk_push'
SMIME.o:SMIME.c:(.text+0xeb2): undefined reference to `BIO_new_file'
SMIME.o:SMIME.c:(.text+0xee1): undefined reference to `PKCS7_encrypt'
SMIME.o:SMIME.c:(.text+0xeff): undefined reference to `BIO_ctrl'
SMIME.o:SMIME.c:(.text+0xf15): undefined reference to `BIO_printf'
SMIME.o:SMIME.c:(.text+0xf2b): undefined reference to `BIO_printf'
SMIME.o:SMIME.c:(.text+0xf41): undefined reference to `BIO_printf'
SMIME.o:SMIME.c:(.text+0xf5f): undefined reference to `SMIME_write_PKCS7'
SMIME.o:SMIME.c:(.text+0xf6b): undefined reference to `PKCS7_free'
SMIME.o:SMIME.c:(.text+0xf79): undefined reference to `X509_free'
SMIME.o:SMIME.c:(.text+0xf7e): undefined reference to `sk_pop_free'
SMIME.o:SMIME.c:(.text+0xf87): undefined reference to `X509_free'
SMIME.o:SMIME.c:(.text+0xf93): undefined reference to `BIO_free'
SMIME.o:SMIME.c:(.text+0xf9f): undefined reference to `BIO_free'
SMIME.o:SMIME.c:(.text+0x1678): undefined reference to `OPENSSL_add_all_algorithms_noconf'
SMIME.o:SMIME.c:(.text+0x167d): undefined reference to `EVP_des_ede3_cbc'
c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.6.3/../../../../x86_64-w64-mingw32/bin/ld.exe: SMIME.o: bad reloc address 0x0 in section `.pdata'
c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.6.3/../../../../x86_64-w64-mingw32/bin/ld.exe: final link failed: Invalid operation
collect2: ld returned 1 exit status
dmake.exe:  Error code 129, while making 'blib\arch\auto\Crypt\OpenSSL\SMIME\SMIME.dll'
------------------------------------------------------------------------------------------------------
what to do: make actions mentioned in this note.

problem, when includes found, and library not found.
------------------------------------------------------------------------------------------------------
CPAN: Module::CoreList loaded ok (v5.20141002)
cp Blowfish.pm blib\lib\Crypt\OpenSSL\Blowfish.pm
Running Mkbootstrap for Crypt::OpenSSL::Blowfish ()
C:\Perl64\bin\perl.exe -MExtUtils::Command -e chmod -- 644 Blowfish.bs
C:\Perl64\bin\perl.exe C:\Perl64\lib\ExtUtils\xsubpp  -typemap C:\Perl64\lib\ExtUtils\typemap  Blowfish.xs > Blowfis
h.xsc && C:\Perl64\bin\perl.exe -MExtUtils::Command -e mv -- Blowfish.xsc Blowfish.c
C:\Perl64\site\bin\gcc.exe -c           -s -O2 -DWIN32 -DWIN64 -DCONSERVATIVE -DPERL_TEXTMODE_SCRIPTS -DUSE_SITECUST
OMIZE -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -fno-strict-aliasing -mms-bitfields -s -O2         -D
VERSION=\"0.02\"        -DXS_VERSION=\"0.02\"  "-IC:\Perl64\lib\CORE"  -DPERL5 -DOPENSSL_NO_KRB5 Blowfish.c
Blowfish.xs:5:30: fatal error: openssl/blowfish.h: No such file or directory
compilation terminated.
dmake.exe:  Error code 129, while making 'Blowfish.o'
  VKRAMSKIH/Crypt-OpenSSL-Blowfish-0.02.tar.gz
  C:\Perl64\site\bin\dmake.exe -- NOT OK
CPAN: YAML::XS loaded ok (v0.52)
Running make test
  Can't test without successful make
Running make install
  Make had returned bad status, install seems impossible
------------------------------------------------------------------------------------------------------
what to do: make actions mentioned in this note - after putting includes in accessible place.

build openssl 1.0.* on windows

problem

It's not trivial task - build OpenSSL library - especially if you are doing it first time.
By the way - OpenSSL has good documentation (and about building process on windows) - but nobody reads documentation.
So, here are my notes of that - mostly for myself, but also it can be useful for people, who find it in google.
It works on openssl 1.0.* versions. 1.1.* builds another way.


tools & versions

Will build OpenSSL by two compilers: Visual Studio 2013 and mingw.
Mingw-compiled version can be useful for binding with perl, for example.

• Openssl source code. For today, last version of openssl - openssl-1.0.2a. Will build it. Download it. [link]
• For Visual Studio compiling need perl - I'm using ActivePerl (v5.20.1.2000 x64). Download and install it. [link]
• Optional - for compilation with Visual Studio for x86 platform Netwide Assembler - nasm (I have v2.11.08). Download and install it - after intalling add path to nasm.exe in %PATH%. [link]
• For compiling with mingw you need msys (I have MSYS-20111123). Download and unpack it. [link]

lyrics

asm
In sources present assembler sources for x32 code. It makes x2-x4 better performance - link.
Earlier you could use masm, but as I see - from 1999 year it slowly disappears and now documentation says only nasm supported. Anyway - it's only for dying 32-bit mode.
If you don't want to install masm or you don't care about x2-x4 performance, you can compile it with 'no-asm' option (will be shown further) - in this case will be used C routines.

compilation time
Compilation with Visual Studio takes ~2 min.
Compilation with mingw takes ~16 min.

symbols
If you just build with visual studio by default - when you will use this library in your project, visual studio will print tons of annoying messages, like:
    1>libeay32.lib(cryptlib.obj) : warning LNK4099: PDB 'lib.pdb' was not found with 'libeay32.lib(cryptlib.obj)' or at 'D:\blablabla\Debug\lib.pdb'; linking object as if no debug info
so, if you also don't like this stuff - you have 2 paths to solve this problem:

• embed symbols into lib files
• copy symbols manually

embedding symbols
To embed symbols - after you generated nt.mak file (by command 'ms\do_ms', 'ms\do_nasm' or 'ms\do_win64a') but before you called nmake (described further) - open file 'ms\nt.mak' in any text editor and replace all (two) '/Zi' occurrences to '/Z7'.

copying symbols
After building (for example, you built in 'C:\mydir' directory), .pdb symbols will located into paths.

C:\mydir\out32	if you build release static library
C:\mydir\out32dll	if you build release dynamic library
C:\mydir\out32.dbg	if you build debug static library
C:\mydir\out32dll.dbg	if you build debug dynamic library

so, you can just copy symbols from this dir.

safeseh
And good news of SAFESEH - earlier on x86 you needed also fix 'ms\nt.mak' to use safeseh mechanism
(it makes your code more secure and get rid you from another annoying message from visual studio):
    1>libeay32.lib(x86cpuid.obj) : error LNK2026: module unsafe for SAFESEH image.
(http://eran.geek.co.il/wp/archives/3897 wrote about that)
but at the last version this warning disappeared (it seems this warning appeared on modules, compiled by masm). Well, anyway if you will get such warning - you know what to fix to get rid of it.

general principe of building in visual studio
First of all - you need to open visual studio command prompt.
as described here, for example - http://stackoverflow.com/questions/21476588/where-is-developer-command-prompt-for-vs2013
in "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\Tools\Shortcuts" shortcuts - "VS2013 x86 Native Tools Command Prompt" & "VS2013 x64 Native Tools Command Prompt"
You need open version, according to target platform.

Building process consists of 4 steps (and one optional step):

1. First of all you need call 'Configure' perl script. In parameters you should pass:
• target platform & build type:
• VC-WIN32            release x86
• debug-VC-WIN32        debug x86
• VC-WIN64A            release x86-64
• debug-VC-WIN64A        debug x86-64
• on x86 platform if you don't want use asm, you should pass 'no-asm'
• --prefix - it's destination path for builded files
For example: > perl Configure VC-WIN32 no-asm --prefix=c:/_openssl-1.0.2a_x86_release_dynamic_noasm
2. Second step - you need to generate nt.mak file. For that you need to exec one of next scripts:
• > ms\do_ms            for 'no-asm' on x86
• > ms\do_nasm        for x86 with asm
• > ms\do_win64a        for x86-64 target platform
Here are optional step - you can fix generated nt.mak file to add symbols (described above)
3. Third step - build. Here are two options:
• > nmake -f ms\ntdll.mak        to get dynamic libraries
• > nmake -f ms\nt.mak        to get static libraries
4. The last step - you should repeat previous line + add ' install' in the end of that. For example:
• > nmake -f ms\nt.mak install
This step just copied files into dest directory (which you entered in '--prefix' parameter)

Also I don't know how to make 'clear' on this building system - so, I've just removed everything in c:\mydif & unpacked here source files again. If you leave files from previous build with another parameters - it will lead to error during compilation.


action

Visual Studio 32-bit version

- unpack openssl-1.0.2a.tar.gz with any unpacker to C:\mydir (I'm using FAR manager for that)
- launch VS2013 x86 command prompt
> cd c:\mydir
then just choose what do you need, accordingly with general principles:

        ---------------------------------------------------
        # visual studio
        # x86
        # release
        # no-asm
        # dynamic
       
        > perl Configure VC-WIN32 no-asm --prefix=c:/_openssl-1.0.2a_x86_release_dynamic_noasm
        > ms\do_ms
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\ntdll.mak
        > nmake -f ms\ntdll.mak install
        ---------------------------------------------------
        # visual studio
        # x86
        # release
        # no-asm
        # static
       
        > perl Configure VC-WIN32 no-asm --prefix=c:/_openssl-1.0.2a_x86_release_static_noasm
        > ms\do_ms
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\nt.mak
        > nmake -f ms\nt.mak install
        ---------------------------------------------------
        # visual studio
        # x86
        # release
        # nasm
        # static
       
        > perl Configure VC-WIN32 --prefix=c:/_openssl-1.0.2a_x86_release_static_nasm
        > ms\do_nasm
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\nt.mak
        > nmake -f ms\nt.mak install
        ---------------------------------------------------
        # visual studio
        # x86
        # release
        # nasm
        # dynamic
       
        > perl Configure VC-WIN32 --prefix=c:/_openssl-1.0.2a_x86_release_dynamic_nasm
        > ms\do_nasm
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\ntdll.mak
        > nmake -f ms\ntdll.mak install
        ---------------------------------------------------

        ---------------------------------------------------
        # visual studio
        # x86
        # debug
        # no-asm
        # dynamic
       
        > perl Configure debug-VC-WIN32 no-asm --prefix=c:/_openssl-1.0.2a_x86_debug_dynamic_noasm
        > ms\do_ms
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\ntdll.mak
        > nmake -f ms\ntdll.mak install
        ---------------------------------------------------
        # visual studio
        # x86
        # debug
        # no-asm
        # static
       
        > perl Configure debug-VC-WIN32 no-asm --prefix=c:/_openssl-1.0.2a_x86_debug_static_noasm
        > ms\do_ms
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\nt.mak
        > nmake -f ms\nt.mak install
        ---------------------------------------------------
        # visual studio
        # x86
        # debug
        # nasm
        # static
       
        > perl Configure debug-VC-WIN32 --prefix=c:/_openssl-1.0.2a_x86_debug_static_nasm
        > ms\do_nasm
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\nt.mak
        > nmake -f ms\nt.mak install
        ---------------------------------------------------
        # visual studio
        # x86
        # debug
        # nasm
        # dynamic
       
        > perl Configure debug-VC-WIN32 --prefix=c:/_openssl-1.0.2a_x86_debug_dynamic_nasm
        > ms\do_nasm
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\ntdll.mak
        > nmake -f ms\ntdll.mak install
        ---------------------------------------------------

Visual Studio 64-bit version
- unpack openssl-1.0.2a.tar.gz with any unpacker to C:\mydir (I'm using FAR manager)
- launch VS2013 x64 command prompt (you can find it into 'start' menu)
> cd c:\mydir
then just choose what do you need, accordingly with general principles: 

        #---------------------------------------------------
        # visual studio
        # x64
        # release
        # static
       
        > perl Configure VC-WIN64A --prefix=C:\_openssl-1.0.2a_x64_release_static
        > ms\do_win64a
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\nt.mak
        > nmake -f ms\nt.mak install
        ---------------------------------------------------
        # visual studio
        # x64
        # release
        # dynamic
       
        > perl Configure VC-WIN64A --prefix=C:\_openssl-1.0.2a_x64_release_dynamic
        > ms\do_win64a
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\ntdll.mak
        > nmake -f ms\ntdll.mak install
        ---------------------------------------------------
        # visual studio
        # x64
        # debug
        # dynamic
       
        > perl Configure debug-VC-WIN64A --prefix=C:\_openssl-1.0.2a_x64_debug_dynamic
        > ms\do_win64a
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\ntdll.mak
        > nmake -f ms\ntdll.mak install
        ---------------------------------------------------
        # visual studio
        # x64
        # debug
        # static
       
        > perl Configure debug-VC-WIN64A --prefix=C:\_openssl-1.0.2a_x64_debug_static
        > ms\do_win64a
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\nt.mak
        > nmake -f ms\nt.mak install
        ---------------------------------------------------

mingw (here are considered only static mode)

Put openssl-1.0.2a.tar.gz to C:\mydir

Run "MSYS-20111123\msys\msys.bat"

> cd c:\mydir
> tar xf openssl-1.0.2a.tar.gz
That moment is very important - if you just unpack it by some another tool (I've used FAR manager first time) - it can handle wrong way symlinks and you will get error during compiling.
Actually on stackoverflow wrote that if you got bad symlinks - you can either fix some source files of tests or not build tests at all - but I think much simpler use unpacker from msys.
> cd ./openssl-1.0.2a

For x64:
    > perl Configure mingw64 no-shared no-asm --prefix=/C/OpenSSL-x64
For x32:
    > perl Configure mingw no-shared no-asm --prefix=/C/OpenSSL-x32
   
> make depend
> make
> make install


useful sources & links:

Documentation:
    https://github.com/openssl/openssl/blob/master/INSTALL.W32 (or just file 'INSTALL.W32' in openssl sources) for win32
    https://github.com/openssl/openssl/blob/master/INSTALL.W64 (or just file 'INSTALL.W64' in openssl sources) for win64
    https://wiki.openssl.org/index.php/Compilation_and_Installation (here are useful options table, by the way)
Here are a lot of useful advices, but part of them out-of-date, bcs of unsing unsupported masm:
    http://developer.covenanteyes.com/building-openssl-for-visual-studio/
Good explanation of mingw-building
    http://stackoverflow.com/questions/9379363/how-to-build-openssl-with-mingw-in-windows
Good advices about VisualStudio compiling and SAFESEH (somewhy he builds openssl with "no-idea" key (it means without IDEA algorithm))
    http://eran.geek.co.il/wp/archives/3897
Yet another typical guide, but here are described alternative way of dealing with symbols
    http://p-nand-q.com/programming/windows/building_openssl_with_visual_studio_2013.html
Here are something about zlib (I just don't need that):
    http://www.nu42.com/2014/04/building-openssl-101g-on-64-bit-windows.html
    http://stackoverflow.com/questions/23772816/when-do-i-need-zlib-in-openssl
Here are about runtime library:
    http://stackoverflow.com/questions/18486243/how-do-i-build-openssl-statically-linked-against-windows-runtime
Here are a bit more about mingw building:
    https://wiki.qt.io/Compiling-OpenSSL-with-MinGW
Here are (in the end) interesting idea - put openssl into visual studio internals, to not care about paths to includes & libraries every time (I solve it by property sheets, but this method is interesting)
    http://www.libtorrent.org/building.html
Here are somebody wrote scripts for building.
    http://www.michaelboman.org/how-to/building-openssl-on-windows
    http://www.lenholgate.com/blog/2012/08/building-openssl-for-x86-and-x64-on-windows-for-side-by-side-deployment.html
Here are yet another compiling problem - it seems somebody didn't turn off asm on x86
    http://stackoverflow.com/questions/15304327/building-openssl-vc-static-64bit-libs
ClamAV out-of-date building article (bcs of unsing unsupported masm). Nothing interesting. Just leave here for myself.
    http://blog.clamav.net/2014/07/compiling-openssl-for-windows.html
Yet another typical guide, but with free VisualStudio
    http://botsikas.blogspot.ru/2012/10/building-openssl-on-windows.html
Yet another typical guide, with some extra errors described
    http://wiki.openuru.org/index.php?title=Build_OpenSSL_with_MS_Visual_C%2B%2B
Yet another guide. From 2007. Maybe here will be something interesting.
    http://www.ski-epic.com/2007_notes_on_openssl/index.html
Yet another guide. Described running tests.
    http://www.poweradmin.com/blog/building-openssl-with-visual-studio-2012-for-32-or-64-bit/


possible errors (for googling people):

If you unpack archive for mingw with wrong handling of symlinks - you will get next message during compiling:
--------------------------------------------------------------------------------
make[1]: Entering directory `/c/mydir/test'
gcc -I.. -I../include  -DOPENSSL_THREADS -D_MT -DDSO_WIN32 -DL_ENDIAN -O3 -Wall
-DWIN32_LEAN_AND_MEAN -DUNICODE -D_UNICODE   -c -o md2test.o md2test.c
md2test.c:1:10: error: expected '=', ',', ';', 'asm' or '__attribute__' before '
.' token
 dummytest.c
          ^
make[1]: *** [md2test.o] Error 1
make[1]: Leaving directory `/c/mydir/test'
make: *** [build_tests] Error 1
--------------------------------------------------------------------------------
what to do: use tar from msys to unpacking archive


If you are trying to compile with visual studio but not from it's command prompt:
--------------------------------------------------------------------------------
'nmake' is not recognized as an internal or external command
--------------------------------------------------------------------------------
what to do: use visual studio command prompt


If you are trying to compile with mingw from cmd (or far manager)
--------------------------------------------------------------------------------
RC4_CHUNK is unsigned long long
e_os2.h => include/openssl/e_os2.h
"making $target in $dir..."
'TOP' is not recognized as an internal or external command,
operable program or batch file.
make: *** [links] Error 1
--------------------------------------------------------------------------------
what to do: use msys to compiling with mingw


If you are trying to compile with visual studio with masm
--------------------------------------------------------------------------------
Assembling: tmp32\sha1-586.asm
tmp32\sha1-586.asm(1432) : error A2070:invalid instruction operands
tmp32\sha1-586.asm(1576) : error A2070:invalid instruction operands
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\BIN\ml.EXE"' : return code '0x1'   
Stop.
--------------------------------------------------------------------------------
what to do: either use nasm, either no-asm