boost::filesystem::remove_all, RemoveDirectory WinAPI function and 'The directory is not empty' error

Problem

Sometime boost::filesystem::remove_all on windows (or RemoveDirectoryA/RemoveDirectoryW WinAPI functions, which are called into remove_all windows implementation) returns stupid error 'The directory is not empty'.

Problem solving in short

Windows have some problems with long paths (which length more than 260 symbols), and if you want to handle such paths, you need to write instead of C:\my_long_filename stuff like \\?\C:\my_long_filename.
If you have directory C:\dir where located file with long name, RemoveDirectory winapi function called with this path returns error 'The directory is not empty'. But if you call it with \\?\C:\dir parameter - it will work fine.
So, instead of using boost::filesystem::remove_all you can use something like that:

void RemoveAll(const std::wstring & path)
{
    std::wstring current_path = path;

    if (current_path.substr(0, 4) != L"\\\\?\\")
    {
        current_path = L"\\\\?\\" + current_path;
    }

    boost::filesystem::remove_all(current_path);
}

Useful links:

Rot13 in Windows:
https://blog.didierstevens.com/2006/07/24/rot13-is-used-in-windows-you%E2%80%99re-joking/

UserAssist - thing, which stores in registry (in Rot13) what did you run on your PC. And useful util for viewing:
https://blog.didierstevens.com/programs/userassist/

Undetectable windows payload generation (metasploit generates shellcode, then python code generated, who executed this shellcode, then it's aes encrypted & pack to the mzpe):
https://github.com/nccgroup/Winpayloads
(description of UAC bypass used by link (again IFileOperation) - https://www.pretentiousname.com/misc/W7E_Source/win7_uac_poc_details.html)

Masquerade-PEB powershell script (for UAC bypass):
https://github.com/FuzzySecurity/PowerShell-Suite/blob/master/Masquerade-PEB.ps1
with interesting idea:

one more UAC bypass:
https://github.com/FuzzySecurity/PowerShell-Suite/tree/master/Bypass-UAC

Masquerade-PEB uses NtQueryInformationProcess to get a handle to powershell's
PEB. From there itreplaces a number of UNICODE_STRING structs in memory to
give powershell the appearance of a different process. Specifically, the
function will overwrite powershell's "ImagePathName" & "CommandLine" in
_RTL_USER_PROCESS_PARAMETERS and the "FullDllName" & "BaseDllName" in the
_LDR_DATA_TABLE_ENTRY linked list.

This can be useful as it would fool any Windows work-flows which rely solely
on the Process Status API to check process identity. A practical example would
be the IFileOperation COM Object which can perform an elevated file copy if it
thinks powershell is really explorer.exe ;)!

Interesting case, how to run stuff in context of InstallUtil.exe from win dir! It can be god damn autorun, for example:
http://www.blackhillsinfosec.com/?p=4881

How to run console program with parameters, when cmd.exe disabled:
http://www.blackhillsinfosec.com/?p=5257

some interesting tool:
https://github.com/goldshtn/etrace

CVE-2016-3308 - corrupt heap in win32k
https://github.com/55-AA/CVE-2016-3308

blind sql framework
http://www.darknet.org.uk/2016/09/bbqsql-blind-sql-injection-framework/

LLMNR/NBNS spoofer:
https://github.com/Kevin-Robertson/Inveigh

notes of yara 3.5.0 compiling

It's modified version of my 'notes of yara 3.4.0 compiling' post

 

What's good in 3.5.0 in comparison with 3.4.0

Official description is quite short, so I watched commits:

• speed up into 2.6x times! - https://twitter.com/plusvic/status/763753320381046784
• bugfixes: ~70 bugfixes - some of them led to crashes - I personally occured crashes bcs of 2 bugs, which is fixed by now
• new stuff:
• length operator ! (don't know who will really use it)
• useful stuff in pe module:
• imports(dll_name)
• imports(dll_name, ordinal)
• is_dll()
• is_3bit()
• is_64bit()
• 2 new functions in 'rich_signature' in 'pe' module:
• version(version, [toolid])
• toolid(toolid, [version])

Also in my yara module I will need to change stuff like:
    foreach_memory_block(context, block)
to
    foreach_memory_block(iterator, block)
and declare this iterator before, and change the way how to deal with this stuff - now need to write smth like 'block_data = block->fetch_data(block);'

and also struct _YR_MATCH changed - match->length became match->match_length so I'd need to fix it in my module source accordingly.

tools & version:

• windows 8.1
• visual studio 2013
• yara library 3.5.0

foreword

Just notes of yara compiling process on windows with visual studio. I will compile without CUCKOO support - bcs I don't need this.

action

1) unpack archive

2) go to yara-3.5.0\windows\lib and delete all these libraries. I prefer to compile everything what I need by myself. And these libraries will interfere with libraries which I will compile.

3) open solution: .\yara-3.5.0\windows\vs2010\yara.sln

4) open 'utils.h' -> replace '#define YR_API EXTERNC __declspec(dllexport)' to '#define YR_API EXTERNC' (bcs I don't like exported symbols in my exe files, and link I wanna statically)

5) choose platform & mode

6) set runtime library for all projects (yara & yarac & libyara):
    properties -> c/c++ -> code generation -> runtime library -> /MTd for debug or /MT for release
    (you can select several projects in time - using 'ctrl'+left_mouse_button_click)

7) add to "Preprocessor Definitions" of 'libyara' project
    (Properties -> Configuration Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions)
    lines to not conflict with mysql c connector, for example:
strlcat=libyara_internal_strlcat

8) open 'strutils.c' and replace '#if !HAVE_STRLCAT && !defined(strlcat)' to '#if !HAVE_STRLCAT', open 'strutils.h' and make the same.

9) Go to libyara properties:
    Properties -> Configuration Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions
    and delete CUCKOO from this list.
Then go to libyara properties:
    Properties -> Configuration Properties -> Librarian -> General -> Additional Dependencies
    and delete jansson64.lib from this list.

10) Here are you must choose - you want to compile it with openssl or without.
Why do you need openssl in yara library:
    - Generate an import hash: https://www.mandiant.com/blog/tracking-malware-import-hashing/ (uses define HAVE_LIBCRYPTO)
    - PE module of yara can extract some info from pe digital signature certificate. (uses define HAVE_LIBCRYPTO)
#if defined(HAVE_LIBCRYPTO)
begin_struct_array("signatures");
  declare_string("issuer");
  declare_string("subject");
  declare_integer("version");
  declare_string("algorithm");
  declare_string("serial");
  declare_integer("not_before");
  declare_integer("not_after");
  declare_function("valid_on", "i", "i", valid_on);
  end_struct_array("signatures");
declare_integer("number_of_signatures");
#endif   
    - HASH module of yara can calc provide you cryptographic hash functions: md5, sha1, sha256, checksum32 (uses define HASH, appeared in 3.3.0 version)
   
If you need some of this functionality - you need to build openssl & you need add for all projects:
    - Additional library directory
    - library file of openssl (libeay32.lib on my pc)
    - add HASH_MODULE to preprocessor of libyara project

If you don't need this functionality
    - delete HAVE_LIBCRYPTO from "Preprocessor Definitions" of libyara, and insert HAVE_TIMEGM line - else you get undefined type 'tm'.
        Properties -> Configuration Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions
    - delete libeay64.lib from
        Properties -> Configuration Properties -> Librarian -> General -> Additional Dependencies

11) If you need add your module - you need add it to 'libyara\modules\module_list'

After that everything will compiles fine.

build libzippp in visual studio 2013

Here are some c++ wrapper over libzip - https://github.com/ctabin/libzippp
But it has terrible building system

• in it hardcoded version of visual studio (well, it's fixable)
• in it hardcoded version of libzip (hardcoded version: 1.1.2, last version: 1.1.3 - well, and it's fixable)
• it's difficult to change building script to adjust zlib & libzip - for example for static linking.

So, it's much easier to create empty project (god bless this project have one cpp & one h file (and one more cpp for tests - author has tests - that's really cool)). So, just create solution with 2 projects - libzippp & tests. Adjust headers, path to headers, lib files, path to lib files, change /MD to /MT & /MTd accordingly and build. Check if tests works fine. Done)

--------------------------------------------------------------------
and for using it as static library, go to libzippp.h, add #include <cstdint> and change:

#ifdef WIN32
        typedef long long libzippp_int64;
        typedef unsigned long long libzippp_uint64;
       
        //special declarations for windows to use libzippp from a DLL
        #define SHARED_LIBRARY_EXPORT __declspec(dllexport)
        #define SHARED_LIBRARY_IMPORT __declspec(dllimport)
#else
        //standard ISO c++ does not support long long
        typedef long int libzippp_int64;
        typedef unsigned long int libzippp_uint64;
       
        #define SHARED_LIBRARY_EXPORT
        #define SHARED_LIBRARY_IMPORT
#endif

to

typedef int64_t libzippp_int64;
typedef uint64_t libzippp_uint64;

#define SHARED_LIBRARY_EXPORT
#define SHARED_LIBRARY_IMPORT
--------------------------------------------------------------------


headers paths for libzippp:
D:\projects\libraries\libzip-1.1.3\lib
D:\projects\libraries\libzip-1.1.3\xcode

headers paths for tests:
D:\projects\libraries\libzippp\libzippp

lib paths for debug tests:
D:\projects\libraries\libzippp\x64\Debug
D:\projects\libraries\libzip-1.1.3\_build_x64_static_mt_mtd\lib\Debug
D:\projects\libraries\zlib-1.2.8\_libraries_debug

lib paths for release tests:
D:\projects\libraries\libzippp\x64\Release
D:\projects\libraries\libzip-1.1.3\_build_x64_static_mt_mtd\lib\Release
D:\projects\libraries\zlib-1.2.8\_libraries_release

lib files:
libzippp.lib
zipstatic.lib
zlibstat.lib

And respect for the author of libzippp - despite of bad building system, I hope project will be useful)
 

building libzip in visual studio 2013

As always - x64, static, debug/release.
You need compiled zlib - I wrote of compiling zlib here
Download from http://www.nih.at/libzip/index.html archive libzip-1.1.3.tar.gz, unpack.

md _build_x64_static_mt_mtd
cd _build_x64_static_mtd
if you want to use only static lib

• go to 'D:\projects\libraries\libzip-1.1.3\lib\CMakeLists.txt' & comment pre-last block & uncomment last block.

• go to D:\projects\libraries\libzip-1.1.3\lib\zip.h and insert into the beginning (after include guard): #define ZIP_STATIC 

• go to D:\projects\libraries\libzip-1.1.3\lib\compat.h and replace '#define ZIP_EXTERN __declspec(dllexport)' -> '#define ZIP_EXTERN'

cmake .. -G "Visual Studio 12 2013 Win64" -DZLIB_LIBRARY:FILEPATH="D:/projects/libraries/zlib-1.2.8/_libraries_debug/zlib.lib" -DZLIB_INCLUDE_DIR:PATH="D:/projects/libraries/zlib-1.2.8"
debug: Md -> MTd, build
release: Md -> MT, build

the simplest nginx cfg for sharing files

just memo - how to share directory

worker_processes  1;
error_log ./logs/error_log.log;
events {
    worker_connections 1024;
}

http {
    server {
      listen 80;
      server_name myvhost;
      access_log ./logs/access_log.log;

      location / {
        root D:/shared_dir/;
        autoindex on;
      }
    }
}

errno values link

Useful link with description of values 'errno' variable on linux - [link]

how to launch debugger when specific process start

Found way to attach automatically debugger when specific process launched:
link
and in parameter 'debugger' you can set  fullpath to ollydbg and it will work - for example: D:\tools\odbg110\OLLYDBG.EXE

asciihex to hex in perl

Found great script on stackoverflow:

my $str = <>;
$str =~ s/(..)/chr(hex($1))/eg;
print($str);

Enable telnet client on windows

Somewhy telnet client not present in windows by default (well, once I've found it present by default in one server windows, but only once).
Here are algorithm - how to install in by standard tools - link.
backuped page:

C++ on windows and linux

Once I've encounted with problem - size of some variable on linux x86-64 differs from size of same variable on windows x86-64. And some years after that I couldn't remember - was it int or long or what. So here are link about it.
backuped article:

and here are another angry post about same problem)
backuped post:

So, problem in 'long' data type.

earlier I've used special macros to get DWORD == 4 bytes on x86 windows, x86-64 windows, x86 linux & x86-64 linux:
#if defined ( _MSC_VER )
    typedef unsigned long DWORD;
#else // for g++
    typedef unsigned int DWORD;
#endif

but now I would prefer _int32_t

Download file on windows by standard tools

Found interesting way - how to download file on windows only with standard tools (like how to make wget/curl on windows by standard tools) - [link]
and backuped article:

It can be useful, when you need do stuff like that and you don't add extra dependencies.

Sort file by lines lengths

Often I need to sort file by length of lines and always I can't remember code which is doing this.

my @elements;

while(my $line = <>){
    push @elements, $line;
}

@sorted = sort { length $a <=> length $b } @elements;

foreach my $l (@sorted){
   print($l);
}

OpenPGP and annoying pinentry window

Foreword

I've started to use PGP in jabber (GnuPG for windows - Gpg4win - I've used this instruction). Backup of instruction just in case:

 

Problem

And every time when I've got incoming message in jabber - appeared windows 'pinentry' and asked me password (passphrase). It's very annoying and in the internet I didn't find solution for Windows OS.
Window - looks like that:

Lyrics

So, in the internet there are lot of posts where people advices create file with properties - 'gpg-agent.conf', but usually it's about linux. Process monitor showed that in Windows this file expected to be in "C:\Users\username\AppData\Roaming\gnupg\gpg-agent.conf"

Action

1. Create file "C:\Users\username\AppData\Roaming\gnupg\gpg-agent.conf"
2. Write in this file 2 lines (values can be any big number - it's seconds of caching your password):
• max-cache-ttl 2592000
• default-cache-ttl 2592000
3. Restart your gpg-agent.exe process