[yuumi]

I see a lot of posts regarding what people should do after learning the basics of C or asking questions regarding pointers and structures and I figured this would be a fun challenge to try.


Implement a virtual machine which supports five registers: r0,r1,r2,r3,rax
An instruction pointer register: rip

Six operational instructions:
move constant to a register
move register to register
add constant to a register
add register to register
subtract constant from a register
subtract register from a register

and the halt instruction which stops the vm execution.

the semantics of the instructions should be:
Instruction destination source
Eg:
Move rax 20
or
Move rax, r0
Where destination is always a register source can be constant or register .

and each instruction is 3 bytes.

When you've completed it, you can push this bytecode through your VM to test

Code

uint8_t bytecode[] = {
    kMovConst,  kR0,    10,
    kAddConst,  kR0,    5,
    kMovConst,  kR1,    3,
    kSubReg,    kR0,    kR1,
    kMovConst,  kR1,    4,
    kAddReg,    kR0,    kR1,
    kMovReg,    kR1,    kR0,
    kSubConst,  kR1,    7,
    kMovReg,    kRax,   kR1,
    kHalt
};

You can use these enums

Code

enum vm_op : uint8_t{
    kR0,
    kR1,
    kR2,
    kR3,
    kRax,
    kHalt,
    kMovReg,
    kMovConst,
    kAddReg,
    kAddConst,
    kSubReg,
    kSubConst,
};

[movss]

Implement a virtual machine that doesn't use the instruction cycle, and instead uses threaded coding to execute handlers pertaining to the machine's ISA.

An example would include:

mov eax, ecx
>
VM
>
tons of x86_64 opcodes that are responsible for handling your VMs opcodes.

This would be utterly confusing to the reader, and in doing so would have to reverse each "handler" and figure out what they are doing.
With dead code obfuscation, control flow obfuscation, opaque predicates this process becomes non trivial.

But easily defeated with LLVM + MCSEMA

[yuumi]

Implement a virtual machine that doesn't use the instruction cycle, and instead uses threaded coding to execute handlers pertaining to the machine's ISA.

An example would include:

mov eax, ecx
>
VM
>
tons of x86_64 opcodes that are responsible for handling your VMs opcodes.

This would be utterly confusing to the reader, and in doing so would have to reverse each "handler" and figure out what they are doing.
With dead code obfuscation, control flow obfuscation, opaque predicates this process becomes non trivial.

But easily defeated with LLVM + MCSEMA

It isn't meant to be an obfuscation challenge. It has nothing to do with manipulating the .text section at runtime and running the executable code through a separate handler which maps VM operations to actual instruction. It's a simple C based VM like a bytecode emulator.

I'm not asking a beginner in C to write an entire obfuscation virtual machine.

[movss]

It isn't meant to be an obfuscation challenge. It has nothing to do with manipulating the .text section at runtime and running the executable code through a separate handler which maps VM operations to actual instruction. It's a simple C based VM like a bytecode emulator.

I'm not asking a beginner in C to write an entire obfuscation virtual machine.

doesn't pertain to obfuscation
obfuscation is an additive obscurity measure for the aforementioned VM design

using handlers that are threaded is far more a better learning experience than a trivial instruction cycle

[yuumi]

doesn't pertain to obfuscation
obfuscation is an additive obscurity measure for the aforementioned VM design

using handlers that are threaded is far more a better learning experience than a trivial instruction cycle

The point is it's meant to be a trivial instruction cycle. If people want to do it they can.
If they'd rather automate an entire function produce bytecode then compile byte code to be called and translate each instruction through a handler while the .text section of that function looks complete gibberish sure they can do that too. It wouldn't make sense to do that with what I asked, since you'd need to map almost the entire instruction set to a virtual instruction set and you'd probably need some disassembler engine to automate the process.

[EmacsSucksButILikeIt]

I'm a bit out of touch with C and C++ but isn't the below C++ and this is a challenge for C.

Code

enum vm_op : uint8_t{
    kR0,
    kR1,
    kRax,
    kRdx,
    kHalt,
    kMovReg,
    kMovConst,
    kAddReg,
    kAddConst,
    kSubReg,
    kSubConst,
};

[movss]

I'm a bit out of touch with C and C++ but isn't the below C++ and this is a challenge for C.

Code

enum vm_op : uint8_t{
    kR0,
    kR1,
    kRax,
    kRdx,
    kHalt,
    kMovReg,
    kMovConst,
    kAddReg,
    kAddConst,
    kSubReg,
    kSubConst,
};

semantics retard
get the fuck out of this thread

[EmacsSucksButILikeIt]

semantics retard
get the fuck out of this thread

You understand this is for people that have a basic understanding of C and probably programming.... You shouldn't be posting confusing and conflicting information in your challenge.

Nice challenge BTW.

[yuumi]

I'm a bit out of touch with C and C++ but isn't the below C++ and this is a challenge for C.

Code

enum vm_op : uint8_t{
    kR0,
    kR1,
    kRax,
    kRdx,
    kHalt,
    kMovReg,
    kMovConst,
    kAddReg,
    kAddConst,
    kSubReg,
    kSubConst,
};

It's a C challenge. I guess you can use C++ but yes it's just semantics. You don't have to use those specific opcodes or even use enums.

[movss]

You understand this is for people that have a basic understanding of C and probably programming.... You shouldn't be posting confusing and conflicting information in your challenge.

Nice challenge BTW.

bluh bluh bluh

😎

[EmacsSucksButILikeIt]

Might try this latter today.

The return value register should be rax

Does this mean you expect to support function calls?

Also... Which direction are you moving your registers. Are you using AT&T or Intel style moves.

[yuumi]

Might try this latter today.


Does this mean you expect to support function calls?

Also... Which direction are you moving your registers. Are you using AT&T or Intel style moves.

It’s intel syntax
Semantic:
Instruction destination source

No it doesn't have to support function calls the result in rax portion was just something I threw in the bytecode example. You can consider that irrelevant.

Maybe in the next challenge I can make it slightly more complicated by having a virtual stack, as well as a stack register rsp and an actual calling convention.

I made some changes to the original design of the vm: have a look.

[EmacsSucksButILikeIt]

It’s intel syntax
Semantic:
Instruction destination source

No it doesn't have to support function calls the result in rax portion was just something I threw in the bytecode example. You can consider that irrelevant.

I'll use it(Rax) as an exit code for the byte code.

---

This is so easy to implement in a functional language(OCaml)... In C its a bit of a mess.

Code

type instr_reg = R0 | R1 | Rax | Rdx

type op_codes =
    MovConstReg of (int * instr_reg)
  | MovRegReg of (instr_reg * instr_reg)
  | AddConstReg of (int * instr_reg)
  | AddRegReg of (instr_reg * instr_reg)
  | SubConstReg of (int * instr_reg)
  | SubRegReg of (instr_reg * instr_reg)
  | Halt

let byte_code = [
  MovConstReg(10, R0);
  AddConstReg(5, R0);
  MovConstReg(3, Rax);
  SubRegReg(R0, Rax);
  MovConstReg(4, Rax);
  AddRegReg(R0, Rax);
  MovRegReg(Rax, R0);
  SubConstReg(7, Rax);
  Halt;
]

let process_byte_code bc =
  let rax = ref 0 in
  let rdx = ref 0 in
  let r0 = ref 0 in
  let r1 = ref 0 in
  let rec get_reg r =
    match r with
    | Rax -> rax
    | Rdx -> rdx
    | R0 -> r0
    | R1 -> r1 in
  let rec process_byte_code_aux bc =
    match bc with
    | [] -> failwith "Invalid byte code sequence"
    | hd::tl ->
      match hd with
      | Halt ->
        (
          Printf.printf "rax %d, rdx %d, r0 %d, r1 %d\n" !rax !rdx !r0 !r1;
          !rax
        )
      | MovConstReg (c, r) ->
        (
          (get_reg r) := c;
          process_byte_code_aux tl
        )
      | MovRegReg (r1, r2) ->
        (
          (get_reg r1) := !(get_reg r2);
          process_byte_code_aux tl
        )
      | AddConstReg (c, r) ->
          (
            (get_reg r) := !(get_reg r) + c;
            process_byte_code_aux tl
          )
      | SubRegReg (r1, r2) ->
        (
          (get_reg r1) := !(get_reg r1) - !(get_reg r2);
          process_byte_code_aux tl
        )
      | AddRegReg(r1, r2) ->
        (
          (get_reg r1) := !(get_reg r1)+ !(get_reg r2);
          process_byte_code_aux tl
        )
      | SubConstReg(c, r) ->
        (
          (get_reg r) := !(get_reg r) - c;
          process_byte_code_aux tl
        ) in
  process_byte_code_aux bc

let () =
  print_endline(string_of_int (process_byte_code byte_code))

The final register values. I think I got all the moves right.

rax 9, rdx 0, r0 16, r1 0

I'll see if I can get a working C version soon.

[yuumi]

I'll use it(Rax) as an exit code for the byte code.

---

This is so easy to implement in a functional language(OCaml)... In C its a bit of a mess.

Code

type instr_reg = R0 | R1 | Rax | Rdx

type op_codes =
    MovConstReg of (int * instr_reg)
  | MovRegReg of (instr_reg * instr_reg)
  | AddConstReg of (int * instr_reg)
  | AddRegReg of (instr_reg * instr_reg)
  | SubConstReg of (int * instr_reg)
  | SubRegReg of (instr_reg * instr_reg)
  | Halt

let byte_code = [
  MovConstReg(10, R0);
  AddConstReg(5, R0);
  MovConstReg(3, Rax);
  SubRegReg(R0, Rax);
  MovConstReg(4, Rax);
  AddRegReg(R0, Rax);
  MovRegReg(Rax, R0);
  SubConstReg(7, Rax);
  Halt;
]

let process_byte_code bc =
  let rax = ref 0 in
  let rdx = ref 0 in
  let r0 = ref 0 in
  let r1 = ref 0 in
  let rec get_reg r =
    match r with
    | Rax -> rax
    | Rdx -> rdx
    | R0 -> r0
    | R1 -> r1 in
  let rec process_byte_code_aux bc =
    match bc with
    | [] -> failwith "Invalid byte code sequence"
    | hd::tl ->
      match hd with
      | Halt ->
        (
          Printf.printf "rax %d, rdx %d, r0 %d, r1 %d\n" !rax !rdx !r0 !r1;
          !rax
        )
      | MovConstReg (c, r) ->
        (
          (get_reg r) := c;
          process_byte_code_aux tl
        )
      | MovRegReg (r1, r2) ->
        (
          (get_reg r1) := !(get_reg r2);
          process_byte_code_aux tl
        )
      | AddConstReg (c, r) ->
          (
            (get_reg r) := !(get_reg r) + c;
            process_byte_code_aux tl
          )
      | SubRegReg (r1, r2) ->
        (
          (get_reg r1) := !(get_reg r1) - !(get_reg r2);
          process_byte_code_aux tl
        )
      | AddRegReg(r1, r2) ->
        (
          (get_reg r1) := !(get_reg r1)+ !(get_reg r2);
          process_byte_code_aux tl
        )
      | SubConstReg(c, r) ->
        (
          (get_reg r) := !(get_reg r) - c;
          process_byte_code_aux tl
        ) in
  process_byte_code_aux bc

let () =
  print_endline(string_of_int (process_byte_code byte_code))

The final register values. I think I got all the moves right.

I'll see if I can get a working C version soon.

I did it in C which was relatively easy but that could just be because of my experience with C. Just to make it compatible to future changes , you can get rid of the rdx, and rcx register. Instead we'll just use registers r0,r1,r2,r3 and rax

[0x9F]

Did this in around 40 minutes (dont mind the dirt)

enum/struct defs:

Code

enum vm_reg : unsigned char {
    eR0,
    eR1,
    eRAX,
    eRDX,
};

enum vm_op : unsigned char {
    eHALT,
    eMOVER,
    eMOVEC,
    eADDR,
    eADDC,
    eSUBR,
    eSUBC,
};

typedef struct reg_container_t {
    DWORD R0;
    DWORD R1;
    DWORD RAX;
    DWORD RDX;
    DWORD RIP;
}reg_container;

mini-vm.c including execute/handler:

Code

#define MAX_OPCODE_STACK_SIZE 64

struct reg_container_t* vm_registers = NULL;

void handle(vm_op operation, BYTE reg, BYTE v);

DWORD execute(const BYTE* data)
{
BYTE* base = (BYTE*)data;
vm_registers->RIP = 0;

do
{
vm_registers->RIP++;

BYTE op = *data++;
BYTE reg = *data++;
BYTE v = *data++;

handle((vm_op)op, reg, v);

BYTE next = *(base + (3 * vm_registers->RIP));

if (next == eHALT)
break;

} while (vm_registers->RIP < MAX_OPCODE_STACK_SIZE);

return vm_registers->RAX;
}

void handle(vm_op operation, BYTE reg, BYTE v)
{
DWORD* dest = (DWORD*)((DWORD)vm_registers + (sizeof(DWORD) * reg));

switch (operation)
{
case eMOVEC:
*dest = v;
break;

case eMOVER:
*dest = *(DWORD*)((DWORD)vm_registers + (sizeof(DWORD) * v));
break;

case eADDC:
*dest += v;
break;

case eADDR:
*dest += *(DWORD*)((DWORD)vm_registers + (sizeof(DWORD) * v));
break;

case eSUBC:
*dest -= v;
break;

case eSUBR:
*dest -= *(DWORD*)((DWORD)vm_registers + (sizeof(DWORD) * v));
break;

default:
printf("Unknown instruction: 0x%x\r\n", operation);
break;
}
}

void initalize_vm()
{
dispose_vm();
vm_registers = (reg_container_t*)malloc(sizeof(reg_container_t));
ZeroMemory(vm_registers, sizeof(reg_container_t));
}

void dispose_vm()
{
if (vm_registers != NULL)
{
free(vm_registers);
vm_registers = NULL;
}
}

example usage:

Code

int main()
{
BYTE bytecode[] = {
eMOVEC,  eR0,    10,
eADDC,  eR0,    5,
eMOVEC,  eRAX,  3,
eSUBR,    eR0,    eRAX,
eMOVEC,  eRAX,  4,
eADDR,    eR0,    eRAX,
eMOVER,    eRAX,  eR0,
eSUBC,  eRAX,  7,
eHALT
};

initalize_vm();

DWORD result = execute(bytecode);

printf("rax: %d (0x%x)\r\n", result, result);

dispose_vm();

return 0;
}

Haven't tested this extensively and probably forgot small things but you get the point