+- Hack Forums (https://hackforums.net)
+-- Forum: Tech (https://hackforums.net/forumdisplay.php?fid=444)
+--- Forum: Coding (https://hackforums.net/forumdisplay.php?fid=151)
+---- Forum: Software Development (https://hackforums.net/forumdisplay.php?fid=118)
+---- Thread: [C] [Advanced] [C] Writing a Metamorphic Engine / Real USG; Part 1 (/showthread.php?tid=5675912)
[Advanced] [C] Writing a Metamorphic Engine / Real USG; Part 1 - movsx - Jul 22, 2017
Introduction
This is the first of a several part tutorial series which will teach you how to write a metamorphic engine / real unique stub generator which will take a subset of the C source language and output unique assembly code. This tutorial is the easiest in the series, each tutorial will be significantly harder than the last. So if you're having difficult following now, I recommend you go back and study some basic computer science and C coding. If this tutorial is popular, I'll release the second one. After releasing all tutorials, I'll release the full source code on github. This tutorial isn't meant to spoonfeed you, only guide you. Don't know what an abstract syntax tree is? Google it and find out, then come back.
Please note that the purpose of these tutorials aren't for malware, nor for you to copy paste and use in your own projects. The purpose of this series is to demonstrate that metamorphic engines are a real threat to computer security, and hopefully motivate individuals to try and find ways to stop them.
This series is not for the faint of heart, as this simple C subset metamorpic engine spans over 15,000 lines of C code and requires a deep understanding of the C language to follow, as well as you being willing to learn the compiler theory concepts mentioned here and study them well. 15,000 lines of C code is a tiny project, in all reality, but some of you may not be used to projects of this size, nor the actual complexity of the code involved, as each phase of the engine is drastically different from the last, and will begin to use some complex algorithms and data structures.
Drawing from Compiler Theory
A metamorphic engine is essentially a compiler, except its goal is not to compile efficient or small code, but to compile code that is unique and well-functioning. Consequently, there is a significant overlap with writing compilers and metamorphic engines, so the front-end of our engine will essentially be the same as writing a front-end for a compiler. The backend is where metamorphic engines differ from compilers, as you will see in later tutorials.
Brief Discussion of Compiler Frontends
- A source language is input to the front end
- First the front end breaks this input into a sequence of tokens.
- Second the front end generates a parse tree from these tokens
- Third the front end uses syntax directed translation to convert the parse tree into an abstract syntax tree
- Fourth the front end creates a symbol table from the AST
- Fifth the front end runs the AST through the semantic analysis stage to determine if the grammar is semantically valid: proper use of types, not using vars out of scope, not using undeclared vars, etc, etc
- Sixth the front end runs the AST through another syntax directed translator to generate an intermediate representation language, which is then fed into the back end to have code mutation applied to it. This is where the majority of effort/code is at: Control flow modifications, junk code generation, instruction selection, instruction scheduling, register allocation, and function inlining / outlining occur here.
Let's get started! Front end: Stage 1; Lexing
Honestly there's not much to say. The lexer is incredibly easy to implement. I won't even bother discussing it. Play around with lexers, write your own, and understand why they're important.
Here's the code for the lexer:
linkedlist.h https://pastebin.com/x556r8DL
linkedlist.c https://pastebin.com/DV058wt3
lexer.h https://pastebin.com/cdxwAdmL
lexer.c https://pastebin.com/Emu3rpzx
Here's an example C block
Code
{
int x;
x = 0;
while (x < 10) {
x = x + 1;
}
}And here's the output of the lexer
Code
[{] [INT] [Identifier x] [;] [Identifier x] [=] [Int Literal=0] [;] [WHILE ] [(] [Identifier x] [<] [Int Literal=10] [)] [{] [Identifier x] [=] [Identifier x] [+] [Int Literal=1] [;] [}] [}]The next tutorial (if this ones popular) will be on recursive descent parsing and will cover generating parse trees -> abstract syntax trees
* - Malicious Intent - Jul 22, 2017
Interesting. Looking forward to next parts of this tutorial. Not often one comes across something like this presented in readable and compressed form, giving just enough to start off and allowing one to dive deeper into less understood parts by encouraging own research..
* - maldev0 - Jul 22, 2017
I'm sure this will get very interesting. Especially the prediction of 15k lines of code. Good work, keep it up!
Kind regards
* - movsx - Jul 22, 2017
(Jul 22, 2017 02:06 PM)maldev0 Wrote: I'm sure this will get very interesting. Especially the prediction of 15k lines of code. Good work, keep it up!
Kind regards
It's not a prediction.
* - ForceJew - Jul 22, 2017
Looks interesting waiting to read the next posts!
* - TouchMyMalware - Jul 22, 2017
(Jul 22, 2017 05:50 PM)FORC3FI3LD Wrote: Looks interesting waiting to read the next posts!
There won't be any. He's just outputting some basic code without a finished product. Trust me, he'l never finish this series.
* - movsx - Jul 22, 2017
(Jul 22, 2017 06:02 PM)TouchMyMalware Wrote: There won't be any. He's just outputting some basic code without a finished product. Trust me, he'l never finish this series.
Wait and see buddy. All I see you do is talk shit towards other people. I've never seen you actually produce anything yourself. Lmao.
* - TouchMyMalware - Jul 22, 2017
(Jul 22, 2017 06:54 PM)Yani Wrote: So what should we be discussing?
This makes absolutely no sense.
It's because he's full of himself. He thinks literally everyone else is wrong and he's right. He also claims to be 23 years old with three bachelor degrees, and an IQ of 140+..No joke.
He thinks because I haven't released anything on HF that I lack any knowledge of programming or computer science. When in reality I couldn't care less to prove myself to that filth. I just like making him look stupid.
* - movsx - Jul 22, 2017
(Jul 22, 2017 06:54 PM)Yani Wrote: So what should we be discussing?
This makes absolutely no sense.
There is a lot to discuss about lexers. For instance. What happens if the user inputs an invalid C program. Does the lexer care about that? What additional information can we propagate to later stages? What do we currently propagate?
How will the lexer be used in the parser?
Try to roll your own lexer, in any language, and think about the next steps.
(Jul 22, 2017 06:59 PM)Yani Wrote: That one's great too..
Source code will be released every tutorial, but it's only meant to be used as reference, not copy pasted. If you note, I never mentioned how to actually use the code either.
(Jul 22, 2017 06:56 PM)TouchMyMalware Wrote: It's because he's full of himself. He thinks literally everyone else is wrong and he's right. He also claims to be 23 years old with three bachelor degrees, and an IQ of 140+..No joke.
He thinks because I haven't released anything on HF that I lack any knowledge of programming or computer science. When in reality I couldn't care less to prove myself to that filth. I just like making him look stupid.
No, you're the arrogant one, constantly coming to my threads and talking shit. Every time I've talked to you on discord about computer science or coding in general (after you seemingly call me out and accuse me of knowing nothing, with no provocation whatsoever), you've proved to know absolutely jack shit beyond the complete fundamentals you'd read in a "Learn C in 12 hours" book.
I'm not going to argue with a kid who doesn't even know what a TLB is. Please kindly leave my thread and stop trolling, unless you wish to discuss the topic at hand.
* - TouchMyMalware - Jul 22, 2017
(Jul 22, 2017 07:01 PM)movsx Wrote: I'm not going to argue with a kid who doesn't even know what a TLB is. Please kindly leave my thread and stop trolling, unless you wish to discuss the topic at hand.
You need to learn to read mate.
* - movsx - Jul 22, 2017
(Jul 22, 2017 07:13 PM)Yani Wrote: I already played around with lexers, and was quite successful. I was trying to make a runtime interpreter, much like PHP. But gave up as I had more rewarding projects going on.
Anyways, I'm not only talking about the discussion of lexers, but also this "tutorial". I can discuss the code of this lexer, I can can discuss metamorphic engines, but really, I can't discuss this "tutorial" because there is none.
There isn't really a way to put this information as a "tutorial". In my eyes you'd better be off writing a paper on it.
There wouldn't be a possible action/branch to take when it reaches the tokens that are invalid. So it can't do anything else than care.
But that's for part 2 of your "tutorial".... We first have to have this useless discussion while we wait for the next parts...
If you're not familiar with octal, get so, as here is a counterexample:
int i;
i = 012399;
The lexer would be responsible for reporting this error, namely, when it runs into the literal 012399 because it is an invalid octal value.
Not only this, there's quite a bit more too. But I leave the discussion to those commenting here.
I'm doing this for a reason. Without fundamentally understanding these "useless parts" there is no metamorphic engine. It's a combination of many pieces, the lexer being one of them.
* - movsx - Jul 22, 2017
(Jul 22, 2017 07:20 PM)Yani Wrote: Then just release all parts together, or write a paper.
Stop posting parts like these to feed your ego.
The purpose is to teach the community. Not feed my ego. If I just release the metamorphic engine, nobody will learn anything.
* - POLY MATH - Jul 22, 2017
Great tutorial but I must say that releasing one part at a time is bad practice around HF.
Try releasing 2-3 Part clusters.
* - movsx - Jul 22, 2017
(Jul 22, 2017 07:24 PM)Yani Wrote: I thought you meant stuff like syntax errors.
Your example doesn't really prove a point tho because you could handle that use-case however you want.
There's actually no other place you would handle it
(Jul 22, 2017 07:24 PM)TRADOC Wrote: Great tutorial but I must say that releasing one part at a time is bad practice around HF.
Try releasing 2-3 Part clusters.
I don't think that is true for metamorphic engines. Parsing is a very heavy topic and introducing it at the same time as lexing is a bad idea.
Believe me when I say that other tutorials will be much more involved, likely 5 times the length as this one.
I also plan on releasing the tutorials at given time intervals, not all at once. Otherwise people wouldn't be able to learn anything.
(Jul 22, 2017 07:24 PM)Yani Wrote: But you're not learning anybody anything. You just give them the info you already know.
Don't call it a tutorial if this isn't one, and this isn't one, this is merely a part of it.
I respect this knowledge but please write something like a paper instead of this super short non-tutorial thread.
Or release all of them together idc.
As stated above, I acknowledge this tutorial is short. It's an introduction coupled with lexing. The other tutorials will easily be 5-10x the length of this one, and much more heavily involved. That's just due to their nature.
* - TouchMyMalware - Jul 22, 2017
(Jul 22, 2017 07:23 PM)movsx Wrote: The purpose is to teach the community. Not feed my ego. If I just release the metamorphic engine, nobody will learn anything.
You need to learn to read mate.