Noteworthy i386 listing of frequently used subroutines and Typical function call protocol

Originally composed on June 13, 2004, editted formatting.

1. String length calculation and key comparison
mov ecx, -1  ; mov ecx, xx (xx=max possible length)
mov al, 0  ;xor eax, eax;sub eax, eax;mov eax, 0
mov edi, string_offset ; lea edi, [ebp+XX]
  ; ES:EDI -> string
repnz scasb     ; repnz scasw
neg ecx         ; xor eax, eax
    sub eax, ecx
mov edi, real string   ; lea edi, [ebp+XX]
mov esi, user string
repnz cmpsb     ; repeat till ecx = 0 or [edi]!=[esi]
test ecx, ecx
jnz  bad
xor eax, eax    ; using eax=0 as good, could also use eax=1 as bad
jmp  good
bad:
mov eax, 1
ret
good
do_good_stuff   ; update registry, update ini file, update memory
ret

2. A==0? 1:0 translated to assembly
mov eax, A
neg eax  ; eax = 0 - eax, unsigned, but sets CF if eax > 0
sbb eax, eax    ; eax = eax - eax - CF
inc eax
ret
if A == 0,
 neg eax  => eax = 0, CF = 0
 sbb eax, eax  => eax = 0
 inc eax  => eax = 1
else
 neg eax  => eax = (UnSigned)-A, CF = 1
 sbb eax, eax => eax = -1
 inc eax  => eax = 0
endif

An alternative form A == 0? 0:1
mov eax, A
cmp eax, 1      ; set CF if eax = 0, CF = 0 if A != 0
sbb eax, eax ; eax = -1 if A = 0, eax = 0 if A != 0
inc eax  ; eax = 0 if A = 0, eax = 1 if A != 0
ret

A little bit mind boggling, isn't it? :-)
These are common code signatures you will find in assembly language
generated from high level language (typically C/C++) from a compiler.
A final note, the counterpart of sbb is adc (add with carry). As an

excersise, build A == 0? 1:0 using abc instead of sbb.

3. The __cdecl,  __stdcall, __fastcall, thiscall signatures and
__declspec(naked)
(Reference: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vclang/html/_core___stdcall.asp)

Normally a subroutine written in C/C++ is translated with the following
signatures, take
int sub_a(arg1, arg2, arg3)
call sub_a(arg1, arg2, arg3)

there is 4 ways to declare its prototype that will affect generated
assembly code.

3.1)
__cdecl:
This is the default calling convention for C and C++ programs.
Because the stack is cleaned up by the caller, it can do vararg functions.
The __cdecl calling convention creates larger executables than __stdcall,
because it requires each function call to include stack cleanup code.
The following list shows the implementation of this calling convention.

Element Implementation
Argument-passing order Right to left
Stack-maintenance responsibility Calling function pops the arguments from
the stack Name-decoration convention Underscore character (_) is prefixed to
names Case-translation convention No case translation performed

So what does all this babblying mean when it's in action?
Callee:
int __cdecl sub_a(arg1, arg2, arg3)
 push ebp
 mov  ebp, esp
 sub  esp, 4 x number_of_local_automatic_variables <-- normally the case    
; pointers could complicate it, and structure alignment requirement  
; could also make the space required on stack seem strange.  
; nevertheless, automatica variables are stored on stack.  
; name it NOLAV: # of local automatic variables  
arg1 = [ebp+8] ; Here we assume it's a near call (*flat memory model)  
arg2 = [ebp+C] ; within the caller's own code segment linear space  
arg3 = [ebp+10] ; normally true but not for some trickery code   ...  
add esp, 4 x NOLAV 
; ebp+10 <- arg3   || stack high   pop ebp  
; ebp+0C <- arg2   ||   ret   
; ebp+8  <- arg1   ||    
; ebp+4  <- eip of the return address ||    
; ebp    <- previous ebp  ||            
        <= esp = ebp pointer  \/ stack low

Caller: call sub_a(arg1, arg2, arg3)  
push arg3 ; see below  
push arg2  
push arg1  
call sub_a  
add esp, 0C  

now it's of course simplified because arg3 cannot be directly referrenced in
assembly language. Very often it's something like this:  
mov eax, [ebp+XX]     ; passed on argument to this subroutine  
push eax
or  
mov eax, [ebp-XX]     ; a LAV: local automatic variable 
push eax   
push XX  ; a direct constant  
mov eax, [XXXXXXXX]       ; a global variable  
push eax 

I hope this clears up a lot of confusions around the myth around a subroutine
call in assembly language. Now we briefly describe __stdcall, __fastcall,
and thiscall 

3.2) __stdcall: The __stdcall calling convention is used to call Win32 API
functions. The callee cleans the stack, so the compiler makes vararg functions
__cdecl. Functions that use this calling convention require a function prototype. 
return-type __stdcall function-name[(argument-list)] The following list shows the
implementation of this calling convention.  Element Implementation  Argument-passing
order Right to left.  Argument-passing convention By value, unless a pointer or
reference type is passed.  Stack-maintenance responsibility Called function pops
its own arguments from the stack.  Name-decoration convention An underscore (_)
is prefixed to the name. The name is followed by the at sign (@) followed by the
number of bytes (in decimal) in the argument list. Therefore, the function declared
as int func( int a, double b ) is decorated as follows: _func@12  Case-translation
convention None  

Callee:
int __stdcall sub_a(arg1, arg2, arg3)  
push ebp  
mov  ebp, esp  
sub  esp, 4 x number_of_local_automatic_variables <-- normally the case    
; pointers could complicate it, and structure alignment requirement  
; could also make the space required on stack seem strange.  
; nevertheless, automatica variables are stored on stack.  
; name it NOLAV: # of local automatic variables  
arg1 = [ebp+8] ; Here we assume it's a near call (*flat memory model)  
arg2 = [ebp+C] ; within the caller's own code segment linear space  
arg3 = [ebp+10] ; normally true but not for some trickery code   ...  
add esp, 4 x NOLAV ; ebp+10 <- arg3  || stack high  
pop ebp  ; ebp+0C <- arg2   ||  
ret 0C  ; ebp+8  <- arg1   ||    
  ; ebp+4  <- eip of the return address ||    
  ; ebp    <- previous ebp  ||            
   <= esp = ebp pointer  \/ stack low  

The difference here is 'ret 0C' instead of 'ret' because in __stdcall the callee
is responsible to clean up the stack. 

Caller: call sub_a(arg1, arg2, arg3)  
push arg3 ; see below  
push arg2  
push arg1  
call sub_a    

Here after call sub_a, the caller code needs not to worry about  stack clean up.
There are 2 important points I want to bring up about __stdcall. First, WINAPI
is __stdcall so they use this convention. This is the most frequently encountered
form on a windows platform. Second, __stdcall will mangle the subroutine name
which will cause trouble if you try to link against __stdcall subroutine.
Unless you are working in a consistent setting, try to avoid __stdcall
declaration.  3.3) __fastcall The __fastcall calling convention specifies that
arguments to functions are to be passed in registers, when possible. The following
list shows the implementation of this calling convention.  Element Implementation 
Argument-passing order The first two DWORD or smaller arguments are passed in ECX
and EDX registers; all other arguments are passed right to left.  Stack-maintenance
responsibility Called function pops the arguments from the stack.  Name-decoration
convention At sign (@) is prefixed to names; an at sign followed by the number of
bytes (in decimal) in the parameter list is suffixed to names.  Case-translation
convention No case translation performed.   Callee: int __stdcall sub_a(arg1, arg2,
arg3)  

push ebp  
mov  ebp, esp  
sub  esp, 4 x  NOLAV   arg1 = ecx  ; Here we assume it's a near call (*flat memory model)  
arg2 = edx  ; within the caller's own code segment linear space  
arg3 = [ebp+8] ; normally true but not for some trickery code   ...  
add esp, 4 x NOLAV   
pop ebp    
ret 04  ; ebp+8  <- arg1   ||    
  ; ebp+4  <- eip of the return address ||    
  ; ebp    <- previous ebp  ||            
   <= esp = ebp pointer  \/ stack low  

The difference here is 'ret 04' instead of 'ret' in __cdecl or 'ret 0C' in __stdcall
because in __fastcall passes 2 arguments using registers and the callee is
responsible to clean up the rest of the stack. 

Caller:

call sub_a(arg1, arg2, arg3)  
push arg3 ; see below  
mov edx, arg2  
mov ecx, arg1  
call sub_a    

__fastcall is similar to __stdcall except when callee has less than 3 arguments,
no stack reference is needed for arguments so execution speed is improved. Like
__stdcall, subroutine names are also mangled in library.  3.4) thiscall The
__fastcall calling convention specifies that arguments to functions are to be
passed in registers, when possible. The following list shows the implementation of
this calling convention.  Element Implementation  Argument-passing order The first
two DWORD or smaller arguments are passed in ECX and EDX registers; all other
arguments are passed right to left.  Stack-maintenance responsibility Called
function pops the arguments from the stack.  Name-decoration convention At
sign (@) is prefixed to names; an at sign followed by the number of bytes
(in decimal) in the parameter list is suffixed to names.  Case-translation
convention No case translation performed.  

Callee: int __stdcall sub_a(arg1, arg2, arg3)  
push ebp  
mov  ebp, esp  
sub  esp, 4 x  NOLAV  
arg1 = [ebp+10] ; Here we assume it's a near call (*flat memory model)  
arg2 = [ebp+0C] ; within the caller's own code segment linear space  
arg3 = [ebp+8] ; normally true but not for some trickery code   this_pointer = ecx   ...  
add esp, 4 x NOLAV   
pop ebp    
ret 0C  ; ebp+8  <- arg1   ||    
  ; ebp+4  <- eip of the return address ||    
  ; ebp    <- previous ebp  ||            
   <= esp = ebp pointer  \/ stack low  

Pretty much like __stdcall except that the caller secretly puts the "this" pointer
into ecx before it calls sub_a. 

Caller: call sub_a(arg1, arg2, arg3)  
push arg3 ; see below  
push arg2  
push arg1  
mov ecx, this_pointer  
call sub_a    

Pretty much like __stdcall except that the caller secretly puts the "this" pointer
into ecx before it calls sub_a. 

3.5) __cdeclspec(naked) 
Callee: __cdeclspec(naked) int sub_a(arg1, arg2, arg3){  
__asm{    
       do what ever but don't blow it up!  
    }
}
Unlike any of the above declaration decorations, this one is special that it
will not setup the base stack frame pointer (ebp) for the callee. The callee
sub_a will normally be written in assembly code and it's entirely upon sub_a to
not blow up anything! The complier trusts that sub_a knows what it is doing.  On
the caller side, nothing special needs to be done. Although it's not unusual a
chain of naked subroutines are constructed to achieve some specific goal. 

3.6) The default VC behavior when working with .C source code is __cdecl  but
without the _ prefix. When working with .CPP source code, the function name is
automatically mangled unless prefixed with extern "C".  

__declspec(dllexport) int a(int x, int y, int z){  
 int b = x+10;  
 return b+y+z;
}

.text:10001000                
public a .text:10001000 a              
proc near .text:10001000 
.text:10001000 var_4           = dword ptr -4
.text:10001000 arg_0           = dword ptr  8
.text:10001000 arg_4           = dword ptr  0Ch
.text:10001000 arg_8           = dword ptr  10h
.text:10001000 
.text:10001000                 push    ebp
.text:10001001                 mov     ebp, esp
.text:10001003                 push    ecx
.text:10001004                 mov     eax, [ebp+arg_0]
.text:10001007                 add     eax, 0Ah
.text:1000100A                 mov     [ebp+var_4], eax
.text:1000100D                 mov     eax, [ebp+var_4]
.text:10001010                 add     eax, [ebp+arg_4]
.text:10001013                 add     eax, [ebp+arg_8]
.text:10001016                 mov     esp, ebp
.text:10001018                 pop     ebp
.text:10001019                 retn
.text:10001019 a               endp 

Table of name generation
.C __declspec(dllexport) int a    -> a
.C __declspec(dllexport) int __cdecl a   -> a
.C __declspec(dllexport) int __stdcall a  -> _a@12
.CPP __declspec(dllexport) int a  -> ?a@@YAHHH@Z
.CPP extern "C" __declspec(dllexport) int a -> a
.CPP extern "C" __declspec(dllexport) int __stdcall a -> _a@12

So be careful when you name your files. .C and .CPP produce
drastically different function tables and be sure to know what
you want and name your source code files accordingly.