Meditation, The Art of Exploitation

Thinking? At last I have discovered it--thought; this alone is inseparable from me. I am, I exist--that is certain. But for how long? For as long as I am thinking. For it could be, that were I totally to cease from thinking, I should totally cease to exist....I am, then, in the strict sense only a thing that thinks.

Thursday, February 07, 2008

Compilers Part 5, working with in memory data buffers

In the previous entries, we were able to set up the spec for the scripting language. To port the interpreter into a tcp/ip server, the first task is to allow the lexer to work with in memory data buffers instead of stdin and stdout. The reason is simple, user input will come from a network client and there are subtle differences between a network socket and stdin.

Fortunately, flex provides several interface to set up in memory data buffer as token input. The following lex source code demonstrates how to use the relevant interface:

extern "C"{
#include < sys/stat.h>
#include < fcntl.h>
#include < string.h>
#include < iostream>
#include < sstream>
#include < fstream>
#include < string>
#include < vector>
#include < algorithm>
using namespace std;

unsigned int line = 0;
std::vector< std::string> text;


extern int yywrap();

\/\/.*$ { cout << "comment: " << yytext; }
.|\n ;

YY_BUFFER_STATE cur_buffer;
int main(int argc, char * argv[]){

cout << argv[1] << '\n';
ifstream ifs(argv[1]);

char buf[256];
int len;
memset(buf, 0, 256);
ifs.getline(buf, 254);
len = strlen(buf);
buf[len] = '\n';
cout << buf;
cout << "\nlines read: " << text.size() << '\n';

cur_buffer = yy_scan_string(text[line].c_str());
extern int yylex();

return 0;

int yywrap(){

if(line+1 > text.size()) return 1;
cur_buffer = yy_scan_string(text[line].c_str());
line ++;
return 0;

yywrap gets called by yylex whenever a input buffer is exhausted, if yywrap returns 1, yylex will return; Therefore, it's a common technique to set up another available data buffer and return 0 to allow yylex continue processing as done in this example.