btl extension
-- for jedi masters
Michael Taylor

This document describes the advanced features of the btl simulator. Please refer
to the introductory document for basic usage.

Motivation

Debugging applications running on raw is going to be incredibly difficult. Even a
minimalist 16 node raw application has orders of magnitude more state than the
standard processor. To be effective, the user needs to be able to easily select exactly
which state they want to see and then specify the way in which they want to display
that information. For instance, having 32 separate instruction windows cluttering
the desktop is not a viable solution. On the other hand, "focus"ing on each individual
processor is prohibitively labor intensive. Even a simple operation such as clearing all
of the registers so that one can easily view the progress of the computation involves
tremendous effort.

Extensible debuggers

The solution we use is an extensible debugger. The idea is to provide a
debugger framework that allows every aspect of the debugger to be quickly and easily
configured to match the particular debugging problem at hand. Normally, to add a feature
like "clear all registers," the user would have to modify the simulator itself. This creates
inordinate source control and merging problems. Furthermore, the simulator has to be
recompiled. An extensible debugger, however, allows the users to add this functionality
without recompiling the application.

Introducing: btl

btl has three parts: the simulator, an interpreter ("angstroc") and extension code. The simulator
is compiled C++ code that simulates the Raw architecture.  angstroc is a bC (an extension language
that is defined as being "more and more like C")  interpreter. All of the debugging,  user-interface
and devices (ie drams, pci, serial roms) support is written in bC. This allows
the end user to customize the simulator in many ways.

btl's extension architecture

At startup time, btl loads a loads a customization file named ".bug".
btl checks for the first .bug file in the following locations:

 

./.bug	program specific features
~/.bug	user specific customizations
<.bug>	the standard customizations - hooks in the code for all of the disassembly,  breakpointing, stepping, and other functionality (the brackets indicate a path relative to the location of btl executable) includes many of the files located in: <bug/*.bc>        code specifically for the btl user and debugging interface <system/*.bc>  standard library code for the angstroc language

After the code in that file is executed, bug calls the function main_loop which is the main
event loop for the debugger. (If main_loop has not been defined by the .bug file, a default loop is used.)
To take away the mystery, what follows is sample .bug file code for a simple main_loop.

fn main_loop()
{
  local storeSpace;

  storeSpace = malloc(kProgramSize);
  verify(storeSpace != NULL,
        "Out of memory in main_loop");

  while(1)
  {
    if (gets(storeSpace) == NULL)
         return 0;
    ExecuteTextAsFunction(storeSpace,kStackSize);
  }
}

The bolded functions are functions and identifiers that angstroc has dynamically linked against.
gets and malloc are direct from libc. ExecuteTextAsFunction, a native function inside
angstroc, takes a string and interprets within a sub-session of the interpreter.

Since main_loop is specified in the interpreted language, the basic interaction of the user and
the debugger can be easily changed. Perhaps you want to have the debugger print out the current
pc(s)  -- it is easily changed:

 while(1)
  {
    local i;

    for (i = 0; i < numproc; i++)
        printf("%8.8x ", Proc_GetPC(Machine_GetProc(machine,i));

    printf(">");

    if (gets(storeSpace) == NULL)
         return 0;
    ExecuteTextAsFunction(storeSpace,kStackSize);
  }

numproc, Proc_GetPC, Machine_GetProc and machine are identifiers
dynamically linked in from the simulator. You can find these and other useful functions
just by examining the .h files.

Another Example

Since all of the raw group members are prolific coders, it makes sense that we will want to
divide our debugger extensions into separate source files. It would be useful to have a facility
similar to the C "include" feature. This function can be found in <system/nativelink.bc>

fn _FileToString(filename)
{
    local theFile;
    local theString, stringLen = 0;

    theFile = fopen(filename,"r");
    if (theFile == NULL)
    {
        theString = malloc(1);
        verify(theString != NULL, "No memory");
    }
    else
    {
        fseek(theFile,0,2);
        stringLen = ftell(theFile);
        fseek(theFile,0,0);
        theString = malloc(stringLen + 1);
        verify(theString != NULL, "No memory");
        fread(theString,1,stringLen,theFile);
        fclose(theFile);
    }
   
    theFile = theString+stringLen;
    SetByte(theFile,0);
    return theString;
}

fn include(filename)
{
    local theCode;
    local length;

    theCode = _FileToString(filename);
    length = strlen(theCode);

    if (strlen(theCode) == 0)
    {
       printf("Cannot include file: %s\n", filename);
       free(theCode);
       return 0;
    }

   ExecuteTextAsFunction(theCode,32768);
   free(theCode);
   return 1;
}

Now, we can include other source files simply with the function call: include("/usr/uns/bug/lib/util.bc");

The bC language

For some good examples of hardcore bC usage, refer to: <dev/serial_rom.bc> and <dev/basic.bc>.
Use the help function, tab completion and recursive grep of the btl/user/ directory to see what sort
of functions you can call and hook into the simulator with.

The name bC is an abbreviation for "before C." The idea is, that since it only has one type,
unsigned 32 bit int, it is a bit more primitive than C. It also has a few extensions which
make it better suited as an interpretive language:

The syntax of bC is like C with a few exceptions:

0. There are no gotos.
1. There are no types and no arrays. - however, there is the builtin hashmap primitive for implementing structures.
2. a ? b : c does not exist.
3. a[i] is semantic sugar for *(a+i*4).
4. a[i][j] is semantic sugar for (a[i])[j]
3c. bC is cooperatively multithreaded. The yield command gives time back to the other threads.
4. Instead of structures, use string-to-integer hash_maps (from the C++ STL), which can be nested and recursive:

local serialRomStruct = hms_new();         /* create a new hash_map */
serialRomStruct.wordLength = 7;
serialRomStruct.foo = hms_new();           /* nested hash map */
serialRomStruct.foo.bar = serialRomStruct; /* recursive, nested hash map */

or, equivalently,

local serialRomStruct = hms_new();
serialRomStruct.wordLength = 7;
serialRomStruct.foo.bar = serialRomStruct;  /* autocreation hashmap foo */

There are also a variety of functions that can be called on string-to-int hash maps:
(these and other functions are declared in btl/system/nativelink.bc of the distribution)

is_hms(serialRomStruct);                    // detect if it is a hashmap

hms_walk_begin(serialRomStruct);            // begin iterating through fields of hashmap
hms_walk_at_end(serialRomStruct);           // have we walked through all fields
hms_walk_current_key(serialRomStruct);      // current field we are looking at
hms_walk_current_val(serialRomStruct);      // value of current field we are looking at

hms_print(serialRomStruct);                 // print out contents of hashmap
hms_lookup(serialRomStruct);                // lookup a value in a hashmap
hms_lookup_or_add(serialRomStruct);         // lookup a value in a hashmap (adds if not there)
get_sorted_hms_keys(serialRomStruct);       // returns a vector of pointers to field names, sorted
hms_size(serialRomStruct);                  // find number of fields in hashmap

hms_remove(serialRomStruct,"wordLength");   // remove an element from a hashmap
hms_clear(serialRomStruct);                 // clear a hash map;
hms_free(serialRomStruct);                  // frees a hashmap

if (hms_contains_key(serialRomStruct,"wordLength")) // check to see if the hashmap contains a field 
if (serialRomStruct.?wordLength)                    // same as above

Use the listi family of functions for maintaining and traversing lists.
Use the veci family of functions for maintaining and traversing automatically resizing vectors.

Global variables can be declared almost anywhere:global myGlobal;
In some cases, one may want to include a file that defines a global variable,
and then use that variable. In these cases, there is a problem because the variable
has not been defined at the time of parsing. In order to get around this problem,
one can use the function get_variable_val("gMyGlobalVarName"); this defers binding of the variable to runtime.
This function takes the name of the global variable as a char * and returns its value.

Functions also can be declared almost anywhere. Furthermore, function overloading is supported.
Linking is performed once, when the function is first called. Currently, trying to redefine a function
after it's been called is a bad idea.The old function will continue to be called.
Native functions can be dynamically linked against. However, they must be registered using the
NativeFunctionLink(funName,numParam) call which specifies the number of parameters the
function takes (-1 if variable.) See <bug/natives.bc> and <system/nativelink.bc> for a list of native functions
that have been linked in. Feel free to add any entries that might be useful.

fn foo()
{
   local i;
   i = bar() + bar(bar());
   fn bar()      { return 7; }
   fn bar(inOne) { return 6; }
   global k;
   return i;
} 

The symbol table is manipulatable through calls within the angstroc interpreter.

A C like language was chosen to minimize the number of different languages involved in the
raw project. TCL was considered as an extension language but its syntax is overbearing
and actually is not as versatile when it comes to interacting directly with C structures and functions.

  HINT:

(setq auto-mode-alist (cons '("\\.bc\\'" . c++-mode) auto-mode-alist))
(setq auto-mode-alist (cons '("\\.bug\\'" . c++-mode) auto-mode-alist))
will make emacs editing of .bc files easier.

BC supports binary, hex, and decimal constants, and allows '_' inside constants for readability:

global num  = 0x1234_5678;                               // hex value    0x12345678
global num2 = 0b1111_0000_1111_0000_1111_0000_1111_0000; // binary value 0xF0F0F0F0
global num3 = 99_990_000;                                // decimal value 99,990,000
global num4 = 0hFFaa_aaFF;                               // hex value ffaaaaff;

BC supports function pointers:

global device = hms_new();
device.output = &printf(x,x,x,x);   // creates a function pointer to printf, using four parameters
device.output("the answer is %x + %x = %x\n",1,2,3); 

BC supports anonymous functions:

device.calc = & fn(x,y,z) { ? x; ? y; ? z; };       // creates a function that takes
                                                    // three parameters and prints them out.

device.calc(1,2,3); 

BC supports anonymous functions with closures:

device.create_error_function = & fn(x) { return & fn(y) { printf(x,y); }; };
device.my_error_function     = device.create_error_function("There's an error: %x\n");

device.my_error_function(0x23);

 

Adding hardware devices to btl

Btl takes a machine description file as parameter. This machine description file
is responsible for setting up the machine (instruction and data memory sizes, boot rom contents) and
adding any devices on the periphery of the chip. In order to add your own hardware devices on the
periphery of the chip, you need to create your own machine description file. See
 http://cag.lcs.mit.edu/lxr/source/btl/user/bug/devices.bc for the functions which are used to send and
receive data from the Raw periphery I/O ports.

.bc devices can be used without modification on the Raw RTL model and the actual Raw processor motherboard.
(Unless the Raw processor is clocked very slowly, the .bc device will not be able to inject
data as fast as the Raw processor can consume it. Currently only the left-most Raw processor
ports can be attached to .bc devices, although with some work it is probably feasible to expose them all.)

The <dev/*.bc> (located in your source tree at btl/user/dev/) contains all of the bc code that
describes the machine configurations and devices that we use for raw. The standard <dev/basic.bc>
file contains options to enable pci, a boot_rom, a print_service, and four DRAMS on the right
hand side of the chip. These can be selectively enable via the BTL-ARGS parameter in the starsearch
Makefile. The <dev/serial_rom.bc>  describes a serial rom device, which can be used to stream
an arbitrary sequence of words into one of the I/O ports.

In order to create a test program that uses your own custom device, create a file that includes
the <dev/basic.bc> device and then add your own devices afterwards. For instance, see
starsearch/module_tests/dynamic/funny/dev-dev/disciplined/common/4x4.bc.
(The disciplined test is an example of a relatively sophisticated use of the starsearch infrastructure;
a simpler example is in_and_out .) You will also have to set BTL-ARGS in your starsearch makefile
so that it specifies the correct arguments to btl to get your devices running.

To interrupt the simulator as it is executing (for instance, upon error), set the global variable
gInterrupted to 1.

To mark the end of the cycle, use the yield primitive.

To see where in the device code the simulator currently is, use SimListDevices() to list all of the devices.
This command prints the thread number next to each device. Then use VMShowPositionOfThread(thread_number);
This prints a stack trace which includes the line number of the instruction after the last yield that was executed.
The other number printed by SimListDevices is the hms parameter that was passed to the device.
Doing a ? 0x<PARAM> will print out both the field names and the current field values of the hms.
This is a convenient way for printing out the state of the device.

The Raw ports are numbered clockwise, starting with the North port of the northwest tile (tile 0)
Thus, the North and West ports of tile 0 are ports 0 and 15, respectively.

See memo 05 for more details on extensions to btl.

The bC language implementation

Every bC command is parsed via yacc and compiled into bytecodes.

When a function is called for the first time on an i686 or better,
an x86 backend translates the function, and the x86 code is run.
Since the actual simulator code is still in C, we do not have to worry about
bug reducing the simulation performance.

Registering for simulator events.

The simulator has a very light-weight event logging mechanism. You register the event
with a particular function that you want executed whenever that event occurs.
That function takes one parameter, a hms struct which contains the relevant data for
the event. You can use the ? symbol to print the contents of that structure.
The code looks like the following:

EventManager_RegisterHandler( "proc_pass_fire", "__stop_handler");  

fn __stop_handler(hms)
{
   printf("%s\n",hms.name);
   ? hms;
   if (hms.data == gStopAtPass)
      gInterrupted = 1;
}

Debugging memory problems in bC code
(also, how to get a bC stack trace from within gdb)

You can debug memory problems in your bC code as follows:
First, disable the graphical interface by redefining "main_loop" in the very beginning of your .bug file.

fn main_loop()
{  text_loop(); }

include("<.bug>");

==3438== Invalid read of size 4
==3438==    at 0x80A9A78: VMExecute (AngstroVM.c:1400)
==3438==    by 0x80A734F: VMRunThread (AngstroVM.c:672)
==3438==    by 0x80A6ED4: VMCallToFunctionWithParamsWorker (AngstroVM.c:413)
==3438==    by 0x80A6E8D: VMCallToFunctionWithParamsFD (AngstroVM.c:397)
==3438==    Address 0x45397A7C is 4 bytes before a block of size 128 alloc'd
==3438==    at 0x4002BCAD: malloc (vg_replace_malloc.c:153)
==3438==    by 0x80A797A: NativeShunt1 (AngstroVM.c:852)
==3438==    by 0x80A8CCF: DynamicLink (AngstroVM.c:957)
==3438==    by 0x80A9B37: VMExecute (AngstroVM.c:1421)
---- Attach to GDB ? --- [Return/N/n/Y/y/C/c] ---- 

signal 11

(gdb) signal 11
Continuing with signal SIGSEGV.
//**********************************************************$
// AngstroVM -- SEGV segmentation violation at VM 45fa5400
// Stack Crawl (stack = 4218ee7c, curInstr = 45fa5400)
// (0) 45fa5400 <anonymous>(/*4*/                 line 155         <---------- the location
//                00000003, 080fd5dc, 00000004, ffffffff);
// (1) 4769a83c <anonymous>(/*5*/                 line 140
//                00000003, 4262db68, 41ec15fc, 080fd5dc, 0000000d);
// (2) 481a25a9 __gProfiler_issued_instruction_h  line 249 
//***********************************************************