This post is an extract from one of my earlier project under Prof. Alfred Aho for the course – Programming Language & Translators, where I implemented a Test Suite for validating the KFrog Language interpreter.

Report: http://docs.google.com/viewer?url=http://gauravpandey.webs.com/KFrog_FinalProjectReport.pdf

Technical details: https://docs.google.com/View?id=d8gtx9j_164g2gz7k5x

Originally, the idea was to test the graphical output generated by the interpreter. Since, it was JVM dependent we assumed that Java would produce the correct graphical output as long as our interpreter correctly interpreted the given program. Thus, the focus shifted to testing the semantics (grammar) of the language. To accomplish this, we decided of having an intermediate language which would be the de facto representation for debug messages and expanded-gold statements. My general strategy to semantic testing was a 4-step approach as outlined below:

Step 1 involved delving into the gold standard representation format. I had to come up with a format which would enable a tester to perform testing in the simplest possible way.  To that end we decided that a Web based markup language related representation would be ideal. HTML was chosen as the representative language for our gold standard representation of Test Cases.

   1:  <pond xsize="" ysize="" color="">

   2:  <frog id="">

   3:  </center>

   4:  </startdraw>

   5:  <loop value="">

   6:  </action type="" value="">

   7:  </action type="" valuemin="" valuemax="">

   8:  </curve type="" radius="" degree="">

   9:  </curve type="" radiusmin="" radiusmax="" degreemin="" degreemax="">

  10:  </goto xvalue="" yvalue="">

  11:  </goto xvaluemin="" xvaluemax="" yvaluemin="" yvaluemax="">

  12:  </loop>

  13:  </stopdraw>

  14:  </frog>

  15:  </pond>

In Step 2, we fetch the debug messages generated by the Interpreter by passing the parameter –test-semantics. This enables us to generate a .debug file which is a true mapping of the graphics generated on the screen. The following is a typical set of debug statements for generating a square of side 100.

   1:  pondsize 300,300

   2:  pondcolor WHITE

   3:  1 new Frog {

   4:  1 center

   5:  1 frogcolor BLUE

   6:  1 speed 3

   7:  1 startdraw

   8:  1 forward 100

   9:  1 turnleft 90

  10:  1 forward 100

  11:  1 turnleft 90

  12:  1 forward 100

  13:  1 turnleft 90

  14:  1 forward 100

  15:  1 turnleft 90

  16:  1 stopdraw

  17:  1 }

In Step 3, we formulate the test cases which are capable of testing a specific subset of the language all at once. The following is a gold standard representation for generating a square of length 100

   1:  <pond xsize="400" ysize="400" color="white">

   2:  <frog id="1">

   3:  </center>

   4:  </startdraw>

   5:  <loop value="4">

   6:  </action type="forward" value="100">

   7:  </action type="turnleft" value="90">

   8:  </loop>

   9:  </stopdraw>

  10:  </frog>

  11:  </pond>

In step 4, we utilize the Tester program which fetches the content of .debug and .gold file and populates 2 data structures (Vectors). The data stored in the data structures is in an Intermediate Format. In turn, both data structures are compared for exactness and a final response is provided – ‘PASS’ or ‘FAIL’. The following diagram shows the entire process in detail: