CS224 -- Spring 2012

Project #1: Assembly Language Programming

Tips for Testing & Debugging MIPS code

It is strongly recommended that you rigorously test your subroutines before showing them in the live MIPS Code Demo for a grade, in order to find any errors or problems.

Testing: No code should be submitted without undergoing rigorous testing. Your group should dedicate adequate time and resources to testing, including enough people and enough time. Testing (and debugging) requires concentration and focus, therefore middle-of-the-night testing is usually of poor quality. Good testing needs a test plan. The author of a program should not be the primary tester; someone neutral and unbiased, who can think differently, should test the code according to the specifications. Before you bring your code to the demo, TEST, TEST, TEST. See below for some good hints on testing.

How do we know if our utility routine does what it is supposed to do? Answer: use creation and visualization utilities

A very good idea, since you are manipulating the elements in a data structure, is to write a utility that allows you to make that data structure with any values that you want, and to write another utility that allows you to display the data structure that you have. This way, you could call the data structure creation utility to set up the values that you want to have in the data structure for your testing of a particular utility routine, then call that utility routine (check, insert, min-max, etc) to have it operate on the elements of the data structure, then call the visualization utility to display the , to "see" if it is what you expect after your utility routine has worked on it.

Since your routines have to move through the elements of the data structure, you also could use your visualization/display utility at any point or points in the sequence of your routine's code (by adding jal display calls to the code) to “display” intermediate results to you, so that you can follow the progress of the data structure as it is operated on by your code, and thus debug your code.

How do we know if our utility routine has unwanted side effects? Answer: use a saved-register visualization utility

Another good idea for testing is to develop a utility that shows the curent values in all the saved registers ($s0-$s7, $gp, $sp, $fp, and $ra), You can call this utility before the call to your routine under test (RUT), and again after the return from it, so that the values before, and values after, are seen together. This will let you see if any of them have changed, which is helpful for seeing if the registers are preserved as they should be, and to find any side effects, intended or unintended.

What suggestions do you have for doing testing and debugging, to avoid making new errors even while fixing old ones?

1) After making any changes, even simple ones such as copy-paste, be sure to re-test. Do not assume that the change was successful, or that it had no unintended side effects!! You must test and see, prove it. Assumptions are dangerous and risky substitutes for actual testing.

2) Testing should be comprehensive, at least the last stage testing. Small tests may miss some mistakes/errors. The only way to be sure that the whole system works is to test the whole system. Therefore, partial tests should be avoided, especially in final stage testing (the one upon which approval ["Yes, we believe that it works"] is based)

3) Fixing one thing can break another thing. This is a common experience in test and debug engineering. Making a change to correct one error can introduce an unintended new problem. Things can work worse after a "fix" than before it!!! Therefore, #1) and #2) above are extremely important.

Only the 6 utility routines you write should be submitted. DO NOT submit any of your test code, or any main program. Each utility routine will be tested individually, by calling it from suitable places in our test code. In your testing, your test code will be the main program which calls the routines you have written. Students should make a test program that has an extensive set of test cases, to test every aspect of the specification. Be sure to test your code very thoroughly! Therefore, you should test extensively, to be assured that your code is bug-free and meets the specifications. We will cut-and-paste your list utility code (for build:, check:, etc) into our test code in a very similar way to how you are cut-and-pasting it in your test phase. Your code MUST NOT crash the test code, it must return from the call with stack intact and unchanged, with $fp, $gp, $ra, and $s registers unchanged. This requirement (to obey the MIPS software convention) is in addition to the specifications given in the Project #1 specification.