3.3. An example test

3.3.1. Automating testing

Writing a test is a matter of spawning the test program, then using the expect to match the output and report failure as appropriate.

A simple example is as follows.

# Timeout reduced to 3 seconds
set timeout 3

# The name of this test and the command we will run
set test_name "Simple test"
set command_line "or32-elf-sim -f default.cfg test-prog.or32"

# When verbose, tell the user what we are running
if { $verbose > 1 } {
    send_user "starting $command_line\n"
}

# Run the program.
spawn $command_line

expect {
    # Check for any warning messages in the output first
    Warning {
        fail "$test_name: warning: $expect_out(buffer)"
    }

    # Check for any error messages
    ERROR {
        fail "$test_name: error: $expect_out(buffer)"
    }

    # The string indicating successful completion
    "Test complete" {
         pass "$test_name\n"
    }

    # EOF and timeout only come after everything else. EOF must be an error.
    eof {
        fail "$test_name: EOF\n"
    }

    # Timeout requires inspection to determine the cause of failure.
    timeout {
        unresolved "$test_name: timeout"
    }
}

This test will look first for the strings "Warning" or "ERROR" in the output stream from the spawned program. Then it will look for the string Test complete. Finally it will check for end-of-file or timeout.

The ordering matters. For a small program, the spawned output could all fit in the buffer. By that time, the program might have completed execution. So that buffer would match both Test complete and end-of-file. So we must check for successful completion first.

Similarly the program might generate warnings or errors, but still print Test complete. So we should check for warnings or errors before we test for successful completion.