2.2 Testing
2.2.1 check: and where: blocks
Tests in Pyret are written in special testing blocks. These blocks can contain any Pyret code that isn’t toplevel-only (like data definitions and import or provide statements), and are the only places where Testing Operators can be used.
2.2.1.1 check: blocks
The simplest testing blocks are check: blocks. They can be written at the top-level or inside other testing blocks. Check blocks are a unit of reporting test results, so all the test operators that evaluate inside a check block will be reported as part of that block. For example, these two check blocks:
check "a first block": 5 is 5 4 is 5 end check "a second block": 6 is 7 end
will report:
Check block: a first block |
test (5 is 5): ok |
test (4 is 5): failed, reason: |
Values not equal: |
4 |
5 |
1/2 tests passed in check block: a first block |
|
Check block: a second block |
test (6 is 7): failed, reason: |
Values not equal: |
6 |
7 |
The test failed. |
|
1/3 tests passed in all check blocks |
Testing blocks are also a unit of failure: most of the time an error stops the whole program, but inside a check block (and also inside raises, mentioned later), the error is stopped and reported, and Pyret goes on to evaluating the next check block:
check "error-block": raise("an error here doesn't stop the next check block from running") string-length("this test doesn't run") is 21 end check "a later block": string-length("these tests still run") is 21 end
Keep an eye out for the message "Check block <some-block> ended in an error (all tests may not have run):", because it means that later tests in the same block may not have run, so the output doesn’t reflect all the tests that were written.
2.2.1.2 where: blocks
Sometimes a function has tests that are explicitly associated with it. For these cases, the function can end in a where: block rather than immediately with end. where: blocks run the same way that check: blocks do, and their name is taken from the function they are attached to.
fun double(n): n + n where: double(10) is 20 double(15) is 30 end
2.2.2 Testing Operators
Testing operators should be written on their own line inside a check: or where: block. They can check for a number of properties and come in several forms.
2.2.2.1 Binary Test Operators
Many useful tests compare two values, whether for a specific type of equality or a more sophisticated predicate.
expr1 is expr2
Evaluates expr1 and expr2 to values, and checks if two values are equal via equal-always, reporting success if they are equal, and failure if they are not.
expr1 is-not expr2
Like is, but failure and success are reversed.
expr1 is-roughly expr2
Like is, but tolerant of roughnum values: specifically, this is a shorthand for is%(within(0.000001)).
expr1 is%(pred) expr2
Evaluates expr1 and expr2 to values, and pred to a value that must be a function (an error is reported if pred is not a function). It then applies pred to the two values from expr1 and expr2. If the result of that call is true, reports success, otherwise reports failure.
expr1 is-not%(pred) expr2
Like is%, but failure and success are reversed.
check: fun less-than(n1, n2): n1 < n2 end 1 is%(less-than) 2 2 is-not%(less-than) 1 end check: fun longer-than(s1, s2): string-length(s1) > string-length(s2) end "abc" is%(longer-than) "ab" "" is-not%(longer-than) "" end check: fun equal-any-order<a>(l1 :: List<a>, l2 :: List<a>): same-length = (l1.length() == l2.length()) all-present = for lists.all(elt from l1): lists.member(l2, elt) end same-length and all-present end [list: 1, 2, 3] is%(equal-any-order) [list: 3, 2, 1] [list: 1, 2, 3] is%(equal-any-order) [list: 2, 1, 3] [list: 1, 2, 3, 3] is-not%(equal-any-order) [list: 2, 1, 3] end check: fun one-of(ans, elts): lists.member(elts, ans) end some-strings = [list: "123", "132", "213", "231", "312", "321"] "321" is%(one-of) some-strings "123" is%(one-of) some-strings end check: fun around(delta): lam(actual, target): num-abs(target - actual) <= delta end end 5.05 is%(around(0.1)) 5 5.00002 is-not%(around(0.00001)) 5 end
expr1 is== expr2
Shorthand for expr1 is%(equal-always) expr1. Same as is.
expr1 is-not== expr2
Like is==, but failure and success are reversed. Same as is-not.
expr1 is=~ expr2
Shorthand for expr1 is%(equal-now) expr1
expr1 is-not=~ expr2
Like is=~, but failure and success are reversed.
expr1 is<=> expr2
Shorthand for expr1 is%(identical) expr1
expr1 is-not<=> expr2
Like is<=>, but failure and success are reversed.
2.2.2.2 Unary Test Operators
expr satisfies pred
Evaluates expr to a value and pred to a value expected to be a function (if not a function, an error is thrown). Then, pred(val) is evaluated, and if the result is true, the test succeeds, and if false, the test fails.
expr violates pred
Like satisfies, but failure and success are reversed.
check: [list:] satisfies is-empty [list:] satisfies lam(l): l.length() == 0 end is-odd = lam(n :: Number): num-modulo(n, 2) == 1 end 5 satisfies is-odd 6 violates is-odd end
2.2.2.3 Exception Test Operators
expr raises exn-string
Evaluates expr and expects an error to be raised. If no error is raised, the test fails.
If an error is the result, the torepr function is called on the exception value, and raises checks that exn-string is contained within that string. If so, the test passes, otherwise, it fails.
For simple errors (like those in many programming assignments), it works to use raise on a string value and check that that string is raised. For larger programs, it can be useful to construct more sophisticated error values and use raises-satisfies to test them.
check: raise("the roof!") raises "the roof" string-length("too", "many", "strings") raises "arity-mismatch" {}.x raises "field-not-found" end
Warning! These two tests are not equivalent:
check "actually catches the error": raise("error!") raises "error!" end check "error happens before raises": value = raise("error!") value raises "error!" end
This is because the left-hand-side of raises is a special position that can detect and catch errors, which normal expressions do not do. So the second check block fails before even getting to the raises line; try it out and see what happens.
expr raises-other-than exn-string
Like raises, but the result must not contain exn-string.
expr does-not-raise
Evaluates expr and checks that no error is raised while evaluating it. The expression can evaluate to any value.
expr raises-satisfies pred
As the name suggests, this combines the idea of raises with satisfies and calls pred on the exception that expr raises (if any). Still fails if no exception is raised.
import is-field-not-found from error check: o = {} o.x raises-satisfies is-field-not-found end
expr raises-violates pred
Like raises-satisfies, but the predicate must return false. Still fails if no exception is raised.