This week started easy:
- Added integration tests for the about and help subcommands. These were pretty easy to do.
- Added integration tests for the list subcommand. I initially added these tests as expected failures to reason about the appearance and behavior of this command from the point of view of the user before actually working on the code... and I am still writing such code to make these tests pass!
This is where things got a bit awry. Polishing the behavior of the list command so that its interface is consistent among all flag and argument combinations is tricky and non-trivial. I ended up having to change code deep down in the source tree to implement missing features and to change existing bits and pieces:
- Implemented support to print all test case properties, not only the user-specific ones. The properties recognized by the runtime engine are stored as individual arguments of a structure, so these required some externalization code.
- Implemented "test case filtering". This allows users to select what tests to run on the command line at the test case granularity. For example, foo/bar selects all tests in a subdirectory if bar is a directory, or all the tests in the bar test program if it is a binary. But what is new (read: not found in ATF) is that you can even do foo/bar:test-1 where test-1 is the name of a test case within the foo/bar test program. I've been silently wishing for this feature to be available in ATF because it shortens the build/test/edit cycle, but it was not easy to add.
- Changed the internal representation of test suites to ensure all test program names are relative to the root of the test suite. The root of the test suite is considered to be the directory in which the top-level Kyuafile lives (/usr/tests/ in NetBSD). The whole point of doing this is to provide some consistency to the filters when the user decides to execute tests that are not in the current directory. For example, the following are now equivalent:
$ cd /usr/tests && kyua list fs/tmpfs
$ kyua list -k /usr/tests/Kyuafile fs/tmpfs
The plans for the upcoming week are to finish with the clean up of the list command (which involves adding proper error reporting, refactoring of the command module and addition of unit tests) and start cleaning up the test command.
Also, remember that BSDCan 2011 is now around the corner and that I will be talking about ATF, Kyua and NetBSD in it! My plan was to have a kyua-cli-0.1 release by the time of the conference, although this will be tricky to achieve...
