ATF is a program, and as happens with any application, it must be (automatically) tested to ensure it works according to its specifications. But as you already know, ATF is a testing framework so... is it possible to automatically test it? Can it test itself? Should it do it? The thing is: it can and it should, but things are not so simple.

ATF can test itself because it is possible to define test programs through ATF to check the ATF tools and libraries. ATF should test itself because the resulting test suite will be a great source of example code and because its execution will be on its own a good stress test for the framework. See the tests/atf directory to check what I mean; specially, the unit tests for the fs module, which I've just committed, are quite nice :-) (For the record: there currently are 14 test programs in that directory, which account for a total of 60 test cases.)

However, ATF should not be tested exclusively by means of itself. If it did so, any failure (even the most trivial one) in the ATF's code could result in false positives or false negatives during the execution of the test suite, leading to wrong results hard to discover and diagnose. Imagine, for example, that a subtle bug made the reporting of test failures to appear as test passes. All tests could start to succeed immediately and nobody could easily notice, surely leading to errors in further modifications.

This is why a bootstrapping test suite is required: one that ensures that the most basic functionality of ATF works as expected, but which does not use ATF to run itself. This additional test suite is already present in the source tree, and is written using GNU Autotest, given that I'm using the GNU Autotools as the build system. Check the tests/bootstrap directory to see what all this is about.

ATF's self-testing is, probably, the hardest thing I've encountered in this project so far. It is quite tricky and complex to get right, but it's cool! Despite being hard, having a complete test suite for ATF is a must so it cannot be left aside. Would you trust a testing framework if you could not quickly check that it worked as advertised? I couldn't ;-)