README -- Tcl test suite design document.

 RCS: @(#) $Id: README,v 1.11.2.1 2003/04/01 21:13:07 dgp Exp $

 Contents:

 ---------

     1. Introduction

     2. Running tests

     3. Adding tests

     4. Incompatibilities with prior Tcl versions

 1. Introduction:

 ----------------

 This directory contains a set of validation tests for the Tcl commands

 and C Library procedures for Tcl.  Each of the files whose name ends

 in ".test" is intended to fully exercise the functions in the C source

 file that corresponds to the file prefix.  The C functions and/or Tcl

 commands tested by a given file are listed in the first line of the

 file.

 2. Running tests:

 -----------------

 We recommend that you use the "test" target of Tcl's Makefile to run

 the test suite.  From the directory in which you build Tcl, simply

 type "make test".  This will create a special executable named

 tcltest in which the testing scripts will be evaluated.  To create

 the tcltest executable without running the test suite, simple type

 "make tcltest".

 All the configuration options of the tcltest package are available

 during a "make test" by defining the TESTFLAGS environment variable.

 For example,if you wish to run only those tests in the file append.test,

 you can type:

 	make test TESTFLAGS="-file append.test"

 For interactive testing, the Tcl Makefile provides the "runtest" target.

 Type "make runtest" in your build directory, and the tcltest executable

 will be created, if necessary, then it will run interactively.  At the

 command prompt, you may type any Tcl commands.  If you type

 "source ../tests/all.tcl", the test suite will run.  You may use the

 tcltest::configure command to configure the test suite run as an

 alternative to command line options via TESTFLAGS.  You might also

 wish to use the tcltest::testConstraint command to select the constraints

 that govern which tests are run.  See the documentation for the tcltest

 package for details.

 3. Adding tests:

 ----------------

 Please see the tcltest man page for more information regarding how to

 write and run tests.

 Please note that the all.tcl file will source your new test file if

 the filename matches the tests/*.test pattern (as it should).  The

 names of test files that contain regression (or glass-box) tests

 should correspond to the Tcl or C code file that they are testing.

 For example, the test file for the C file "tclCmdAH.c" is

 "cmdAH.test".  Test files that contain black-box tests may not

 correspond to any Tcl or C code file so they should match the pattern

 "*_bb.test". 

 Be sure your new test file can be run from any working directory.

 Be sure no temporary files are left behind by your test file.

 Use [tcltest::makeFile], [tcltest::removeFile], and [tcltest::cleanupTests]

 properly to be sure of this.

 Be sure your tests can run cross-platform in both a build environment

 as well as an installation environment.  If your test file contains

 tests that should not be run in one or more of those cases, please use

 the constraints mechanism to skip those tests.

 4. Incompatibilities of package tcltest 2.1 with 

    testing machinery of very old versions of Tcl:

 ------------------------------------------------

 1) Global variables such as VERBOSE, TESTS, and testConfig of the

    old machinery correspond to the [configure -verbose], 

    [configure -match], and [testConstraint] commands of tcltest 2.1,

    respectively.

 2) VERBOSE values were longer numeric.  [configure -verbose] values

    are lists of keywords.

 3) When you run "make test", the working dir for the test suite is now

    the one from which you called "make test", rather than the "tests"

    directory.  This change allows for both unix and windows test

    suites to be run simultaneously without interference with each

    other or with existing files.  All tests must now run independently

    of their working directory.

 4) The "all" file is now called "all.tcl"

 5) The "defs" and "defs.tcl" files no longer exist.

 6) Instead of creating a doAllTests file in the tests directory, to

    run all nonPortable tests, just use the "-constraints nonPortable"

    command line flag.  If you are running interactively, you can run

    [tcltest::testConstraint nonPortable 1] (after loading the tcltest

    package).

 7) Direct evaluation of the *.test files by the "source" command is no

    longer recommended.  Instead, "source all.tcl" and use the "-file" and

    "-notfile" options of tcltest::configure to control which *.test files

    are evaluated.