sl@0: README -- Tcl test suite design document.
sl@0: 
sl@0: RCS: @(#) $Id: README,v 1.11.2.1 2003/04/01 21:13:07 dgp Exp $
sl@0: 
sl@0: Contents:
sl@0: ---------
sl@0: 
sl@0:     1. Introduction
sl@0:     2. Running tests
sl@0:     3. Adding tests
sl@0:     4. Incompatibilities with prior Tcl versions
sl@0: 
sl@0: 1. Introduction:
sl@0: ----------------
sl@0: 
sl@0: This directory contains a set of validation tests for the Tcl commands
sl@0: and C Library procedures for Tcl.  Each of the files whose name ends
sl@0: in ".test" is intended to fully exercise the functions in the C source
sl@0: file that corresponds to the file prefix.  The C functions and/or Tcl
sl@0: commands tested by a given file are listed in the first line of the
sl@0: file.
sl@0: 
sl@0: 2. Running tests:
sl@0: -----------------
sl@0: 
sl@0: We recommend that you use the "test" target of Tcl's Makefile to run
sl@0: the test suite.  From the directory in which you build Tcl, simply
sl@0: type "make test".  This will create a special executable named
sl@0: tcltest in which the testing scripts will be evaluated.  To create
sl@0: the tcltest executable without running the test suite, simple type
sl@0: "make tcltest".
sl@0: 
sl@0: All the configuration options of the tcltest package are available
sl@0: during a "make test" by defining the TESTFLAGS environment variable.
sl@0: For example,if you wish to run only those tests in the file append.test,
sl@0: you can type:
sl@0: 
sl@0: 	make test TESTFLAGS="-file append.test"
sl@0: 
sl@0: For interactive testing, the Tcl Makefile provides the "runtest" target.
sl@0: Type "make runtest" in your build directory, and the tcltest executable
sl@0: will be created, if necessary, then it will run interactively.  At the
sl@0: command prompt, you may type any Tcl commands.  If you type
sl@0: "source ../tests/all.tcl", the test suite will run.  You may use the
sl@0: tcltest::configure command to configure the test suite run as an
sl@0: alternative to command line options via TESTFLAGS.  You might also
sl@0: wish to use the tcltest::testConstraint command to select the constraints
sl@0: that govern which tests are run.  See the documentation for the tcltest
sl@0: package for details.
sl@0: 
sl@0: 3. Adding tests:
sl@0: ----------------
sl@0: 
sl@0: Please see the tcltest man page for more information regarding how to
sl@0: write and run tests.
sl@0: 
sl@0: Please note that the all.tcl file will source your new test file if
sl@0: the filename matches the tests/*.test pattern (as it should).  The
sl@0: names of test files that contain regression (or glass-box) tests
sl@0: should correspond to the Tcl or C code file that they are testing.
sl@0: For example, the test file for the C file "tclCmdAH.c" is
sl@0: "cmdAH.test".  Test files that contain black-box tests may not
sl@0: correspond to any Tcl or C code file so they should match the pattern
sl@0: "*_bb.test". 
sl@0: 
sl@0: Be sure your new test file can be run from any working directory.
sl@0: 
sl@0: Be sure no temporary files are left behind by your test file.
sl@0: Use [tcltest::makeFile], [tcltest::removeFile], and [tcltest::cleanupTests]
sl@0: properly to be sure of this.
sl@0: 
sl@0: Be sure your tests can run cross-platform in both a build environment
sl@0: as well as an installation environment.  If your test file contains
sl@0: tests that should not be run in one or more of those cases, please use
sl@0: the constraints mechanism to skip those tests.
sl@0: 
sl@0: 4. Incompatibilities of package tcltest 2.1 with 
sl@0:    testing machinery of very old versions of Tcl:
sl@0: ------------------------------------------------
sl@0: 
sl@0: 1) Global variables such as VERBOSE, TESTS, and testConfig of the
sl@0:    old machinery correspond to the [configure -verbose], 
sl@0:    [configure -match], and [testConstraint] commands of tcltest 2.1,
sl@0:    respectively.
sl@0: 
sl@0: 2) VERBOSE values were longer numeric.  [configure -verbose] values
sl@0:    are lists of keywords.
sl@0: 
sl@0: 3) When you run "make test", the working dir for the test suite is now
sl@0:    the one from which you called "make test", rather than the "tests"
sl@0:    directory.  This change allows for both unix and windows test
sl@0:    suites to be run simultaneously without interference with each
sl@0:    other or with existing files.  All tests must now run independently
sl@0:    of their working directory.
sl@0: 
sl@0: 4) The "all" file is now called "all.tcl"
sl@0: 
sl@0: 5) The "defs" and "defs.tcl" files no longer exist.
sl@0: 
sl@0: 6) Instead of creating a doAllTests file in the tests directory, to
sl@0:    run all nonPortable tests, just use the "-constraints nonPortable"
sl@0:    command line flag.  If you are running interactively, you can run
sl@0:    [tcltest::testConstraint nonPortable 1] (after loading the tcltest
sl@0:    package).
sl@0: 
sl@0: 7) Direct evaluation of the *.test files by the "source" command is no
sl@0:    longer recommended.  Instead, "source all.tcl" and use the "-file" and
sl@0:    "-notfile" options of tcltest::configure to control which *.test files
sl@0:    are evaluated.