Table Utility Programs

(A Brief Documentation of the Programs dom / tmerge / tsplit / opc / xmat)

Contents

• Introduction
• Determining Attribute Domains
• Merge Tables
• Split a Table
• Computing One Point Coverages
• Computing a Confusion Matrix
• Copying
• Download
• Contact

Introduction

I am sorry that there is no detailed documentation yet. Below you can find a brief explanation of how to grow a decision tree with the program dti, how to prune a decision tree with the program dtp, how to execute a decision tree with the program dtx, and how to extract rules from a decision tree with the program dtr. For a list of options, call the programs without any arguments.

Enjoy,
Christian Borgelt

As a simple example for the explanations below I use the dataset in the file table/ex/drug.tab, which lists 12 records of patient data (sex, age, and blood pressure) together with an effective drug (effective w.r.t. some unspecified disease). The contents of this file is:

   Sex    Age Blood_pressure Drug
   male   20  normal         A
   female 73  normal         B
   female 37  high           A
   male   33  low            B
   female 48  high           A
   male   29  normal         A
   female 52  normal         B
   male   42  low            B
   male   61  normal         B
   female 30  normal         A
   female 26  low            B
   male   54  high           A

back to the top

Determining Attribute Domains

The domains of the columns of th table drug.tabcan be determined with the program dom

  dom -a drug.tab drug.dom

The program dom assumes that the first line of the table file contains the column names. (This is the case for the example file drug.tab.) If you have a table file without column names, you can let the program read the column names from another file (using the -h option) or you can let the program generate default names (using the -d option), which are simply the column numbers. The -a option tells the program to determine automatically the column data types. Thus the values of the Age column are automatically recognized as integer values.

After dom has finished, the contents of the file drug.dom should look like this:

  dom(Sex) = { male, female };
  dom(Age) = ZZ;
  dom(Blood_pressure) = { normal, high, low };
  dom(Drug) = { A, B };

The special domain ZZ represents the set of integer numbers, the special domain IR (not used here) the set of real numbers. (The double Z and the I in front of the R are intended to mimic the bold face or double stroke font used in mathematics to write the set of integer or the set of real numbers. All programs that need to read a domain description also recognize a single Z or a single R.)

back to the top

Merge Tables

The program tmerge can be used to merge two tables and to project a table to a subset of its columns. The latter consists simply in merging a table to another which contains less columns. I only demonstrate the projection by merging the table in the file drug.tab to the empty table in the file drug.hdr:

  tmerge -a drug.hdr drug.tab drug.prj

This command removes the Age column from the table drug.tab (since this column is missing in the file drug.hdr) and writes the result to the file drug.prj. After the program tmerge has finished, the contents of the file drug.prj should be:

  Sex    Blood_pressure Drug
  male   normal         A
  female normal         B
  female high           A
  male   low            B
  female high           A
  male   normal         A
  female normal         B
  male   low            B
  male   normal         B
  female normal         A
  female low            B
  male   high           A

Since the option -a is given, the columns of the output file are aligned. If the file drug.hdr contained tuples, these tuples would precede the tuples from the file drug.tab in the output file drug.prj.

back to the top

Split a Table

If you are interested in the sets of patients with low, normal, or high blood pressure, you can split the table into subtables, each of which contains only tuples with a specific value for the column Blood_pressure, with

  tsplit -a -c Blood_pressure drug.tab

This should result in three files - 0.tab, 1.tab and 2.tab - with the following contents:

  0.tab:   Sex    Age Blood_pressure Drug
           male   20  normal         A
           female 73  normal         B
           male   29  normal         A
           female 52  normal         B
           male   61  normal         B
           female 30  normal         A

  1.tab:   Sex    Age Blood_pressure Drug
           female 37  high           A
           female 48  high           A
           male   54  high           A

  2.tab:   Sex    Age Blood_pressure Drug
           male   33  low            B
           male   42  low            B
           female 26  low            B

That is, the file 0.tab contains all patients with normal blood pressure, the file 1.tab all patients with high blood pressure, and the file 2.tab all patients with low blood pressure. The tables are aligned since the option -a was given. With the -c option the column is specified on which the split is based. Similarly, the table can be split in such a way that the relative frequencies of the values are maintained (stratified split). For example, calling the program tsplit with

  tsplit -a -t3 -c Blood_pressure drug.tab

should result in three files (3 because of the -t3 option) - 0.tab, 1.tab and 2.tab - with the following contents:

  0.tab:   Sex    Age Blood_pressure Drug
           male   20  normal         A
           female 52  normal         B
           female 37  high           A
           male   33  low            B

  1.tab:   Sex    Age Blood_pressure Drug
           female 73  normal         B
           male   61  normal         B
           female 48  high           A
           male   42  low            B

  2.tab:   Sex    Age Blood_pressure Drug
           male   29  normal         A
           female 30  normal         A
           male   54  high           A
           female 26  low            B

back to the top

Computing One Point Coverages

The program opc can be used to reduce a table. This does not change anything for the original table, but simplifies the table that resulted from the application of the program tmerge shown above. This table can be reduced by calling the program opc with

  opc -a drug.prj drug.red

After the program opc has finished, the contents of the file drug.red should read like this:

  Sex    Blood_pressure Drug #
  male   normal         A    2
  male   normal         B    1
  male   high           A    1
  male   low            B    2
  female normal         A    1
  female normal         B    2
  female high           A    2
  female low            B    1

The number in the last column indicates the number of occurences of the corresponding tuple (table row) in the original table.

The opc program can also be used to compute one point coverages, either in a fully expanded or in a compressed form. One point coverages are considered in possibility theory and computing them is important for inducing possibilistic network from data. However, explaining this in detail would lead too far.

back to the top

Computing a Confusion Matrix

The program xmat can be used to evaluate a classification result. It reads a table file and computes a confusion matrix from two columns of this table. It uses the last two columns by default (the last column for the x- and the semi-last for the y-direction). Other columns can be selected via the options -x and -y followed by the name of the columns that are to be used for the x- or y-direction of the confusion matrix. To demonstrate this program we use the file drug.cls, which contains simply the data from the file drug.tab with an additional classification column:

  Sex    Age Blood_pressure Drug Class
  male   20  normal         A    B
  female 73  normal         B    B
  female 37  high           A    A
  male   33  low            B    B
  female 48  high           A    B
  male   29  normal         A    A
  female 52  normal         B    B
  male   42  low            B    A
  male   61  normal         B    B
  female 30  normal         A    A
  female 26  low            B    B
  male   54  high           A    A

To determine a confusion matrix for this table, simply call the program xmat with

  xmat drug.cls

The output, which by default is written to the terminal, should read like this:

  confusion matrix for Drug vs. Class:
   no | value  |      1      2 | errors
  ----+--------+---------------+-------
    1 | A      |      4      2 |      2
    2 | B      |      1      5 |      1
  ----+--------+---------------+-------
      | errors |      1      2 |      3

In this matrix the x-direction (columns) corresponds to the column Class and the y-direction (rows) to the column Drug. As you can see, for drug A the classification is wrong in two cases (first line, second column of the matrix), for drug B it is wrong in one case (second line, first column). Overall there are three errors.

back to the top

Copying

dti/dtp/dtx/dtr/rsx - induce, prune, and execute decision and regression trees
copyright © 1996-2003 Christian Borgelt

These programs are free software; you can redistribute them and/or modify them under the terms of the GNU Lesser (Library) General Public License as published by the Free Software Foundation.

These programs are distributed in the hope that they will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser (Library) General Public License for more details.

back to the top

Download

Download page with most recent version.

back to the top

Contact

back to the top