Lab 2

Due: Monday, March 13, the beginning of class

More specifically, I am giving you several files which will help you in your work, all of which are located in the following directory (which you should probably copy to your work space):

         ~karl/www-docs/courses/mcs394-s00/labs/lab2/decision-trees/

• The Scheme file tennis-data.scm contains the tennis data from Table 3.2 on page 59 of Mitchell. You should read through this file carefully so you understand the form the data must be in.

• The Scheme file utility.scm contains various Scheme procedures you should find helpful. In summary, here are the procedures you will need:
  • A variety of procedures that allow you to access the attributes and values in the data-set you are working with. In particular, you will probably find it handy to manipulate the attributes through their indices, which are their positions within the various lists. These comprise the six procedures after the two utility procedures filter and delq.

  • The procedure random-fractional-list, which will seful when partitioning the data into training and testing sets. This procedure is well-documented in the final large comment in utility.scm.

• The Scheme file house-votes-data.scm is the Scheme form of the House voting data I briefly described in class. The data files I used to create this Scheme datafile are in the following two files:

  • The file house-votes-84.names

  • The file house-votes-84.data

• For your reference, the house voting data came from the University of California, Irvine, Machine Learning Repository. Following are links there that you might find interesting:

  • The web page for UCI Machine Learning Group.

  • The web page for the Machine Learning Database Repository.

  • The summary page for the repository. Our data set is listed there as Congressional Voting Records Database.

Your tasks:

1. Implement the ID3 algorithm as described on page 56 of Mitchell. I recommend that you write a procedure called id3 that takes a list of training examples and returns the decision tree that ID3 should return for that list. In particular, if you use all of the tennis-data in tennis-data.scm, you should get the tree in Figure 3.1 on page 53 of Mitchell.

   One remark: I recommend that you use a fairly simple, concrete representation for your decision tree. For example, my representation for the decision tree in Figure 3.1 is:

           (outlook (sunny (humidity (high no)
                                     (normal yes)))
                    (overcast yes)
                    (rain (wind (strong no)
                                (weak yes))))
   

2. You should write a procedure classify which takes a decision tree and an example, and returns the target classification that the decision tree gives for the example.

3. Since the house voting data has missing attributes, you should extend your ID3 procedure to allow for missing attributes, using one of the methods described on page 75 of Mitchell.

4. Finally, you should test run your ID3 procedure on the house voting data using various fractions of the data as training examples, and seeing how well it classifies the non-training examples. Remember that the procedure random-fractional-list in utility.scm will be helpful for this task.