Written Assignment 7

The assignment should be submitted via Blackboard.

IMPORTANT: If you need to include figures and drawings (for example in designing perceptrons and neural networks), those figures and drawings must be electronic. Scans of handmade figures and drawings will not be accepted.

In all questions involving manually designing perceptrons and neural networks, you should assume that the activation function of a perceptron:

These assumptions are stated repeatedly in each question involving manually designing perceptrons and neural networks.

Task 1 (45 points).

We have a binary classification problem, where the two classes are A and B, a pattern is denoted as x, and P(A | x) is uniform and equal to 0.9 for every x.

Part a: What is the error rate of a true Bayes classifier, averaged over all examples? In other words, what is the probability that the Bayes classifier will give the wrong answer for a random x? Justify your answer.

Part b: What is the error rate of a nearest neighbor classifier? In other words, what is the probability that the nearest neighbor classifier will give the wrong answer for a random x? Justify your answer.

Part c: What is the error rate of a 3-nearest neighbor classifier (i.e., a k-nearest neighbor with k=3)? In other words, what is the probability that the 3-nearest neighbor classifier will give the wrong answer for a random x? Justify your answer.

Task 2 (10 points).

At the M-step of the EM algorithm, we recompute the mean and std of every Gaussian by taking weighted averages over all training objects. What would happen if we changed that step, to take unweighted averages instead of weighted averages?

Task 3 (15 points).

Note: In this question you should assume that the activation function of a perceptron:Design a perceptron that takes (in addition to the bias input) three Boolean inputs (i.e., inputs that are equal to 0 for false, and 1 for true), and outputs: 1 if at least two of the three inputs are true, 0 otherwise.

Task 4 (15 points).

Note: In this question you should assume that the activation function of a perceptron:Design a perceptron (i.e, an individual neuron) that takes in two Boolean inputs X and Y and outputs the Boolean value of (X => Y). As a reminder, 0 stands for "false" and 1 stands for "true". You should NOT worry about what your perceptron does when the input values are not 0 or 1

Task 5 (15 points).

Note: In this question you should assume that the activation function of a perceptron:Design a neural network that:

Other Instructions