Deep neural networks contain input layers, hidden layers and, output layers.

## NAND function example

We will use the above simple neural network with and the activation function f chosen to be the unit step function U(z) :

## Hidden Layer Models

let’s consider a simple 2-dimensional classification task. The training set is made up of 4 points listed below:

The dataset is illustrated below (blue – positive, red – negative):

For simplicity, y^{(i)} can be either -1 or 1.

denote the output of the hidden layer.

The weights of the network are given as follows:

If we consider a set:

, and make linearly separable.

gives the following results:

gives the following results: