Neural Nets - Revision history

Monakhos: /* Tensorflow Playground */

2020-05-04T12:20:15Z

Tensorflow Playground

← Older revision		Revision as of 05:20, 4 May 2020
Line 23:		Line 23:


	~~<!--~~
	~~<html>~~
	<iframe src="http://playground.tensorflow.org/#activation=tanh&regularization=L1&batchSize=15&dataset=gauss&regDataset=reg-plane&learningRate=0.003&regularizationRate=0.001&noise=0&networkShape=1&seed=0.73576&showTestData=false&discretize=false&percTrainData=40&x=true&y=false&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false&showTestData_hide=true&stepButton_hide=true&problem_hide=true&noise_hide=true&discretize_hide=true&regularization_hide=true&dataset_hide=true&batchSize_hide=true&percTrainData_hide=true&regularizationRate_hide=true&learningRate_hide=true&numHiddenLayers_hide=true" height="1050" width="1050" scrolling="no" style="border:none; overflow:hidden; padding-top:-220px; margin-top:-220px;"></iframe>
	~~</html>~~
	~~-->~~

	{{Clear}}		{{Clear}}

Monakhos at 11:52, 4 May 2020

2020-05-04T11:52:29Z

← Older revision		Revision as of 04:52, 4 May 2020
Line 19:		Line 19:

	===Tensorflow Playground===		===Tensorflow Playground===
	~~<html>~~
	~~<iframe src="http://bradleymonk.com"></iframe>~~
	~~</html>~~

			<iframe key="tf" path="#activation=tanh&regularization=L1&batchSize=15&dataset=gauss&regDataset=reg-plane&learningRate=0.003&regularizationRate=0.001&noise=0&networkShape=1&seed=0.73576&showTestData=false&discretize=false&percTrainData=40&x=true&y=false&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false&showTestData_hide=true&stepButton_hide=true&problem_hide=true&noise_hide=true&discretize_hide=true&regularization_hide=true&dataset_hide=true&batchSize_hide=true&percTrainData_hide=true&regularizationRate_hide=true&learningRate_hide=true&numHiddenLayers_hide=true" />


			<!--
	<html>		<html>
	<iframe src="http://playground.tensorflow.org/#activation=tanh&regularization=L1&batchSize=15&dataset=gauss&regDataset=reg-plane&learningRate=0.003&regularizationRate=0.001&noise=0&networkShape=1&seed=0.73576&showTestData=false&discretize=false&percTrainData=40&x=true&y=false&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false&showTestData_hide=true&stepButton_hide=true&problem_hide=true&noise_hide=true&discretize_hide=true&regularization_hide=true&dataset_hide=true&batchSize_hide=true&percTrainData_hide=true&regularizationRate_hide=true&learningRate_hide=true&numHiddenLayers_hide=true" height="1050" width="1050" scrolling="no" style="border:none; overflow:hidden; padding-top:-220px; margin-top:-220px;"></iframe>		<iframe src="http://playground.tensorflow.org/#activation=tanh&regularization=L1&batchSize=15&dataset=gauss&regDataset=reg-plane&learningRate=0.003&regularizationRate=0.001&noise=0&networkShape=1&seed=0.73576&showTestData=false&discretize=false&percTrainData=40&x=true&y=false&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false&showTestData_hide=true&stepButton_hide=true&problem_hide=true&noise_hide=true&discretize_hide=true&regularization_hide=true&dataset_hide=true&batchSize_hide=true&percTrainData_hide=true&regularizationRate_hide=true&learningRate_hide=true&numHiddenLayers_hide=true" height="1050" width="1050" scrolling="no" style="border:none; overflow:hidden; padding-top:-220px; margin-top:-220px;"></iframe>
	</html>		</html>
			-->

	{{Clear}}		{{Clear}}

Monakhos at 11:41, 4 May 2020

2020-05-04T11:41:21Z

← Older revision		Revision as of 04:41, 4 May 2020
Line 19:		Line 19:

	===Tensorflow Playground===		===Tensorflow Playground===
			<html>
			<iframe src="http://bradleymonk.com"></iframe>
			</html>

	<html>		<html>

Monakhos: /* Tensorflow Playground */

2020-05-04T11:38:34Z

Tensorflow Playground

← Older revision		Revision as of 04:38, 4 May 2020
Line 20:		Line 20:
	===Tensorflow Playground===		===Tensorflow Playground===

	~~{{SmallBox\|display=block~~		<html>
	~~\|float~~=~~none~~		<iframe src="http://playground.tensorflow.org/#activation=tanh&regularization=L1&batchSize=15&dataset=gauss&regDataset=reg-plane&learningRate=0.003&regularizationRate=0.001&noise=0&networkShape=1&seed=0.73576&showTestData=false&discretize=false&percTrainData=40&x=true&y=false&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false&showTestData_hide=true&stepButton_hide=true&problem_hide=true&noise_hide=true&discretize_hide=true&regularization_hide=true&dataset_hide=true&batchSize_hide=true&percTrainData_hide=true&regularizationRate_hide=true&learningRate_hide=true&numHiddenLayers_hide=true" height="1050" width="1050" scrolling="no" style="border:none; overflow:hidden; padding-top:-220px; margin-top:-220px;"></iframe>
	~~\|clear~~=~~none~~		</html>
	~~\|width~~=~~1060px~~
	~~\|height~~=~~685px~~
	~~\|margin~~=~~2px 2%~~
	~~\|padding~~=~~0px 0px 0px 0px~~
	~~\|border~~-width=~~2px~~
	\|border-~~radius=2px~~
	~~\|{{#widget~~:~~Tensorflow1}}~~
	}}

	{{Clear}}		{{Clear}}

Bradley Monk: /* Tensorflow */

2020-05-04T07:31:24Z

Tensorflow

← Older revision		Revision as of 00:31, 4 May 2020
Line 18:		Line 18:
	{{Clear}}		{{Clear}}

	===Tensorflow===		===Tensorflow Playground===

	{{SmallBox\|display=block		{{SmallBox\|display=block
	\|float=none		\|float=none

Bradley Monk at 07:30, 4 May 2020

2020-05-04T07:30:12Z

← Older revision		Revision as of 00:30, 4 May 2020
Line 1:		Line 1:
	~~<big>TUTORIAL ON MACHINE LEARNING AND NEURAL NETWORKS</big>~~{{SmallBox\|float=right\|clear=none\|margin=0px 0px 8px 18px\|width=170px\|font-size=13px\|Tutorial Pages\|txt-size=11px\|		{{SmallBox\|float=right\|clear=none\|margin=0px 0px 8px 18px\|width=170px\|font-size=13px\|Tutorial Pages\|txt-size=11px\|
	1. [[Neural Nets\|Intro]]<br>		1. [[Neural Nets\|Intro]]<br>
	2. [[Neural Nets 2\|Network Inputs]]<br>		2. [[Neural Nets 2\|Network Inputs]]<br>
Line 5:		Line 5:
	4. [[Neural Nets 4\|Network Outputs]]<br>		4. [[Neural Nets 4\|Network Outputs]]<br>
	}}		}}
			Below there is an embedded a neural network classifier, rendered using [https://www.tensorflow.org Tensorflow][http://playground.tensorflow.org Playground]. There are a variety of knobs and buttons on the interface; as we move along, more of these options will become available. Don't worry though, all these will be explained in detail, in due time. For now though, let's define our primary goal throughout this tutorial: '''classification'''.

	~~{{Clear}}~~		The primary task is to train neural nets to classify items into categories, based on some limited information. Like fruit or vegetable; or undergrad major; or Alzheimer's Disease patient (CASE) or control participant (CTRL). In the Tensorflow playground below, you can see a bunch of orange and blue dots. Instead of simply thinking about these as dots at arbitrary spatial coordinates, it will be helpful to think of these as representing people in a clinical study. Let's define the blue dots are patients from the CASE group, and orange dots are CTRL participants. What is our 'limited information' about them? Let's say we have collected information about their age and their score on a dementia screening exam (scores represent number of items forgotten). So the first thing we'd probably want to do is make a scatter plot of these two variables. Let's define their respective dimensions on the plot axes as:

	Let's just dive right in... Below I've embedded a neural network classifier rendered using [https://www.tensorflow.org Tensorflow][http://playground.tensorflow.org Playground]. There are a variety of knobs and buttons on the interface; as we move along, more of these options will become available. Don't worry though, all these will be explained in detail, in due time. For now though, let's define our primary goal throughout this tutorial: '''classification'''.

	~~Our~~ primary task is to train neural nets to classify items into categories, based on some limited information. Like fruit or vegetable; or undergrad major; or Alzheimer's Disease patient (CASE) or control participant (CTRL). In the Tensorflow playground below, you can see a bunch of orange and blue dots. Instead of simply thinking about these as dots at arbitrary spatial coordinates, it will be helpful to think of these as representing people in a clinical study. Let's define the blue dots are patients from the CASE group, and orange dots are CTRL participants. What is our 'limited information' about them? Let's say we have collected information about their age and their score on a dementia screening exam (scores represent number of items forgotten). So the first thing we'd probably want to do is make a scatter plot of these two variables. Let's define their respective dimensions on the plot axes as:

	* x-axis \| dim1 \| current age ('''''AGE''''')		* x-axis \| dim1 \| current age ('''''AGE''''')
Line 17:		Line 14:
	Notice that after plotting, the dots seem to form clusters. That very promising! If you were asked to draw a line on this plane, to separate these two clusters, it could be easily done. Our brain's neural nets have already solved the the spatial problem. Now let's see if an artificial neural net can solve the same problem.		Notice that after plotting, the dots seem to form clusters. That very promising! If you were asked to draw a line on this plane, to separate these two clusters, it could be easily done. Our brain's neural nets have already solved the the spatial problem. Now let's see if an artificial neural net can solve the same problem.

	~~Go ahead and click~~ the blue ''start'' button below; let it run for about 500 epochs (~5 seconds), then click pause.		Click the blue ''start'' button below; let it run for about 500 epochs (~5 seconds), then click pause.

	{{Clear}}		{{Clear}}
Line 27:		Line 24:
	\|width=1060px		\|width=1060px
	\|height=685px		\|height=685px
	\|margin=2px ~~-100px 25px -50px~~		\|margin=2px 2%
	\|padding=0px 0px 0px 0px		\|padding=0px 0px 0px 0px
	\|border-width=2px		\|border-width=2px
Line 36:		Line 33:
	{{Clear}}		{{Clear}}

	~~<br><br><br>~~		Is one neuron with input from ''a single feature'' (the dim1 data: AGE) performing well in the separation task? If so, an orange-colored background should have formed behind the orange dots, while a blue-colored background should have formed behind the blue dots. This colored surface gradient can be understood as the neural network's prediction value at that given coordinate. We will explore prediction values in more detail later on in the tutorial. First let's take a look at what the neural net is taking as inputs.
	~~<big>'''''Finished?'''''</big>~~

	~~How'd it do?~~ Is one neuron with input from ''a single feature'' (the dim1 data: AGE) performing well in the separation task? If so, an orange-colored background should have formed behind the orange dots, while a blue-colored background should have formed behind the blue dots. This colored surface gradient can be understood as the neural network's prediction value at that given coordinate. We will explore prediction values in more detail later on in the tutorial. First let's take a look at what the neural net is taking as inputs.

	~~<br>~~
	{{Clear}}		{{Clear}}

Line 47:		Line 40:
	----		----
	{{SmallBox\|display=block		{{SmallBox\|display=block
	\|float=~~none~~		\|float=right
	\|clear=none		\|clear=none
	\|width=250px		\|width=250px
	\|margin=5px ~~-10~~% ~~35px 10px~~		\|margin=5px 2%
	\|border-width=2px		\|border-width=2px
	\|border-radius=2px		\|border-radius=2px
Line 57:		Line 50:
	}}		}}

	~~{{Clear}}~~		To assess the performance, take a look at the input options in ''Figure-1''. There are a bunch of ''X'' variables with subscripts and superscripts, and next to each is a box with various color gradients. For now, let's focus on just two of those symbols, and what they mean to us...

	~~Take~~ a ~~close~~ look at the input options in Figure-1 ~~on the right~~. There are a bunch of ''X'' variables with subscripts and superscripts, and next to each is a box with various color gradients. For now, let's focus on just two of those symbols, and what they mean to us...

	~~{{Clear}}~~

	{\| class="wikitable" width=30% align=center		{\| class="wikitable" width=30% align=center
Line 72:		Line 61:
	\|colspan=2 style="background:#f7f7f7; border:3px solid #ffffff"\| SCORE		\|colspan=2 style="background:#f7f7f7; border:3px solid #ffffff"\| SCORE
	\|}		\|}

	~~{{Clear}}~~

	These are parsed such that subscripts (''X''<sub>1</sub> , ''X''<sub>2</sub> ,... ''X''<sub>i</sub> ) represent each predictor variable, like AGE and SCORE. As you can see, the first two input options ''X''<sub>1</sub> and ''X''<sub>2</sub> are just ''X''<sub>AGE</sub> and ''X''<sub>SCORE</sub>. Note that since ''X''<sub>1</sub> is plotted on the x-axis, it has a color gradient that changes horizontally, but is constant in the vertical dimension. Conversely the ''X''<sub>2</sub> feature plotted on the y-axis has a vertical color gradient. To clarify why this happens...		These are parsed such that subscripts (''X''<sub>1</sub> , ''X''<sub>2</sub> ,... ''X''<sub>i</sub> ) represent each predictor variable, like AGE and SCORE. As you can see, the first two input options ''X''<sub>1</sub> and ''X''<sub>2</sub> are just ''X''<sub>AGE</sub> and ''X''<sub>SCORE</sub>. Note that since ''X''<sub>1</sub> is plotted on the x-axis, it has a color gradient that changes horizontally, but is constant in the vertical dimension. Conversely the ''X''<sub>2</sub> feature plotted on the y-axis has a vertical color gradient. To clarify why this happens...
Line 84:		Line 71:

	{{SmallBox\|display=block		{{SmallBox\|display=block
	\|float=~~right~~		\|float=left
	\|clear=none		\|clear=none
	\|width=420px		\|width=420px
	\|margin=~~25px -10~~% ~~5px 10px~~		\|margin=15px 5%
	\|border-width=2px		\|border-width=2px
	\|border-radius=2px		\|border-radius=2px
Line 94:		Line 81:
	}}		}}

	~~{{Clear}}~~

	This isn't a shortcoming of having just one single neuron in the entire network. You could add as many neurons and layers as you want (go ahead and try it)...... if the network only gets input about one feature dimension, the output will be the same, whether there is 1 neuron, or 1 billion. To realize this fact, pretend you can only see the dots as they are plotted in along the number line in 1D (in Figure 2); if we were unable to see the 2D cluster clouds above that line, the billions of neurons in our brain would tell us to draw the classification line in basically the same place as that one single neuron in our artificial neural net. This is a very interesting concept worth noting: neural net classifiers can fail for two very different reasons.		This isn't a shortcoming of having just one single neuron in the entire network. You could add as many neurons and layers as you want (go ahead and try it)...... if the network only gets input about one feature dimension, the output will be the same, whether there is 1 neuron, or 1 billion. To realize this fact, pretend you can only see the dots as they are plotted in along the number line in 1D (in Figure 2); if we were unable to see the 2D cluster clouds above that line, the billions of neurons in our brain would tell us to draw the classification line in basically the same place as that one single neuron in our artificial neural net. This is a very interesting concept worth noting: neural net classifiers can fail for two very different reasons.

	(1) The neural network itself might be ill-formulated in such a way that, no matter how much information you provide, it cannot seem to learn to solve the classification problem. (2) On the other hand, you might have implemented an apposite deep neural network; yet if the input data is insufficient to solve the classification problem, it will appear to you that this potentially very good neural network performs like garbage. <br>		(1) The neural network itself might be ill-formulated in such a way that, no matter how much information you provide, it cannot seem to learn to solve the classification problem. (2) On the other hand, you might have implemented an apposite deep neural network; yet if the input data is insufficient to solve the classification problem, it will appear to you that this potentially very good neural network performs like garbage. <br>

	~~{{Quote\|A perfectly capable neural net might end up performing like garbage because, with the info you were feeding in, it never stood a chance.\|source=anonymous social worker}} <br><br>~~

	With that said, there are ways to help prevent that later scenario from happening. These involve doing things like you see for the rest of the input features. The next page will discuss the full set of possible network inputs we have here, which includes...		With that said, there are ways to help prevent that later scenario from happening. These involve doing things like you see for the rest of the input features. The next page will discuss the full set of possible network inputs we have here, which includes...

Bradley Monk at 07:18, 4 May 2020

2020-05-04T07:18:33Z

← Older revision		Revision as of 00:18, 4 May 2020
Line 5:		Line 5:
	4. [[Neural Nets 4\|Network Outputs]]<br>		4. [[Neural Nets 4\|Network Outputs]]<br>
	}}		}}

			{{Clear}}

	Let's just dive right in... Below I've embedded a neural network classifier rendered using [https://www.tensorflow.org Tensorflow][http://playground.tensorflow.org Playground]. There are a variety of knobs and buttons on the interface; as we move along, more of these options will become available. Don't worry though, all these will be explained in detail, in due time. For now though, let's define our primary goal throughout this tutorial: '''classification'''.		Let's just dive right in... Below I've embedded a neural network classifier rendered using [https://www.tensorflow.org Tensorflow][http://playground.tensorflow.org Playground]. There are a variety of knobs and buttons on the interface; as we move along, more of these options will become available. Don't worry though, all these will be explained in detail, in due time. For now though, let's define our primary goal throughout this tutorial: '''classification'''.
Line 16:		Line 18:

	Go ahead and click the blue ''start'' button below; let it run for about 500 epochs (~5 seconds), then click pause.		Go ahead and click the blue ''start'' button below; let it run for about 500 epochs (~5 seconds), then click pause.

			{{Clear}}

	===Tensorflow===		===Tensorflow===
	{{SmallBox\|display=block		{{SmallBox\|display=block
	\|float=~~left~~		\|float=none
	\|clear=none		\|clear=none
	\|width=1060px		\|width=1060px
Line 29:		Line 33:
	\|{{#widget:Tensorflow1}}		\|{{#widget:Tensorflow1}}
	}}		}}

			{{Clear}}

	<br><br><br>		<br><br><br>
Line 36:		Line 42:

	<br>		<br>
			{{Clear}}

	===Inputs===		===Inputs===
	----		----
	{{SmallBox\|display=block		{{SmallBox\|display=block
	\|float=~~right~~		\|float=none
	\|clear=none		\|clear=none
	\|width=250px		\|width=250px
Line 49:		Line 56:
	\| Figure 1		\| Figure 1
	}}		}}

			{{Clear}}

	Take a close look at the input options in Figure-1 on the right. There are a bunch of ''X'' variables with subscripts and superscripts, and next to each is a box with various color gradients. For now, let's focus on just two of those symbols, and what they mean to us...		Take a close look at the input options in Figure-1 on the right. There are a bunch of ''X'' variables with subscripts and superscripts, and next to each is a box with various color gradients. For now, let's focus on just two of those symbols, and what they mean to us...

			{{Clear}}

	{\| class="wikitable" width=30% align=center		{\| class="wikitable" width=30% align=center
Line 61:		Line 72:
	\|colspan=2 style="background:#f7f7f7; border:3px solid #ffffff"\| SCORE		\|colspan=2 style="background:#f7f7f7; border:3px solid #ffffff"\| SCORE
	\|}		\|}

			{{Clear}}

	These are parsed such that subscripts (''X''<sub>1</sub> , ''X''<sub>2</sub> ,... ''X''<sub>i</sub> ) represent each predictor variable, like AGE and SCORE. As you can see, the first two input options ''X''<sub>1</sub> and ''X''<sub>2</sub> are just ''X''<sub>AGE</sub> and ''X''<sub>SCORE</sub>. Note that since ''X''<sub>1</sub> is plotted on the x-axis, it has a color gradient that changes horizontally, but is constant in the vertical dimension. Conversely the ''X''<sub>2</sub> feature plotted on the y-axis has a vertical color gradient. To clarify why this happens...		These are parsed such that subscripts (''X''<sub>1</sub> , ''X''<sub>2</sub> ,... ''X''<sub>i</sub> ) represent each predictor variable, like AGE and SCORE. As you can see, the first two input options ''X''<sub>1</sub> and ''X''<sub>2</sub> are just ''X''<sub>AGE</sub> and ''X''<sub>SCORE</sub>. Note that since ''X''<sub>1</sub> is plotted on the x-axis, it has a color gradient that changes horizontally, but is constant in the vertical dimension. Conversely the ''X''<sub>2</sub> feature plotted on the y-axis has a vertical color gradient. To clarify why this happens...
Line 67:		Line 80:

	When the neural net only gets input about a single feature of each person in the dataset, its synaptic weights will only adapt output along that one dimension. Thus, if for example the network sees that a person is 3 years above the dataset average (considering the data has been ''mean deviated'' and centered), it won't matter what that person's cognitive SCORE was (since the neural net doesn't have access to that info), the network will always make the same guess for anyone 3 years above average age. This is why color is constant at ''x''=3 for any ''y'' value.		When the neural net only gets input about a single feature of each person in the dataset, its synaptic weights will only adapt output along that one dimension. Thus, if for example the network sees that a person is 3 years above the dataset average (considering the data has been ''mean deviated'' and centered), it won't matter what that person's cognitive SCORE was (since the neural net doesn't have access to that info), the network will always make the same guess for anyone 3 years above average age. This is why color is constant at ''x''=3 for any ''y'' value.

			{{Clear}}

	{{SmallBox\|display=block		{{SmallBox\|display=block
Line 78:		Line 93:
	\| Figure 2		\| Figure 2
	}}		}}

			{{Clear}}

	This isn't a shortcoming of having just one single neuron in the entire network. You could add as many neurons and layers as you want (go ahead and try it)...... if the network only gets input about one feature dimension, the output will be the same, whether there is 1 neuron, or 1 billion. To realize this fact, pretend you can only see the dots as they are plotted in along the number line in 1D (in Figure 2); if we were unable to see the 2D cluster clouds above that line, the billions of neurons in our brain would tell us to draw the classification line in basically the same place as that one single neuron in our artificial neural net. This is a very interesting concept worth noting: neural net classifiers can fail for two very different reasons.		This isn't a shortcoming of having just one single neuron in the entire network. You could add as many neurons and layers as you want (go ahead and try it)...... if the network only gets input about one feature dimension, the output will be the same, whether there is 1 neuron, or 1 billion. To realize this fact, pretend you can only see the dots as they are plotted in along the number line in 1D (in Figure 2); if we were unable to see the 2D cluster clouds above that line, the billions of neurons in our brain would tell us to draw the classification line in basically the same place as that one single neuron in our artificial neural net. This is a very interesting concept worth noting: neural net classifiers can fail for two very different reasons.
Line 86:		Line 103:

	With that said, there are ways to help prevent that later scenario from happening. These involve doing things like you see for the rest of the input features. The next page will discuss the full set of possible network inputs we have here, which includes...		With that said, there are ways to help prevent that later scenario from happening. These involve doing things like you see for the rest of the input features. The next page will discuss the full set of possible network inputs we have here, which includes...


			{{Clear}}

	{\| class="wikitable" width=30% align=center		{\| class="wikitable" width=30% align=center
Line 113:		Line 133:


	~~<br>~~		{{Clear}}
	{{SmallBox\|'''[[Neural Nets 2\|Continue to Neural Nets Tutorial Page 2]]'''}}		{{SmallBox\|'''[[Neural Nets 2\|Continue to Neural Nets Tutorial Page 2]]'''}}

	<!-- <btn data-toggle="tooltip">Neural Nets 2</btn> -->		<!-- <btn data-toggle="tooltip">Neural Nets 2</btn> -->

Bradley Monk at 22:17, 22 January 2018

2018-01-22T22:17:52Z

← Older revision		Revision as of 15:17, 22 January 2018
Line 6:		Line 6:
	}}		}}

	Let's just dive right in... Below I've embedded a neural network classifier rendered using [https://www.tensorflow.org Tensorflow][http://playground.tensorflow.org Playground]. There are a variety of knobs and buttons on the interface; as we move along, more of these options will become available. Don't worry though, all these will be explained in detail, in due time. For now though, let's define our primary goal throughout this tutorial: '''~~categorization~~'''		Let's just dive right in... Below I've embedded a neural network classifier rendered using [https://www.tensorflow.org Tensorflow][http://playground.tensorflow.org Playground]. There are a variety of knobs and buttons on the interface; as we move along, more of these options will become available. Don't worry though, all these will be explained in detail, in due time. For now though, let's define our primary goal throughout this tutorial: '''classification'''.

	Our primary task is to train neural nets to classify items into categories, based on some limited information. Like fruit or vegetable; or undergrad major; or Alzheimer's Disease patient (CASE) or control participant (CTRL). In the Tensorflow playground below, you can see a bunch of orange and blue dots. Instead of simply thinking about these as dots at arbitrary spatial coordinates, it will be helpful to think of these as representing people in a clinical study. Let's define the blue dots are patients from the CASE group, and orange dots are CTRL participants. What is our 'limited information' about them? Let's say we have collected information about their age and their score on a dementia screening exam (scores represent number of items forgotten). So the first thing we'd probably want to do is make a scatter plot of these two variables. Let's define their respective dimensions on the plot axes as:		Our primary task is to train neural nets to classify items into categories, based on some limited information. Like fruit or vegetable; or undergrad major; or Alzheimer's Disease patient (CASE) or control participant (CTRL). In the Tensorflow playground below, you can see a bunch of orange and blue dots. Instead of simply thinking about these as dots at arbitrary spatial coordinates, it will be helpful to think of these as representing people in a clinical study. Let's define the blue dots are patients from the CASE group, and orange dots are CTRL participants. What is our 'limited information' about them? Let's say we have collected information about their age and their score on a dementia screening exam (scores represent number of items forgotten). So the first thing we'd probably want to do is make a scatter plot of these two variables. Let's define their respective dimensions on the plot axes as:

Bradley Monk at 21:44, 22 January 2018

2018-01-22T21:44:46Z

← Older revision		Revision as of 14:44, 22 January 2018
Line 1:		Line 1:
	<big>TUTORIAL ON MACHINE LEARNING AND NEURAL NETWORKS</big>{{SmallBox\|float=right\|clear=none\|margin=0px 0px 8px 18px\|width=170px\|font-size=13px\|Tutorial Pages\|txt-size=11px\|		<big>TUTORIAL ON MACHINE LEARNING AND NEURAL NETWORKS</big>{{SmallBox\|float=right\|clear=none\|margin=0px 0px 8px 18px\|width=170px\|font-size=13px\|Tutorial Pages\|txt-size=11px\|
	1. [[Neural Nets\|~~Tensorflow~~ Intro]]<br>		1. [[Neural Nets\|Intro]]<br>
	2. [[Neural Nets 2\|~~Neural Net Classifiers~~]]<br>		2. [[Neural Nets 2\|Network Inputs]]<br>
	3. [[Neural Nets 3\|~~DIY Machine Learning~~]]<br>		3. [[Neural Nets 3\|Network Activation]]<br>
	4. [[Neural Nets 4\|~~PCA   t-SNE~~]]<br>		4. [[Neural Nets 4\|Network Outputs]]<br>
	}}		}}

Bradley Monk at 21:43, 22 January 2018

2018-01-22T21:43:20Z

← Older revision		Revision as of 14:43, 22 January 2018
Line 8:		Line 8:
	Let's just dive right in... Below I've embedded a neural network classifier rendered using [https://www.tensorflow.org Tensorflow][http://playground.tensorflow.org Playground]. There are a variety of knobs and buttons on the interface; as we move along, more of these options will become available. Don't worry though, all these will be explained in detail, in due time. For now though, let's define our primary goal throughout this tutorial: '''categorization'''		Let's just dive right in... Below I've embedded a neural network classifier rendered using [https://www.tensorflow.org Tensorflow][http://playground.tensorflow.org Playground]. There are a variety of knobs and buttons on the interface; as we move along, more of these options will become available. Don't worry though, all these will be explained in detail, in due time. For now though, let's define our primary goal throughout this tutorial: '''categorization'''

	Our primary task is to train neural nets to classify items into categories, based on some limited information. Like fruit or vegetable; or undergrad major; or Alzheimer's Disease patient (CASE) or control participant (CTRL). In the Tensorflow playground below, you can see a bunch of orange and blue dots. Instead of simply thinking about these as dots or arbitrary spatial coordinates, ~~I think~~ it will be helpful to think of these as representing people in a clinical study. Let's define the blue dots are patients from the CASE group, and orange dots are CTRL participants. ~~Now what~~ is our 'limited information' about them? Let's say we have collected information about their age and their score on a ~~cognitive~~ exam. So the first thing we'd probably want to do is make a scatter plot of these two variables. Let's define their respective dimensions on the plot axes as:		Our primary task is to train neural nets to classify items into categories, based on some limited information. Like fruit or vegetable; or undergrad major; or Alzheimer's Disease patient (CASE) or control participant (CTRL). In the Tensorflow playground below, you can see a bunch of orange and blue dots. Instead of simply thinking about these as dots at arbitrary spatial coordinates, it will be helpful to think of these as representing people in a clinical study. Let's define the blue dots are patients from the CASE group, and orange dots are CTRL participants. What is our 'limited information' about them? Let's say we have collected information about their age and their score on a dementia screening exam (scores represent number of items forgotten). So the first thing we'd probably want to do is make a scatter plot of these two variables. Let's define their respective dimensions on the plot axes as:

	* x-axis \| dim1 \| current age ('''''AGE''''')		* x-axis \| dim1 \| current age ('''''AGE''''')
Line 62:		Line 62:
	\|}		\|}

	These are parsed such that subscripts (''X''<sub>1</sub> , ''X''<sub>2</sub> ,... ''X''<sub>i</sub> ) represent each predictor variable, like AGE and SCORE. As you can see, the first two input options ''X''<sub>1</sub> and ''X''<sub>2</sub> are just ''X''<sub>AGE</sub> and ''X''<sub>SCORE</sub>.		These are parsed such that subscripts (''X''<sub>1</sub> , ''X''<sub>2</sub> ,... ''X''<sub>i</sub> ) represent each predictor variable, like AGE and SCORE. As you can see, the first two input options ''X''<sub>1</sub> and ''X''<sub>2</sub> are just ''X''<sub>AGE</sub> and ''X''<sub>SCORE</sub>. Note that since ''X''<sub>1</sub> is plotted on the x-axis, it has a color gradient that changes horizontally, but is constant in the vertical dimension. Conversely the ''X''<sub>2</sub> feature plotted on the y-axis has a vertical color gradient. To clarify why this happens...

	Note that since ''X''<sub>1</sub> is plotted on the x-axis, it has a vertical color gradient; since ''X''<sub>2</sub> is plotted on the y-axis, it has a horizontal gradient. This might seem backwards, so to clarify: if the only thing we know about these study participants is their AGE, ''X''<sub>1</sub>, we can only make a 1-D plot with each person's age along the x-axis, such that [ x = ''AGE''<sub>i</sub> , y = 0 ].		If the only thing we know about these study participants is their AGE, ''X''<sub>1</sub>, we can only make a 1-D plot with each person's age along the x-axis, such that [ x = ''AGE''<sub>i</sub> , y = 0 ]. If you take a look at Figure-2, it should be clear that when information is collapsed onto its single dimension and plotted along the x-axis, the best line we can draw to separate the dim-1 data will be orthogonal to the x-axis (a vertical line). As you move horizontally along the x-axis your categorical guess will likely change, along with the confidence in that guess, which is precisely what is being represented by the color gradient. On the other hand, knowing nothing about exam score, moving up and down on the y-axis will have no effect on your decision, which is why color is constant in the y-dimension.

	If you take a look at Figure-2, it should be clear that when information is collapsed onto its single dimension and plotted along the x-axis, the best line we can draw to separate the dim-1 data will be orthogonal to the x-axis. ~~In plain terms~~, ~~if given a pencil~~, ~~and asked to separate~~ the ~~orange~~ and ~~blue dots plotted near~~ the ~~number line~~, ~~it'd best be a vertical line!~~

	When the neural net only gets input about a single feature of each person in the dataset, its synaptic weights will only adapt output along that one dimension. Thus, if for example the network sees that a person is 3 years above the dataset average (considering the data has been ''mean deviated'' and centered), it won't matter what that person's cognitive SCORE was (since the neural net doesn't have access to that info), the network will always make the same guess for anyone 3 years above average age. This is why color is constant at ''x''=3 for any ''y'' value.		When the neural net only gets input about a single feature of each person in the dataset, its synaptic weights will only adapt output along that one dimension. Thus, if for example the network sees that a person is 3 years above the dataset average (considering the data has been ''mean deviated'' and centered), it won't matter what that person's cognitive SCORE was (since the neural net doesn't have access to that info), the network will always make the same guess for anyone 3 years above average age. This is why color is constant at ''x''=3 for any ''y'' value.
Line 83:		Line 81:
	This isn't a shortcoming of having just one single neuron in the entire network. You could add as many neurons and layers as you want (go ahead and try it)...... if the network only gets input about one feature dimension, the output will be the same, whether there is 1 neuron, or 1 billion. To realize this fact, pretend you can only see the dots as they are plotted in along the number line in 1D (in Figure 2); if we were unable to see the 2D cluster clouds above that line, the billions of neurons in our brain would tell us to draw the classification line in basically the same place as that one single neuron in our artificial neural net. This is a very interesting concept worth noting: neural net classifiers can fail for two very different reasons.		This isn't a shortcoming of having just one single neuron in the entire network. You could add as many neurons and layers as you want (go ahead and try it)...... if the network only gets input about one feature dimension, the output will be the same, whether there is 1 neuron, or 1 billion. To realize this fact, pretend you can only see the dots as they are plotted in along the number line in 1D (in Figure 2); if we were unable to see the 2D cluster clouds above that line, the billions of neurons in our brain would tell us to draw the classification line in basically the same place as that one single neuron in our artificial neural net. This is a very interesting concept worth noting: neural net classifiers can fail for two very different reasons.

	(1) The neural network itself might be ill-formulated in such a way that, no matter how much information you provide, it cannot seem to learn to solve the classification problem. (2) On the other hand, you might have ~~coded the finest most optimized~~ deep neural network ~~ever conceived~~; yet if the input data is insufficient to solve the classification problem, it will appear to you that ~~your creation~~ performs like garbage. <br>		(1) The neural network itself might be ill-formulated in such a way that, no matter how much information you provide, it cannot seem to learn to solve the classification problem. (2) On the other hand, you might have implemented an apposite deep neural network; yet if the input data is insufficient to solve the classification problem, it will appear to you that this potentially very good neural network performs like garbage. <br>

	{{Quote\|A perfectly ~~young and~~ capable neural net might end up performing like garbage because, with the info you were feeding in, it never stood a chance.\|source=anonymous social worker}} <br><br>		{{Quote\|A perfectly capable neural net might end up performing like garbage because, with the info you were feeding in, it never stood a chance.\|source=anonymous social worker}} <br><br>

	With that said, there are ways to help prevent that later scenario from happening. These involve doing things like you see for the rest of the input features. ~~Again,~~ the full set of possible network inputs here includes:		With that said, there are ways to help prevent that later scenario from happening. These involve doing things like you see for the rest of the input features. The next page will discuss the full set of possible network inputs we have here, which includes...

	{\| class="wikitable" width=30% align=center		{\| class="wikitable" width=30% align=center
Line 114:		Line 112:
	\|}		\|}



	~~{{Clear}}~~
	~~<br><br><br>~~

	~~===Outputs===~~
	~~----~~

	More directly, it is the value spit-out by the activation function of the 'output layer'. Here, since we only have a single layer, our hidden 'hidden layer' and 'output layer' are one in the same. The output function of our neuron is known as the '''tanh''' function.

	The tanh function is an extremely common choice for an output function in artificial neural network machine learning frameworks because it yields a nice sigmoid shape, and no matter the magnitude of its inputs, the output from the tanh function is bounded between { 0 : 1}. These are very desirable properties for neural net nodes. Here you see the tanh function evaluated across various x-dim inputs...


	~~<br><br><br><br>~~
	~~{{Clear}}~~
	~~[[File: Tanh.png\|thumb\|500px\|left\|see [http://reference.wolfram.com/language/ref/Tanh.html tanh on wolfram alpha] for many details about tanh function.]]~~
	~~{{Clear}}~~

	Tanh produces a sigmoid output over the range {-2 : 2}, and automatically evaluates to exact values when its argument is the natural logarithm. Speaking of the natural log, that is another very common choice of output function for the same reasons as tanh.

	For now, let's not belabor the point that our neuron (and in going forward, all our neurons) are using the tanh function. Maybe just keep this in mind if you're wondering what sorts of numbers are travelling along the axons of these neurons, and ultimately those colored gradients underneath the dots.

	~~This tutorial continues on the next page. Don't worry about playing around too much with the TensorFlow GUI, there will be plenty of that on the next page, and those that follow.~~

	<br>		<br>

← Older revision		Revision as of 14:44, 22 January 2018
Line 1:		Line 1:
	<big>TUTORIAL ON MACHINE LEARNING AND NEURAL NETWORKS</big>{{SmallBox\|float=right\|clear=none\|margin=0px 0px 8px 18px\|width=170px\|font-size=13px\|Tutorial Pages\|txt-size=11px\|		<big>TUTORIAL ON MACHINE LEARNING AND NEURAL NETWORKS</big>{{SmallBox\|float=right\|clear=none\|margin=0px 0px 8px 18px\|width=170px\|font-size=13px\|Tutorial Pages\|txt-size=11px\|
	1. [[Neural Nets\|~~Tensorflow~~ Intro]]<br>		1. [[Neural Nets\|Intro]]<br>
	2. [[Neural Nets 2\|~~Neural Net Classifiers~~]]<br>		2. [[Neural Nets 2\|Network Inputs]]<br>
	3. [[Neural Nets 3\|~~DIY Machine Learning~~]]<br>		3. [[Neural Nets 3\|Network Activation]]<br>
	4. [[Neural Nets 4\|~~PCA   t-SNE~~]]<br>		4. [[Neural Nets 4\|Network Outputs]]<br>
	}}		}}