<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Leonid,<br>
<br>
Good question: How logic emerges from illogical neural firings?<br>
<br>
Many neural net researchers have a background in electrical
engineering or physics, but not much in computer science. To
understand how a new category of neural networks (DN) can do
abstraction, one needs to be familiar with the <br>
automata theory typically taught in computer science. Mathematical
logic (proposition logic, first order logic, second order logic,
etc.) is useful for understanding what formal logic means. But
mathematical logic is not sufficient to explain brain functions. <br>
Brain functions are not based on mathematical logic. <br>
<br>
Why? <br>
<br>
-John<br>
<br>
<br>
On 10/22/11 10:15 AM, leonid wrote:
<blockquote cite="mid:8D6BCA6916CA463CA63BF96D152976E9@jung"
type="cite">
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
<meta name="GENERATOR" content="MSHTML 8.00.6001.19154">
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><span class="890360614-22102011"><font
color="#0000ff" face="Arial" size="2">John,</font></span></div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><span class="890360614-22102011"></span> </div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><span class="890360614-22102011"><font
color="#0000ff" face="Arial" size="2">Thank you for the good
words.</font></span></div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><span class="890360614-22102011"></span> </div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><span class="890360614-22102011"><font
color="#0000ff" face="Arial" size="2">Michael,</font></span></div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><span class="890360614-22102011"></span> </div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><span class="890360614-22102011"><font
color="#0000ff" face="Arial" size="2">When you were at MIT
we were developing similar techniques. Now you rely on
logic. The problem with logic is: How it emerges from
illogical neural firings?</font></span></div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><span class="890360614-22102011"> </span></div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us">
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><span class="890360614-22102011"><font
color="#0000ff" face="Arial" size="2">We would be glad to
get you involved.</font></span></div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><span class="890360614-22102011"></span> </div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><span class="890360614-22102011"><font
color="#0000ff" face="Arial" size="2">Best</font></span></div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><span class="890360614-22102011"><font
color="#0000ff" face="Arial" size="2">Leonid</font></span></div>
</div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><span class="890360614-22102011"></span> </div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><span class="890360614-22102011"> </span></div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us">
<hr tabindex="-1">
</div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us"><font face="Tahoma" size="2"><b>From:</b> Juyang
Weng [<a class="moz-txt-link-freetext" href="mailto:weng@cse.msu.edu">mailto:weng@cse.msu.edu</a>] <br>
<b>Sent:</b> Thursday, October 20, 2011 8:09 PM<br>
<b>To:</b> leonid; <a class="moz-txt-link-abbreviated" href="mailto:leonid@seas.harvard.edu">leonid@seas.harvard.edu</a><br>
<b>Cc:</b> 'bmilist'; Michael I Jordan<br>
<b>Subject:</b> Re: BMI debate: Can we start to look at the
brain-mind from the entire system point of view?<br>
</font><br>
</div>
Hi Leonid,<br>
<br>
You provided a great list of items. I also agree that views from
top and views from bottom and many levels in the middle are all
needed. <br>
<br>
I provide a key divide between symbolic representations and
emergent representations for us to start:<br>
Michael I. Jordan correctly stated at the David Rumelhart Memorial
Plenary Talk, IJCNN 2011 that neural networks do not abstract
well. He talked about symbolic models instead.<br>
<br>
(a) What did he mean by neural networks do not abstract well?<br>
(b) Why did a researcher who has done neural nets before talked
about instead symbolic models at a major neural network conference
in honor of a neural network pioneer?<br>
<br>
I give him a CC just in case he is interested.<br>
<br>
-John<br>
<br>
On 10/20/11 4:45 PM, leonid wrote:
<blockquote cite="mid:5B22AABFC3A044BB931CBE2FE81CBEBF@jung"
type="cite">
<meta name="GENERATOR" content="MSHTML 8.00.6001.19154">
<div dir="ltr" align="left"><span class="546400720-20102011"><font
color="#0000ff" face="Arial" size="2">Hello to everybody</font></span></div>
<div dir="ltr" align="left"><span class="546400720-20102011"></span> </div>
<div dir="ltr" align="left"><span class="546400720-20102011"><font
color="#0000ff" face="Arial" size="2">Views from the top
and views from the bottom must be combined. Physics is a
successful science because it concentrates on fundamental
principles, and then proceeds to experimentally verifiable
predictions. There are first principles operating in the
mind and brain. There are first principles at every level
of organization of matter. For the brain-mind I would list
few:</font></span></div>
<div dir="ltr" align="left"><span class="546400720-20102011"></span> </div>
<div dir="ltr" align="left"><span class="546400720-20102011"><font
color="#0000ff" face="Arial" size="2">- hierarchical
organization of mental representations</font></span></div>
<div dir="ltr" align="left"><span class="546400720-20102011"><font
color="#0000ff" face="Arial" size="2">- bottom-up and
top-down signal interactions</font></span></div>
<div dir="ltr" align="left"><span class="546400720-20102011"><font
color="#0000ff" face="Arial" size="2">- instinctual drives
measuring vital organismic parameters and communicating
results to decision-making mechanisms</font></span></div>
<div dir="ltr" align="left"><span class="546400720-20102011"><font
color="#0000ff" face="Arial" size="2">- emotions serving
as neural signals communicating (as above) satisfaction or
dissatisfaction of instinctual drives</font></span></div>
<div dir="ltr" align="left"><span class="546400720-20102011"><font
color="#0000ff" face="Arial" size="2">(the two principles
above are discovered by Grossberg-Levine theory)</font></span></div>
<div><span class="546400720-20102011"></span><font face="Arial"><font
color="#0000ff"><font size="2">-<span
class="546400720-20102011"> </span>the most important
instinctual drive is<span class="546400720-20102011">
the "instinct for knowledge." It drives matching of
bottom-up and top-down signals so that mental
representations are similar to reality (it is more
important than</span> survival or procreation<span
class="546400720-20102011">, because survival is not
possible without perception and cognition)</span></font></font></font></div>
<div><font face="Arial"><font color="#0000ff"><font size="2"><span
class="546400720-20102011">- "vague-to-crisp" process
evolves mental representations to match reality, this
is the operation of the instinct for knowledge, which
overcomes exponential complexity</span></font></font></font></div>
<div><font face="Arial"><font color="#0000ff"><font size="2"><span
class="546400720-20102011">- special emotions
correspond to the knowledge instinct; these are not
basic, but aesthetic emotions explaining higher human
cognitive abilities from understanding and cognition
of objects an situations to abstract concepts, and
higher up to "mysterious" meanings of life and
emotions of the beautiful</span></font></font></font></div>
<div><font face="Arial"><font color="#0000ff"><font size="2"><span
class="546400720-20102011">- we need to understand the
difference between language and cognitive
representations, and how they interact</span></font></font></font></div>
<div><font face="Arial"><font color="#0000ff"><font size="2"><span
class="546400720-20102011">- how the hierarchy of
cognition is learned by every human child</span></font></font></font></div>
<div><font face="Arial"><font color="#0000ff"><font size="2"><span
class="546400720-20102011">- what are emotions of
cognitive dissonances, and how human evolution
overcame these (most likely - music)</span></font></font></font></div>
<div><font face="Arial"><font color="#0000ff"><font size="2"><span
class="546400720-20102011">- some of the above are
described by mathematical models - this is a must</span></font></font></font></div>
<div><font face="Arial"><font color="#0000ff"><font size="2"><span
class="546400720-20102011">- some of the above is
experimentally confirmed - this is a must</span></font></font></font></div>
<div><font face="Arial"><font color="#0000ff"><font size="2"><span
class="546400720-20102011"></span></font></font></font> </div>
<div><font face="Arial"><font color="#0000ff"><font size="2"><span
class="546400720-20102011">What did I miss ? (possibly
something) - please add fundamental laws, explaining a
lot from few assumptions, mathematical models of these
processes making experimental predictions, and finally
experimental tests of all of the above.</span></font></font></font></div>
<div><font face="Arial"><font color="#0000ff"><font size="2"><span
class="546400720-20102011"></span></font></font></font> </div>
<div><font face="Arial"><font color="#0000ff"><font size="2"><span
class="546400720-20102011">Best</span></font></font></font></div>
<div><font face="Arial"><font color="#0000ff"><font size="2"><span
class="546400720-20102011">Leonid</span></font></font></font></div>
<div> </div>
<div> </div>
<div><font face="Arial"><font color="#0000ff"><font size="2"><span
class="546400720-20102011"></span></font></font></font> </div>
<div><br>
</div>
<div dir="ltr" class="OutlookMessageHeader" align="left"
lang="en-us">
<hr tabindex="-1"> <font face="Tahoma" size="2"><b>From:</b>
Juyang Weng [<a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="mailto:weng@cse.msu.edu">mailto:weng@cse.msu.edu</a>]
<br>
<b>Sent:</b> Thursday, October 20, 2011 2:08 PM<br>
<b>To:</b> bmilist<br>
<b>Subject:</b> BMI debate: Can we start to look at the
brain-mind from the entire system point of view?<br>
</font><br>
</div>
Dear all: <br>
<br>
After talking to some of my colleagues, we here kick of a BMI
debate via this email on <a class="moz-txt-link-abbreviated"
href="mailto:bmi@lists.cse.msu.edu" moz-do-not-send="true">bmi@lists.cse.msu.edu</a>.<br>
Many of you on this anonymous list told me that they are
interested and want to be posted. However, we will use this<br>
anonymous list sparely. If you want to keep posted about this
debate and other BMI activities, sign on bmi mailing list <br>
at <a class="moz-txt-link-freetext"
href="http://lists.cse.msu.edu/cgi-bin/mailman/listinfo/bmi"
moz-do-not-send="true">http://lists.cse.msu.edu/cgi-bin/mailman/listinfo/bmi</a>
or simply Google it with key words like "BMI mailing list MSU".<br>
Once you receive email from the mailing list, you can post
simply via reply. BMI mailing list is a moderated list to
avoid<br>
unrelated emails. If there are sufficient interest, BMI might
host a live web debate in a few weeks. Post your views!<br>
<br>
The following email I sent to Dave Touretzky is the kick-off for
the BMI debates. I will provide some interesting examples soon.<br>
<br>
On 10/20/11 12:59 PM, Juyang Weng wrote:
<blockquote cite="mid:4EA05385.5050906@cse.msu.edu" type="cite">Hi
Dave,<br>
<br>
I read some of your papers about hippocampus, which are very
interesting. Let me inject some basic but probably very
controversial ideas you probably will reject. If you do not
mind, I will post this discussion to the BMI mailing list.
The main purpose is to attract more talented researchers to
this important brain-mind subject. <br>
<br>
How about looking at the brain from a top system point of
view? I believe that top (but detailed) theory is powerful,
since the brain basically does signal processing (not in the
traditional sense). Maybe with this view, our future design
of experiments could be more productive? Let me start from
one example:<br>
<br>
One of your papers is "Synaptic Learning Models of Map
Separation in the Hippocampus", <i>Neurocomputing</i>, <b>32</b>:379-384,
2000. The co-authors wrote: "If the perforant path
projection to CA3 functions as a pattern completion mechanism,
and the DG projection via the mossy fibers performs pattern
separation (O'Reilly and McClelland, 1994), then ..."<br>
<br>
My new perspectives about the brain benefited from such local
views, but I think that such local views can also benefit from
the entire brain-mind point of view, in the sense of a giant
Finite Automaton (FA). This brain FA is not handcrafted, but
rather developed, since all phenotypes emerge from a single
cell (zygote). So, I model such a developmental FA as the
Developmental Network (DN). Then, the Hippocampus is simply a
very small part of a giant DN. According to how the DN works,
I predict the following: If we focus on a small part (e.g.,
Hippocampus) of this DN, we definitely will get hopelessly
lost, like a hiker in a forest without a global map. He can
see some local phenomena from where he stands, but he did not
see the entire forest. <br>
<br>
Focused, per-phenomenon discoveries have been prevailing in
the brain science literature in the modern science, with few
exceptions (Charles Darwin is one). This is probably because
only such papers can be accepted and funded in the modern
time. Although those phenomena are useful, they are piece
meals. Now, there seem to have enough pieces to put the grand
puzzle together. I have established what a DN can do in real
time, by modeling the brain-mind from the entire FA (DN) point
of view. Since all pieces of DN seem to fit what we know
about the brain science, the brain should not be less
efficient than a DN.<br>
<br>
You can say that this is just fantasy, but I have a series of
rigorous proofs. <br>
<br>
Daniel M. Wolpert said at SfN 2009 that the over 1400-page
long volume of "Principles of Neural Science" by Kandel et al.
could be much condensed if we could model the entire brain in
computational theory. I hope that the DN theory can help
that condensing process. <br>
<br>
A major infrastructure problem is that what I talked about
above spans at least 6 disciplines. Meaningful conversations
are extremely difficult. If you feel angry or insulted by my
above text, I feel that it is partially because of this huge
divide. <br>
<br>
I am giving a CC to Jay, as his work was cited. <br>
<br>
Best regards,<br>
<br>
-John</blockquote>
<br>
-John<br>
<pre class="moz-signature" cols="72">--
--
Juyang (John) Weng, Professor
Department of Computer Science and Engineering
MSU Cognitive Science Program and MSU Neuroscience Program
3115 Engineering Building
Michigan State University
East Lansing, MI 48824 USA
Tel: 517-353-4388
Fax: 517-432-1061
Email: <a class="moz-txt-link-abbreviated" href="mailto:weng@cse.msu.edu" moz-do-not-send="true">weng@cse.msu.edu</a>
URL: <a class="moz-txt-link-freetext" href="http://www.cse.msu.edu/%7Eweng/" moz-do-not-send="true">http://www.cse.msu.edu/~weng/</a>
----------------------------------------------
</pre>
<hr noshade="noshade" size="1">
<p class="avgcert" color="#000000" align="left">No virus found
in this message.<br>
Checked by AVG - <a href="http://www.avg.com"
moz-do-not-send="true">www.avg.com</a><br>
Version: 10.0.1411 / Virus Database: 1522/3963 - Release Date:
10/20/11</p>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
--
Juyang (John) Weng, Professor
Department of Computer Science and Engineering
MSU Cognitive Science Program and MSU Neuroscience Program
3115 Engineering Building
Michigan State University
East Lansing, MI 48824 USA
Tel: 517-353-4388
Fax: 517-432-1061
Email: <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:weng@cse.msu.edu">weng@cse.msu.edu</a>
URL: <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.cse.msu.edu/%7Eweng/">http://www.cse.msu.edu/~weng/</a>
----------------------------------------------
</pre>
<hr noshade="noshade" size="1">
<p class="avgcert" color="#000000" align="left">No virus found in
this message.<br>
Checked by AVG - <a moz-do-not-send="true"
href="http://www.avg.com">www.avg.com</a><br>
Version: 10.0.1411 / Virus Database: 1522/3963 - Release Date:
10/20/11</p>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
--
Juyang (John) Weng, Professor
Department of Computer Science and Engineering
MSU Cognitive Science Program and MSU Neuroscience Program
3115 Engineering Building
Michigan State University
East Lansing, MI 48824 USA
Tel: 517-353-4388
Fax: 517-432-1061
Email: <a class="moz-txt-link-abbreviated" href="mailto:weng@cse.msu.edu">weng@cse.msu.edu</a>
URL: <a class="moz-txt-link-freetext" href="http://www.cse.msu.edu/~weng/">http://www.cse.msu.edu/~weng/</a>
----------------------------------------------
</pre>
</body>
</html>