human
Write me a Python program to find good values given the following:
I am working on the file "SYDE 556 Lecture 9 Action Selection.ipynb".
The first task is:
Much less memory, but it's still there
And slower to respond to changes
Note that this speed is dependent on $e$, $i$, and the time constant of the neurotransmitter used

Can be hard to find good values

And this gets harder to balance as the number of actions increases

Also hard to balance for a wide range of $Q$ values
(Does it work for $Q$=[0.9, 0.9, 0.95, 0.9] and $Q$=[0.2, 0.2, 0.25, 0.2]?)



But this is still a pretty standard approach

Nice and easy to get working for special cases
Don't really need the NEF (if you're willing to assume non-realistic non-spiking neurons)
(Although really, if you're not looking for biological realism, why not just compute the max function?)


Example: OReilly, R.C. (2006). Biologically Based Computational Models of High-Level Cognition. Science, 314, 91-94.

Leabra


They tend to use a "kWTA" (k-Winners Take All) approach in their models

Set up inhibition so that only $k$ neurons will be active
But since that's complex to do, just do the math instead of doing the inhibition
We think that doing it their way means that the dynamics of the model will be wrong (i.e. all the effects we saw above are being ignored).


Any other options?

Biology

Let's look at the biology
Where is this action selection in the brain?
General consensus: the basal ganglia

<img src="files/lecture_selection/basal_ganglia.jpg" width="500">

Pretty much all of cortex connects in to this area (via the striatum)

Output goes to the thalamus, the central routing system of the brain

Disorders of this area of the brain cause problems controlling actions:

Parkinson's disease
Neurons in the substantia nigra die off
Extremely difficult to trigger actions to start
Usually physical actions.
as disease progresses and more of the SNc is gone, can get cognitive effects too

Huntington's disease
Neurons in the striatum die off
Actions are triggered inappropriately (disinhibition)
Small uncontrollable movements
Trouble sequencing cognitive actions too



Also heavily implicated in reinforcement...
Can you write Python program for it?
assistant
#Here is Python program to find good values
.

%pylab inline
mm=1
mp=1
me=1
mg=1

#connection strengths from original model
ws=1
wt=1
wm=1
wg=1
wp=0.9
we=0.3

#neuron lower thresholds for various populations
e=0.2  
ep=-0.25
ee=-0.2
eg=-0.2

le=0.2
lg=0.2

D = 10
tau_ampa=0.002
tau_gaba=0.008
N = 50
radius = 1.5

import nengo
from nengo.dists import Uniform

model = nengo.Network('Basal Ganglia', seed=4)

with model:
    stim = nengo.Node([0]*D)

    StrD1 = nengo.networks.EnsembleArray(N, n_ensembles=D, intercepts=Uniform(e,1), 
                                 encoders=Uniform(1,1), radius=radius)
    StrD2 = nengo.networks.EnsembleArray(N, n_ensembles=D, intercepts=Uniform(e,1), 
                                 encoders=Uniform(1,1), radius=radius)
    STN = nengo.networks.EnsembleArray(N, n_ensembles=D, intercepts=Uniform(ep,1), 
                                 encoders=Uniform(1,1), radius=radius)
    GPi = nengo.networks.EnsembleArray(N, n_ensembles=D, intercepts=Uniform(eg,1), 
                                 encoders=Uniform(1,1), radius=radius)
    GPe = nengo.networks.EnsembleArray(N, n_ensembles=D, intercepts=Uniform(ee,1), 
                                 encoders=Uniform(1,1), radius=radius)

    nengo.Connection(stim, StrD1.input, transform=ws*(1+lg), synapse=tau_ampa)
    nengo.Connection(stim, StrD2.input, transform=ws*(1-le), synapse=tau_ampa)
    nengo.Connection(stim, STN.input, transform=wt, synapse=tau_ampa)
    
    def func_str(x): #relu-like function
        if x[0]<e: return 0
        return mm*(x[0]-e)
    strd1_out = StrD1.add_output('func_str', func_str)
    strd2_out = StrD2.add_output('func_str', func_str)
    
    nengo.Connection(strd1_out, GPi.input, transform=-wm, synapse=tau_gaba)
    nengo.Connection(strd2_out, GPe.input, transform=-wm, synapse=tau_gaba)
    
    def func_stn(x):
        if x[0]<ep: return 0
        return mp*(x[0]-ep)
    stn_out = STN.add_output('func_stn', func_stn)
    
    tr=[[wp]*D for i in range(D)]    
    nengo.Connection(stn_out, GPi.input, transform=tr, synapse=tau_ampa)
   ...