notwa/smbot
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity

scale param decay by sigma (prevents unlearning?)

Browse source
This commit is contained in:
Connor Olding 2018-06-16 21:46:22 +02:00
parent 31f153aec0
commit 49dfcfe5b3
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
main.lua
View file

@ -294,7 +294,7 @@ local function learn_from_epoch()
end end
if cfg.param_decay > 0 then if cfg.param_decay > 0 then
for i, v in ipairs(base_params) do for i, v in ipairs(base_params) do
base_params[i] = v * (1 - es.param_rate * cfg.param_decay) base_params[i] = v * (1 - es.param_rate * cfg.param_decay * es.sigma)
end end
end end
else else