notwa/optim
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
157 Commits 1 Branch 0 Tags 1.1 MiB
Commit Graph

157 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Connor Olding c81ce0afbb rename stuff and add a couple missing imports 2018-01-21 22:16:36 +00:00
Connor Olding bbdb91fcb1 merge and split modules into a package 2018-01-21 22:07:57 +00:00
Connor Olding db65fbdd62 add Neumann optimizer 2018-01-12 15:42:04 +00:00
Connor Olding 1ebb897f14 use @ operator 2017-10-19 04:12:16 +00:00
Connor Olding a85ee67780 allow CLRs to use optimizer's learning rate 2017-10-19 04:03:44 +00:00
Connor Olding 763246df98 add RMSpropCentered to model from config 2017-09-26 23:12:40 +00:00
Connor Olding 9bb26b1ec5 add Huber loss 2017-09-25 16:37:52 +00:00
Connor Olding eb16377ba6 add Adagrad optimizer 2017-09-25 16:06:45 +00:00
Connor Olding c964f143d2 not true 2017-09-25 07:12:19 +00:00
Connor Olding 5b6fd6259f update example 2017-09-25 06:28:59 +00:00
Connor Olding a760c4841b add fallback to optim.lr in AnnealingLearner 2017-09-25 06:10:54 +00:00
Connor Olding 916c6fe1f0 assert that rituals have been prepared 2017-09-25 06:10:04 +00:00
Connor Olding 615f43c550 support AddSign and PowerSign in config 2017-09-25 06:09:36 +00:00
Connor Olding fe54002671 remove DumbLearner 2017-09-25 06:09:07 +00:00
Connor Olding 9a7ffe5f0d add AddSign and PowerSign optimizers 2017-09-25 04:02:17 +00:00
Connor Olding 5c83f063be remove keras stuff so it won't show on google 
bleh
 2017-09-18 04:42:41 +00:00
Connor Olding d38e2076f0 allow multi-input and multi-output models 2017-09-16 18:28:05 +00:00
Connor Olding 3386869b30 move actreg tweaking into if statement 
i was getting division by zero.
 2017-09-16 17:33:47 +00:00
Connor Olding 65ba80bb96 skip over irrelevant nodes 2017-09-16 17:31:39 +00:00
Connor Olding dcbaef3032 use MomentumClip in warmup for stability 2017-09-16 17:30:52 +00:00
Connor Olding 7878f94f43 auto-increment loss filenames to stop clobbering 2017-09-16 17:30:02 +00:00
Connor Olding e22316a4c9 move losses into Model and refactor methods 2017-09-16 17:05:25 +00:00
Connor Olding 910facf98d move NLL to core 2017-08-05 10:59:05 +00:00
Connor Olding 0b9c1fe117 allow SGDR to anneal optimizer's learning rate 
e.g. YellowFin
 2017-08-05 10:43:38 +00:00
Connor Olding dbd6c31ea5 fix final rate calculation 2017-08-05 10:43:18 +00:00
Connor Olding 915b39d783 allow Optimizers to inspect Models (currently unused) 
the thing that takes advantage of this may or may not be committed,
so this may or may not get reverted.
 2017-08-05 10:41:35 +00:00
Connor Olding de5af4f7f4 allow argument passthru to normalizer in _mr_make_norm 2017-08-05 10:40:39 +00:00
Connor Olding 957ee86e20 add PolyLearner: polynomial learning scheduler 2017-08-05 10:40:06 +00:00
Connor Olding cc89465adc tweak comment 2017-08-05 10:39:59 +00:00
Connor Olding 058a779f6c remove some unused arguments 2017-08-05 10:39:32 +00:00
Connor Olding 2e74c9160c tweak CubicGB defaults 2017-08-03 03:39:25 +00:00
Connor Olding 001a997e09 correction: batches, not epochs. 2017-08-03 03:38:07 +00:00
Connor Olding 9138f73141 update mnist training 
crank up the learning rate on emnist and use momentum with gradient clipping.
add a simple restart callback.
remove batch size adapation crap.
remove confidence measures.
 2017-08-03 03:36:46 +00:00
Connor Olding 7ac67fba8f fix Bias layer 2017-08-02 11:37:39 +00:00
Connor Olding 049d966710 remove biasing from Conv1Dper in favor of Bias layer 2017-08-02 11:30:08 +00:00
Connor Olding 4ee2181691 add standalone Bias layer 2017-08-02 11:28:41 +00:00
Connor Olding e7c12c1f44 add ad-hoc weight-sharing method 2017-08-02 11:28:18 +00:00
Connor Olding f507dc10f8 remove DenseOneLess 
not useful.
 2017-08-02 10:52:26 +00:00
Connor Olding 4d2251f69f allow weight sharing; disableable gradient clearing 2017-08-02 10:29:58 +00:00
Connor Olding 89fcd25962 fix wording 2017-08-02 07:00:33 +00:00
Connor Olding e4fa5bf63f add positional control to convolution 2017-08-02 06:47:37 +00:00
Connor Olding 5074dcb2aa add Decimate and Undecimate layers 2017-08-02 06:47:15 +00:00
Connor Olding f28e8d3a54 add/remove comments and fix code style 2017-08-02 03:59:15 +00:00
Connor Olding 8b3b8d8288 add rough 1D circular convolution 2017-08-02 03:58:24 +00:00
Connor Olding 5d9efa71c1 move SquaredHalved to core 2017-07-25 22:14:17 +00:00
Connor Olding f43063928e rename Linear activation to Identity layer 2017-07-25 22:12:27 +00:00
Connor Olding e5fd937ef6 remove cruft from YellowFin 
i might just remove YellowFin itself because it isn't working for me.
 2017-07-25 21:38:09 +00:00
Connor Olding 2cf38d4ece finally fix learning rate scheduling for real 
okay, this is a disaster, but i think i've got it under control now.

the way batch-based learners now work is:
the epoch we're working towards is the truncated part of the epoch variable,
and how far we are into the epoch is the fractional part.

epoch starts at 1, so subtract by 1 when doing periodic operations.
 2017-07-25 04:25:35 +00:00
Connor Olding 93547b1974 add a linear (identity) activation for good measure 2017-07-25 04:24:32 +00:00
Connor Olding 6933e21e0e update mnist example 2017-07-23 04:23:57 +00:00