Connor Olding
e4fa5bf63f
add positional control to convolution
2017-08-02 06:47:37 +00:00
Connor Olding
5074dcb2aa
add Decimate and Undecimate layers
2017-08-02 06:47:15 +00:00
Connor Olding
f28e8d3a54
add/remove comments and fix code style
2017-08-02 03:59:15 +00:00
Connor Olding
8b3b8d8288
add rough 1D circular convolution
2017-08-02 03:58:24 +00:00
Connor Olding
5d9efa71c1
move SquaredHalved to core
2017-07-25 22:14:17 +00:00
Connor Olding
f43063928e
rename Linear activation to Identity layer
2017-07-25 22:12:27 +00:00
Connor Olding
e5fd937ef6
remove cruft from YellowFin
...
i might just remove YellowFin itself because it isn't working for me.
2017-07-25 21:38:09 +00:00
Connor Olding
2cf38d4ece
finally fix learning rate scheduling for real
...
okay, this is a disaster, but i think i've got it under control now.
the way batch-based learners now work is:
the epoch we're working towards is the truncated part of the epoch variable,
and how far we are into the epoch is the fractional part.
epoch starts at 1, so subtract by 1 when doing periodic operations.
2017-07-25 04:25:35 +00:00
Connor Olding
93547b1974
add a linear (identity) activation for good measure
2017-07-25 04:24:32 +00:00
Connor Olding
6933e21e0e
update mnist example
2017-07-23 04:23:57 +00:00
Connor Olding
5183cd38f8
add GB output layers for classification
2017-07-23 03:55:19 +00:00
Connor Olding
ee83ffa88e
add debug mode to MomentumClip to print norms
2017-07-23 03:54:37 +00:00
Connor Olding
b20a34c2de
fix MomentumClip with nesterov enabled
2017-07-22 05:05:29 +00:00
Connor Olding
be1795f6ed
use in-place (additive) form of filters
2017-07-21 21:02:47 +00:00
Connor Olding
7c4ef4ad05
fix Softplus derivative
2017-07-21 21:02:04 +00:00
Connor Olding
c2bb2cfcd5
add centered variant of RMS Prop
2017-07-21 20:20:42 +00:00
Connor Olding
fb22f64716
tweak semantics etc.
2017-07-21 19:45:58 +00:00
Connor Olding
217a19110a
fix case when no callbacks are given
2017-07-21 19:45:34 +00:00
Connor Olding
4a108a10ae
allow MomentumClip, SineCLR, WaveCLR in config
2017-07-21 19:43:57 +00:00
Connor Olding
e7a6974829
yeah probably not
2017-07-12 09:07:22 +00:00
Connor Olding
928850c2a8
lower process priority
2017-07-11 12:44:26 +00:00
Connor Olding
9f8ac737db
update mnist network
2017-07-11 12:11:47 +00:00
Connor Olding
112e263056
fix code i forgot to test, plus some tweaks
2017-07-11 11:36:11 +00:00
Connor Olding
7bd5518650
note to self on how to handle generators
2017-07-11 11:23:27 +00:00
Connor Olding
e145f0422f
more YellowFin bullshit
...
i give up on this optimizer. its design is flawed.
2017-07-11 11:22:36 +00:00
Connor Olding
436f45fbb0
rewrite Ritual to reduce code duplication
2017-07-03 11:54:37 +00:00
Connor Olding
85c9b3b5c1
finish(?) implementing YellowFin
2017-07-03 09:48:22 +00:00
Connor Olding
c41700ab8d
add momentum with gradient clipping
...
i may refactor out clipping later,
but it's too nice not have right now.
2017-07-03 09:44:16 +00:00
Connor Olding
6a3f047ddc
rename alpha to lr where applicable
2017-07-02 05:39:51 +00:00
Connor Olding
9706aaabbb
add WIP YellowFin optimizer implementation
2017-07-02 02:55:19 +00:00
Connor Olding
d8bf6d1c5b
easier logging of losses etc.
2017-07-02 02:53:31 +00:00
Connor Olding
1b1184480a
allow optimizers to adjust their own learning rate
2017-07-02 02:52:07 +00:00
Connor Olding
22dc651cce
move lament into core
2017-07-01 02:22:34 +00:00
Connor Olding
7da93e93a8
move graph printing into Model class
2017-07-01 02:17:46 +00:00
Connor Olding
1352de7006
merge the right commit this time
2017-07-01 01:15:16 +00:00
Connor Olding
69786b40a1
begin work on multiple input/output nodes
2017-07-01 00:44:56 +00:00
Connor Olding
a530558fc1
begin support for multiple input/output layers
2017-06-30 21:51:26 +00:00
Connor Olding
a7c4bdaa2e
remove dead line and punctuate comment
2017-06-30 21:13:37 +00:00
Connor Olding
a4c1c5c04c
update readme
2017-06-26 01:49:46 +00:00
Connor Olding
c02fba01e2
various
...
use updated filenames.
don't use emnist by default.
tweak expando integer handling.
add some comments.
2017-06-26 00:16:51 +00:00
Connor Olding
a770444199
shorten names
2017-06-25 22:08:07 +00:00
Connor Olding
1a2a8702a0
add missing imports
2017-06-21 13:07:57 +00:00
Connor Olding
c7193690c1
fix default ftml parameter
2017-06-21 12:31:41 +00:00
Connor Olding
5a4b3d2934
add Follow The Moving Leader optimizer
2017-06-21 12:20:27 +00:00
Connor Olding
9c0cbba3ff
move epsilon out of square-root (big improvement)
2017-06-21 00:13:53 +00:00
Connor Olding
9b85b49ee5
add minimal example to readme
2017-06-18 01:58:40 +00:00
Connor Olding
264c3abd83
allow integer expando values
2017-06-18 01:58:34 +00:00
Connor Olding
35f380c5d0
rename broken variables
2017-06-17 23:41:02 +00:00
Connor Olding
a4081606f7
shuffle by default
2017-06-17 17:12:59 +00:00
Connor Olding
cf1b7c1c13
various parameter tweaks
2017-06-17 16:46:39 +00:00
Connor Olding