notwa/mnists
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity

add prefixes and links to shape table

Browse source
This commit is contained in:
Connor Olding 2018-03-24 06:42:29 +01:00
parent eac50b5681
commit 466b87035d
3 changed files with 29 additions and 17 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

22
README.md
View file

@ -56,17 +56,17 @@ there is only one color channel, and thus the size of this dimension is 1.
in alphabetical order, using default `mnists.prepare` arguments: in alphabetical order, using default `mnists.prepare` arguments:
| subdirectory | dataset | train images shape | train labels shape | test images shape | test labels shape |
| :--- | :--- | ---: | ---: | ---: | ---: |
| [emnist][] | emnist\_balanced | (112800, 1, 28, 28) | (112800, 47) | (18800, 1, 28, 28) | (18800, 47) |
| [emnist][] | emnist\_byclass | (697932, 1, 28, 28) | (697932, 62) | (116323, 1, 28, 28) | (116323, 62) |
| [emnist][] | emnist\_bymerge | (697932, 1, 28, 28) | (697932, 47) | (116323, 1, 28, 28) | (116323, 47) |
| [emnist][] | emnist\_digits | (240000, 1, 28, 28) | (240000, 10) | (40000, 1, 28, 28) | (40000, 10) |
| [emnist][] | emnist\_letters | (124800, 1, 28, 28) | (124800, 26) | (20800, 1, 28, 28) | (20800, 26) |
| [emnist][] | emnist\_mnist | (60000, 1, 28, 28) | (60000, 10) | (10000, 1, 28, 28) | (10000, 10) |
| [fashion-mnist][] | fashion\_mnist | (60000, 1, 28, 28) | (60000, 10) | (10000, 1, 28, 28) | (10000, 10) |
| [mnist][] | mnist | (60000, 1, 28, 28) | (60000, 10) | (10000, 1, 28, 28) | (10000, 10) |
[emnist]: //www.nist.gov/itl/iad/image-group/emnist-dataset [emnist]: //www.nist.gov/itl/iad/image-group/emnist-dataset
[fashion-mnist]: //github.com/zalandoresearch/fashion-mnist [fashion-mnist]: //github.com/zalandoresearch/fashion-mnist
[mnist]: http://yann.lecun.com/exdb/mnist/ [mnist]: http://yann.lecun.com/exdb/mnist/
| dataset | train images shape | train labels shape | test images shape | test labels shape |
| :--- | ---: | ---: | ---: | ---: |
| emnist\_balanced | (112800, 1, 28, 28) | (112800, 47) | (18800, 1, 28, 28) | (18800, 47) |
| emnist\_byclass | (697932, 1, 28, 28) | (697932, 62) | (116323, 1, 28, 28) | (116323, 62) |
| emnist\_bymerge | (697932, 1, 28, 28) | (697932, 47) | (116323, 1, 28, 28) | (116323, 47) |
| emnist\_digits | (240000, 1, 28, 28) | (240000, 10) | (40000, 1, 28, 28) | (40000, 10) |
| emnist\_letters | (124800, 1, 28, 28) | (124800, 26) | (20800, 1, 28, 28) | (20800, 26) |
| emnist\_mnist | (60000, 1, 28, 28) | (60000, 10) | (10000, 1, 28, 28) | (10000, 10) |
| fashion\_mnist | (60000, 1, 28, 28) | (60000, 10) | (10000, 1, 28, 28) | (10000, 10) |
| mnist | (60000, 1, 28, 28) | (60000, 10) | (10000, 1, 28, 28) | (10000, 10) |

2
TODO
View file

@ -1,7 +1,5 @@
TODO TODO
* add prefixes and links to shape table
* finish writing README * finish writing README
* finish npz functionality * finish npz functionality

22
mnists/__main__.py
View file

@ -1,15 +1,22 @@
from . import metadata, prepare from . import metadata, prepare
headers = ("dataset", headers = ("subdirectory",
"dataset",
"train images shape", "train images shape",
"train labels shape", "train labels shape",
"test images shape", "test images shape",
"test labels shape") "test labels shape")
row = "| {:<20} | {:<20} | {:<20} | {:<20} | {:<20} |" row = "| {:<20} | {:<20} | {:<20} | {:<20} | {:<20} | {:<20} |"
separators = (":---", "---:", "---:", "---:", "---:") separators = (":---", ":---", "---:", "---:", "---:", "---:")
urls = {
"emnist": "//www.nist.gov/itl/iad/image-group/emnist-dataset",
"fashion-mnist": "//github.com/zalandoresearch/fashion-mnist",
"mnist": "http://yann.lecun.com/exdb/mnist/",
}
print(row.format(*headers)) print(row.format(*headers))
print(row.format(*separators)) print(row.format(*separators))
@ -18,6 +25,13 @@ for name in metadata.keys():
# verify every dataset, downloading if necessary. # verify every dataset, downloading if necessary.
# print out the shape table for use in the README. # print out the shape table for use in the README.
data = prepare(name) data = prepare(name)
row_data = [name.replace("_", "\\_")] prefix = metadata[name][0]
row_data = [f"[{prefix}][]"]
row_data += [name.replace("_", "\\_")]
row_data += [str(array.shape) for array in data] row_data += [str(array.shape) for array in data]
print(row.format(*row_data)) print(row.format(*row_data))
print()
for anchor, url in urls.items():
print(f"[{anchor}]: {url}")