Commit Graph

33 Commits

Author SHA1 Message Date
Connor Olding 44e2a491a0 merge a couple lines 2021-08-14 17:42:22 -07:00
Connor Olding 6508ee1210 rewrite rate-limiter to not create any tasks 2021-08-14 00:55:06 -07:00
Connor Olding f1a5cbe69f clean up some commented-out stuff 2021-08-13 02:33:43 -07:00
Connor Olding 12d0b4b738 detect and handle task cancellations semi-properly (again) 2021-08-13 02:28:41 -07:00
Connor Olding 3ca869ccad overhaul the try_ip function's async stuff 2021-08-13 02:09:48 -07:00
Connor Olding 55024634f7 add a configurable soft limit to number of DNS connections
yes, i know UDP connections don't technically exist
outside of sockets at the OS level, don't @ me.
2021-08-13 00:52:10 -07:00
Connor Olding 367329cf8c fix missing handling of deferred IPs 2021-08-12 22:38:19 -07:00
Connor Olding a71d9c33d7 fix stupid and dangerous bugs with sleeps not working 2021-08-12 22:18:57 -07:00
Connor Olding 0b198cc5c0 overhaul the main function's async stuff
after i confirm i didn't break anything,
next up is the underlying try_ip function.
2021-08-12 21:10:09 -07:00
Connor Olding d2d4917a53 detect and handle task cancellations semi-properly 2021-08-11 18:56:15 -07:00
Connor Olding 1395e7d6f6 detect blocks that self-redirect 2021-08-06 14:22:27 -07:00
Connor Olding f0e6f2d6da add a mode to update country codes of all known IPs 2021-06-09 11:00:54 +02:00
Connor Olding 73f4ca2b53 invert bad conditional 2020-09-06 12:13:39 +02:00
Connor Olding 6d6c2d52cd only flush ip info occasionally 2020-09-05 04:09:21 +02:00
Connor Olding c65b465560 make country code lookup a background task 2020-09-05 03:45:05 +02:00
Connor Olding 49ad63750e enable archive.is again but as a new "ignore" group 2020-09-04 16:16:59 +02:00
Connor Olding 35f2ce9206 add newline for pycodestyle 2020-09-04 15:34:29 +02:00
Connor Olding a8a7b9e7b8 deduplicate main loop 2020-09-04 15:08:03 +02:00
Connor Olding d659763672 move blocking_file_io into Options 2020-09-04 15:01:58 +02:00
Connor Olding 70308230e6 write and use async iterator for IpReader
also fixes the counter in the blocking version when progress is enabled
2020-09-04 14:17:04 +02:00
Connor Olding b26a4e7646 rewrite IpReader iterator to accumulate IPs before yielding
still needs work
2020-09-04 13:36:59 +02:00
Connor Olding e6c080bf32 support reading IPs from multiple files at once
this also adds the progress flag that was missing previously.
2020-09-04 13:09:49 +02:00
Connor Olding 92ee219ffc make database syncing asynchronous 2020-09-04 13:08:04 +02:00
Connor Olding db9eb0236b replace some IPs with a single subnet check 2020-09-03 15:02:59 +02:00
Connor Olding 0603081c49 rewrite ipinfo into a class 2020-09-03 12:41:47 +02:00
Connor Olding 51df54be80 add country code column with remote lookup
this needs work
2020-09-03 11:28:38 +02:00
Connor Olding 24338417a8 split and abstract block ranges into a function 2020-09-03 11:24:39 +02:00
Connor Olding 1c85e7b329 increase default timeout duration 2020-09-02 10:38:38 +02:00
Connor Olding 3a09d52345 update GFW IPs 2020-09-02 00:51:34 +02:00
Connor Olding 3b31919f75 add more known GFW and China IPs 2020-09-01 08:30:53 +02:00
Connor Olding bdb2a88f40 abide to PEP 808 (passes pycodestyle) 2020-08-29 15:34:46 +02:00
Connor Olding 2a00288a9c even more reorganization 2020-08-29 15:13:59 +02:00
Connor Olding 85045f397f further reorganization 2020-08-29 15:04:56 +02:00