YGGverse/YGGo
1
0
Fork 0
mirror of https://github.com/YGGverse/YGGo.git synced 2025-01-09 14:27:55 +00:00
Code Issues Projects Releases Wiki Activity
204 Commits 2 Branches 0 Tags 4.2 MiB
d4f66c83e7
Commit Graph

4 Commits

Author SHA1 Message Date
ghost
79878d17fe add crawler / proxy user agent settings 2023-05-04 07:38:22 +03:00
ghost
0741a3e9ef implement image crawler 2023-05-04 01:04:39 +03:00
ghost
eb3e70a7b7 fix robots.txt conditions 2023-05-03 04:17:58 +03:00
ghost
8e8d89db0e implement database cleaner 2023-04-09 00:06:28 +03:00