YGGverse
/
YGGo
mirror of https://github.com/YGGverse/YGGo.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
240 Commits
2 Branches
0 Tags
4.3 MiB
Tree: 0bd95d7f4d
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '0bd95d7f4d'
${ noResults }
Commit Graph

6 Commits (0bd95d7f4d42f10f7172385ca6cc2c999ec52c78)

Author SHA1 Message Date
ghost f88d2ee9ff implement MIME content-type crawler filter
2 years ago
ghost 5999fb3a73 add distributed hosts crawling using yggo nodes manifest
2 years ago
ghost 79878d17fe add crawler / proxy user agent settings
2 years ago
ghost 0741a3e9ef implement image crawler
2 years ago
ghost eb3e70a7b7 fix robots.txt conditions
2 years ago
ghost 8e8d89db0e implement database cleaner
2 years ago