YGGverse
/
YGGo
mirror of https://github.com/YGGverse/YGGo.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
316 Commits
2 Branches
0 Tags
4.3 MiB
Tree: a69270034a
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'a69270034a'
${ noResults }
Commit Graph

316 Commits (a69270034a39865d28bfb53db1efb72beed152e8)
 

Author SHA1 Message Date
ghost 5c8d299a4a add meta:robots tag support #2
2 years ago
ghost 6550eb310f update host.robotsPostfix rules
2 years ago
ghost 6cee58214e update host.robotsPostfix rules
2 years ago
ghost 6f4daf7a25 update host.robotsPostfix rule
2 years ago
ghost f4db66d53f add new host.robotsPostfix rules
2 years ago
ghost 9018acd0e2 update meta tags
2 years ago
ghost 139e2c88eb add robots.txt
2 years ago
ghost e505c76aaa update roadmap item by #5 answer
2 years ago
ghost be7eae501b add host.status registry #1, #5
2 years ago
ghost bee5086f22 add crontab configuration example, check roadmap item
2 years ago
ghost 8e8d89db0e implement database cleaner
2 years ago
ghost 3c9bc1adaa add required user-agent construction #5
2 years ago
ghost 0484d43482 fix trim path levels in the relative links
2 years ago
ghost b819fda025 init yggdrasil robots.txt registry #5
2 years ago
ghost df6f2a1869 implement CRAWL_ROBOTS_POSTFIX_RULES configuration #5
2 years ago
ghost 505544c8c9 add affiliate link
2 years ago
ghost b3c668706b trim path levels in the relative links
2 years ago
ghost 71a3e7dd0e skip x-raw-image links crawl
2 years ago
ghost 50b6e90380 Merge branch 'main' of https://github.com/YGGverse/YGGo into main
2 years ago
ghost 8d102ecdf7 index hosts with enabled status only
2 years ago
ghost 0b12e872a3 add host name to the search index
2 years ago
d47081 a29d6d5d0a
Update README.md
2 years ago
ghost ab71b3823a update readme
2 years ago
ghost e98146b78b index only 200 http code pages
2 years ago
ghost 9b9d40a97c skip javascript/mailto links index
2 years ago
ghost 2a843449e0 add process locked notice to the debug output
2 years ago
ghost 0f2b772fa8 remove not indexed pages from the search index
2 years ago
ghost ce509ec0a8 remove debug row
2 years ago
ghost 2495a2bbc7 implement MySQL/Sphinx data model #3, add basical robots.txt support #2
2 years ago
d47081 a14d18fedb
Update README.md
2 years ago
d47081 4bb3e26c7b
Update README.md
2 years ago
d47081 9b8bd6d277
Update README.md
2 years ago
d47081 f25e95cb79
Update README.md
2 years ago
d47081 ceed482bd4
Update README.md
2 years ago
d47081 006460381b
Update README.md
2 years ago
d47081 e8059d94ec
Update README.md
2 years ago
d47081 2c08604125
Update README.md
2 years ago
d47081 9377a8d0aa
Update README.md
2 years ago
d47081 9d01f9ab72
Update README.md
2 years ago
d47081 2f99dcb0d7
Update README.md
2 years ago
ghost a07ca1dce1 add ipv6 example
2 years ago
ghost c9cd38f6ac update variable names #2
2 years ago
ghost ed2d4047b4 implement robots.txt library #2
2 years ago
ghost 183ad99ccd change repository address
2 years ago
ghost e7e4bb686c fix curl exec double call
2 years ago
ghost 79663c84db add CRAWL_META_ONLY option
2 years ago
ghost dc55dcb9b5 Merge branch 'main' of https://github.com/d47081/YGGo into main
2 years ago
ghost f0516126e2 add image storage cache folder
2 years ago
d47081 014b56ab03
Update README.md
2 years ago
d47081 60947dbf6e
Update README.md
2 years ago