YGGverse
/
YGGo
mirror of https://github.com/YGGverse/YGGo.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
123 Commits
2 Branches
0 Tags
4.3 MiB
Tree: 24472ea452
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '24472ea452'
${ noResults }
Commit Graph

123 Commits (24472ea452f6ad4d48b2109f0de962dc84167ea4)
 

Author SHA1 Message Date
ghost 24472ea452 update readme
2 years ago
ghost 921317c667 update readme
2 years ago
ghost 7104cf19b7 update readme
2 years ago
ghost fb18a9b955 update README.md
2 years ago
ghost 352466ad03 update host.robotsPostfix registry
2 years ago
ghost e6b1e8029c add missed regex replacement rule
2 years ago
ghost dfbc6132c9 fix robots:noindex condition, add robots:nofollow attribute support
2 years ago
ghost 5c8d299a4a add meta:robots tag support #2
2 years ago
ghost 6550eb310f update host.robotsPostfix rules
2 years ago
ghost 6cee58214e update host.robotsPostfix rules
2 years ago
ghost 6f4daf7a25 update host.robotsPostfix rule
2 years ago
ghost f4db66d53f add new host.robotsPostfix rules
2 years ago
ghost 9018acd0e2 update meta tags
2 years ago
ghost 139e2c88eb add robots.txt
2 years ago
ghost e505c76aaa update roadmap item by #5 answer
2 years ago
ghost be7eae501b add host.status registry #1, #5
2 years ago
ghost bee5086f22 add crontab configuration example, check roadmap item
2 years ago
ghost 8e8d89db0e implement database cleaner
2 years ago
ghost 3c9bc1adaa add required user-agent construction #5
2 years ago
ghost 0484d43482 fix trim path levels in the relative links
2 years ago
ghost b819fda025 init yggdrasil robots.txt registry #5
2 years ago
ghost df6f2a1869 implement CRAWL_ROBOTS_POSTFIX_RULES configuration #5
2 years ago
ghost 505544c8c9 add affiliate link
2 years ago
ghost b3c668706b trim path levels in the relative links
2 years ago
ghost 71a3e7dd0e skip x-raw-image links crawl
2 years ago
ghost 50b6e90380 Merge branch 'main' of https://github.com/YGGverse/YGGo into main
2 years ago
ghost 8d102ecdf7 index hosts with enabled status only
2 years ago
ghost 0b12e872a3 add host name to the search index
2 years ago
d47081 a29d6d5d0a
Update README.md
2 years ago
ghost ab71b3823a update readme
2 years ago
ghost e98146b78b index only 200 http code pages
2 years ago
ghost 9b9d40a97c skip javascript/mailto links index
2 years ago
ghost 2a843449e0 add process locked notice to the debug output
2 years ago
ghost 0f2b772fa8 remove not indexed pages from the search index
2 years ago
ghost ce509ec0a8 remove debug row
2 years ago
ghost 2495a2bbc7 implement MySQL/Sphinx data model #3, add basical robots.txt support #2
2 years ago
d47081 a14d18fedb
Update README.md
2 years ago
d47081 4bb3e26c7b
Update README.md
2 years ago
d47081 9b8bd6d277
Update README.md
2 years ago
d47081 f25e95cb79
Update README.md
2 years ago
d47081 ceed482bd4
Update README.md
2 years ago
d47081 006460381b
Update README.md
2 years ago
d47081 e8059d94ec
Update README.md
2 years ago
d47081 2c08604125
Update README.md
2 years ago
d47081 9377a8d0aa
Update README.md
2 years ago
d47081 9d01f9ab72
Update README.md
2 years ago
d47081 2f99dcb0d7
Update README.md
2 years ago
ghost a07ca1dce1 add ipv6 example
2 years ago
ghost c9cd38f6ac update variable names #2
2 years ago
ghost ed2d4047b4 implement robots.txt library #2
2 years ago