557 Commits (88d2b16699a526563b1c56fc88598d780d1a7b8f)
 

Author SHA1 Message Date
ghost bee5086f22 add crontab configuration example, check roadmap item 2 years ago
ghost 8e8d89db0e implement database cleaner 2 years ago
ghost 3c9bc1adaa add required user-agent construction #5 2 years ago
ghost 0484d43482 fix trim path levels in the relative links 2 years ago
ghost b819fda025 init yggdrasil robots.txt registry #5 2 years ago
ghost df6f2a1869 implement CRAWL_ROBOTS_POSTFIX_RULES configuration #5 2 years ago
ghost 505544c8c9 add affiliate link 2 years ago
ghost b3c668706b trim path levels in the relative links 2 years ago
ghost 71a3e7dd0e skip x-raw-image links crawl 2 years ago
ghost 50b6e90380 Merge branch 'main' of https://github.com/YGGverse/YGGo into main 2 years ago
ghost 8d102ecdf7 index hosts with enabled status only 2 years ago
ghost 0b12e872a3 add host name to the search index 2 years ago
d47081 a29d6d5d0a
Update README.md 2 years ago
ghost ab71b3823a update readme 2 years ago
ghost e98146b78b index only 200 http code pages 2 years ago
ghost 9b9d40a97c skip javascript/mailto links index 2 years ago
ghost 2a843449e0 add process locked notice to the debug output 2 years ago
ghost 0f2b772fa8 remove not indexed pages from the search index 2 years ago
ghost ce509ec0a8 remove debug row 2 years ago
ghost 2495a2bbc7 implement MySQL/Sphinx data model #3, add basical robots.txt support #2 2 years ago
d47081 a14d18fedb
Update README.md 2 years ago
d47081 4bb3e26c7b
Update README.md 2 years ago
d47081 9b8bd6d277
Update README.md 2 years ago
d47081 f25e95cb79
Update README.md 2 years ago
d47081 ceed482bd4
Update README.md 2 years ago
d47081 006460381b
Update README.md 2 years ago
d47081 e8059d94ec
Update README.md 2 years ago
d47081 2c08604125
Update README.md 2 years ago
d47081 9377a8d0aa
Update README.md 2 years ago
d47081 9d01f9ab72
Update README.md 2 years ago
d47081 2f99dcb0d7
Update README.md 2 years ago
ghost a07ca1dce1 add ipv6 example 2 years ago
ghost c9cd38f6ac update variable names #2 2 years ago
ghost ed2d4047b4 implement robots.txt library #2 2 years ago
ghost 183ad99ccd change repository address 2 years ago
ghost e7e4bb686c fix curl exec double call 2 years ago
ghost 79663c84db add CRAWL_META_ONLY option 2 years ago
ghost dc55dcb9b5 Merge branch 'main' of https://github.com/d47081/YGGo into main 2 years ago
ghost f0516126e2 add image storage cache folder 2 years ago
d47081 014b56ab03
Update README.md 2 years ago
d47081 60947dbf6e
Update README.md 2 years ago
ghost bac2ffa635 add image dimentions for low connection UI optimization 2 years ago
ghost abe927b4bc Merge branch 'main' of https://github.com/d47081/YGGo into main 2 years ago
ghost 5c55ee0e3f update search page style 2 years ago
d47081 5d6e50941c
Update README.md 2 years ago
ghost 74578b7aad fix extension to webp 2 years ago
ghost 18edb66ab0 urlencode identicon requests 2 years ago
ghost ff95df72c1 implement hostname identicons 2 years ago
ghost 616fe37d2e update home page screenshot 2 years ago
ghost c00e4c7e70 fix empty request title 2 years ago