43 Commits (377d4935adc0216091e37c68138343a362e343f0)

Author SHA1 Message Date
ghost 23ead4e12c update page / image description models, implement history snap crawling 2 years ago
ghost 0e9d29675f implement host page description history crawling 2 years ago
ghost 32d0f390d3 update http code and mime type on page/image ban event 2 years ago
ghost 84dcecf50b add svg images support, fix mime validation 2 years ago
ghost e6da2e729a fix images ban update 2 years ago
ghost 8fbd7f3516 count totals using sphinx index instead of database 2 years ago
ghost fdd18de373 remove abstraction 2 years ago
ghost b6605b9132 implement not reachable resources ban feature with timeout to prevent extra http requests 2 years ago
ghost 702a14b634 add mime content type crawling #1 2 years ago
ghost 0e7220f7f8 display url decoded links 2 years ago
ghost f0b2eb1613 show images total instead of pages in placeholder on image search page 2 years ago
ghost 297563d4a5 display related pages in priority to the unique host by rank, rand() order 2 years ago
ghost 317a58cfa2 add main container paddings for mobile browsers 2 years ago
ghost 5310c3423b update search page header 2 years ago
ghost 834ac68cce create separated pagination settings for page/image search types 2 years ago
ghost dad9a2633c increase h3 margins 2 years ago
ghost 79878d17fe add crawler / proxy user agent settings 2 years ago
ghost d905e33b4f update host images info on search requests 2 years ago
ghost 68581960a3 add image.data field 2 years ago
ghost 9c24eda833 switch to native curl library 2 years ago
ghost bb4e97eea3 use curl for image connections to prevent queue timeout 2 years ago
ghost 6b18202588 implement proxied image search #1 2 years ago
ghost 0bd765064b implement extended search mode support #9 2 years ago
ghost f7807cf43e add extended syntax filter to prevent sphinxql query error #9 2 years ago
ghost 9ae91ee187 remove phrase search mask, allow sphinx macroses 2 years ago
ghost fcee7f62ef fix max_matches error 2 years ago
ghost 8dbb4a06af add disk quota validation 2 years ago
ghost 9018acd0e2 update meta tags 2 years ago
ghost df6f2a1869 implement CRAWL_ROBOTS_POSTFIX_RULES configuration #5 2 years ago
ghost 2495a2bbc7 implement MySQL/Sphinx data model #3, add basical robots.txt support #2 2 years ago
ghost bac2ffa635 add image dimentions for low connection UI optimization 2 years ago
ghost 5c55ee0e3f update search page style 2 years ago
ghost 18edb66ab0 urlencode identicon requests 2 years ago
ghost ff95df72c1 implement hostname identicons 2 years ago
ghost c00e4c7e70 fix empty request title 2 years ago
ghost a3bdccddd6 fix pagination link condition 2 years ago
ghost 4ea01bf8b4 implement search results pagination 2 years ago
ghost 747fb1e52c fix text contrast in chromium browser 2 years ago
ghost 04dbbc3adf make url/src column ukeys digital by using crc32 2 years ago
ghost b218b8bbc3 make url/src columns unique keys, add insert/ignore construction 2 years ago
ghost 8aeadcb604 fix tags case 2 years ago
ghost d5f33ad643 add ceawl in queue notification 2 years ago
ghost 72985eaf9e initial commit 2 years ago