580 Commits (ea7d1a41edf094687fc506a7393fbd05f0415d6f)
 

Author SHA1 Message Date
ghost 183ae91ccc add composer support, refactor FS tree to psr-4 1 year ago
ghost 7bb1eb5b61 add class deprecation notice 1 year ago
ghost 034a683df7 add YGGstate DB crawl integration 1 year ago
ghost 3d9db381e8 fix CRAWL_MANIFEST_API_VERSION 1 year ago
ghost 3c3443b3fd freeze crawl on remote storage connection lost, infinitely repeat new attempt after 60 seconds until storage connected again 1 year ago
ghost 872ea25d00 remove deprecated condition 1 year ago
ghost fff75d4d86 update debug message 1 year ago
ghost 6eefd9b762 fix undefined variable 1 year ago
ghost e953c01eaa update debug message 1 year ago
ghost bd212edb97 update debug message 1 year ago
ghost 1b287c8d28 update debug message 1 year ago
ghost 562b97ba8f update debug message 1 year ago
ghost c5ae6974bd fix PDO calls 1 year ago
ghost b3ec1d42a7 fix empty URI processing 1 year ago
ghost 7ddb47619a update debug message 1 year ago
ghost 9fe33a3b2c update CLI roadmap 1 year ago
ghost 6e069a86e5 update readme 1 year ago
ghost 513addc7af add query totals counting, update crawler debug 1 year ago
ghost 6e03a76ed8 add CURLOPT_SSL_VERIFYHOST/CURLOPT_SSL_VERIFYPEER options 1 year ago
ghost 004a5336de remove htmls pages ban on title tag not available 1 year ago
ghost f9774f2431 add innodb_buffer_pool_size default value 1 year ago
ghost de28d85a71 add connection exceptions 1 year ago
ghost 142d496108 fix SQL syntax error 1 year ago
ghost d46c4921c5 add page break 1 year ago
ghost 80b33f619c fix PAGES_LIMIT condition 1 year ago
ghost d024ffd770 implement unlimited settings customization for each host 1 year ago
ghost ab6c0379c8 implement hosts crawl queue, move robots, sitemaps, manifests to this task 1 year ago
ghost 6ee5e53ef4 show sitemaps processed debug 1 year ago
ghost 71724ae33f refactor manifest crawling 1 year ago
ghost cb37c57bc4 rename example files 1 year ago
ghost 68d5820f30 reserve one hour for huge load operations 1 year ago
ghost efbbf19601 fix multimedia snaps 1 year ago
ghost 6862fb35cd update readme 1 year ago
ghost 282a6d609d update manifest API 1 year ago
ghost b24d31f360 refactor cleaner, delegate tasks to crawler, init hostSetting table 1 year ago
ghost fd90e2d517 keep banned pages data 1 year ago
ghost ab8b6f6315 rename variables 1 year ago
ghost 02612d098b delete getFoundHostPage method, update API version 1 year ago
ghost 11e02da66d memory usage optimization, rename methods, remove memchached dependency from the model 1 year ago
ghost cbabea595b rename method name 1 year ago
ghost 7e3248ca2c rename method name 1 year ago
ghost 772975059c add mysql conf example 1 year ago
ghost 7c407e0d1f update crontab example 1 year ago
ghost 1249e8d29c fix CRAWL_PAGE_RANK_UPDATE condition 1 year ago
ghost 5df59661d8 add page rank update optional in the crawl queue 1 year ago
ghost a5a2ec233e unify mime-based search results template 1 year ago
ghost 6d5901c101 display shortened page URL instead of host address, change column name 1 year ago
ghost 1d7deffc4c update PR generation, delegate PR value from redirecting pages, update method names 1 year ago
ghost bba718c901 remove host pages total column 1 year ago
ghost b7a48b905e update method names 1 year ago