26 Commits (main)

Author SHA1 Message Date
ghost eccb7ea241 refactor hostPageDom tables, add multiple selectors and children values support 10 months ago
ghost d024ffd770 implement unlimited settings customization for each host 10 months ago
ghost ab6c0379c8 implement hosts crawl queue, move robots, sitemaps, manifests to this task 10 months ago
ghost b24d31f360 refactor cleaner, delegate tasks to crawler, init hostSetting table 10 months ago
ghost 712d67f6bf implement unlimited snap storage mirrors, delete megaCMD integration 10 months ago
ghost 5346b13602 implement custom hostPageDom elements index 11 months ago
ghost ab78e17ca8 add hostPage.size collection 12 months ago
ghost 7892784f5c add httpCode column to hostPageSnapDownload table 12 months ago
ghost 81f7ea1e1e implement multi-storage snap downloads 1 year ago
ghost 1969707eeb integrate optional MEGA/cmd snap storage 1 year ago
ghost 0d19004e86 make local snap storage optimization 1 year ago
ghost 2f7d99079d implement local snaps 1 year ago
ghost db0e66c846 refactor to mime-based content index #1 1 year ago
ghost 1c7cca1446 fix UNIQUE index relation 1 year ago
ghost 23ead4e12c update page / image description models, implement history snap crawling 1 year ago
ghost 25b6bce2ec add crawler/cleaner logs 1 year ago
ghost b6605b9132 implement not reachable resources ban feature with timeout to prevent extra http requests 1 year ago
ghost d8449d4f7d update db prototype 1 year ago
ghost 0741a3e9ef implement image crawler 1 year ago
ghost 1122cb9798 update DB prototype 1 year ago
ghost 74fb0d50be add DB prototype scheme 1 year ago
ghost 2d985f5851 update demo media 1 year ago
ghost 4f8c0bc498 update home page demo 1 year ago
ghost 5c55ee0e3f update search page style 1 year ago
ghost 616fe37d2e update home page screenshot 1 year ago
ghost c9be4d54cc add media files 1 year ago