Commit Graph

  • 3218add372 add custom home page reindex settings ghost 2023-06-30 13:28:22 +0300
  • f4bf6b9fa4 fix crawl queue message ghost 2023-06-27 13:14:53 +0300
  • d912caeb0c fix variable name ghost 2023-06-27 13:01:46 +0300
  • 6702c3f402 add hostPageDom generate [selectors] attribute ghost 2023-06-26 17:15:28 +0300
  • 29197ab904 remove testing construction ghost 2023-06-26 15:59:08 +0300
  • ed240d53b0 show available snaps only ghost 2023-06-25 23:29:30 +0300
  • 2c5128382b fix semaphore ID ghost 2023-06-25 22:16:05 +0300
  • a79943dbae update readme ghost 2023-06-25 22:12:07 +0300
  • ebc1a573dc initiate CLI tool ghost 2023-06-25 22:11:49 +0300
  • 5346b13602 implement custom hostPageDom elements index ghost 2023-06-25 22:10:47 +0300
  • 5df598a1d4 fix variable name ghost 2023-06-24 15:21:47 +0300
  • 1c5346bc07 remove single char words ghost 2023-06-22 13:37:12 +0300
  • e16a7b8171 fix HY000/1366 error processing ghost 2023-06-17 11:33:32 +0300
  • dc2d971ba0 clean up banned pages extra data ghost 2023-06-16 16:53:14 +0300
  • a657d31e1d fix enum data type ghost 2023-06-16 16:32:46 +0300
  • d96abb8ea8 ban host page on encoding not detected ghost 2023-06-16 13:23:52 +0300
  • d2469e9adc fix meta variables overwrite ghost 2023-06-14 02:53:14 +0300
  • 0949d7f871 set default encoding ghost 2023-06-14 02:20:09 +0300
  • 1d5d5ead5d fix DomDocument initiation without encoding provided ghost 2023-06-14 02:20:00 +0300
  • fcda6b9885 remove MIME filters from explorer search form ghost 2023-06-13 23:25:17 +0300
  • f3475035c2 show page size in explorer view, hide not available data ghost 2023-06-13 23:20:22 +0300
  • 8a747de341 fix HTML/multimedia content detection ghost 2023-06-13 23:09:44 +0300
  • 93c6067fd9 fix host page mime detection ghost 2023-06-13 22:29:28 +0300
  • c07d6af52f add new mime preset ghost 2023-06-13 21:57:01 +0300
  • 052b08ea26 show results quantity in the mime filter titles ghost 2023-06-13 21:20:02 +0300
  • 80d3912bc7 allow x-raw-image links ghost 2023-06-13 20:26:17 +0300
  • b23f550a1b skip magnet links ghost 2023-06-13 20:25:37 +0300
  • acba2816e2 remove transaction from tables optimization case ghost 2023-06-13 17:45:02 +0300
  • be81299c84 update readme ghost 2023-06-13 17:35:47 +0300
  • b2cf9fc6a5 do table optimization in separated transaction ghost 2023-06-13 16:51:16 +0300
  • ab78e17ca8 add hostPage.size collection ghost 2023-06-13 12:45:12 +0300
  • 830e96b03d increase minimum requirements ghost 2023-06-13 03:16:29 +0300
  • 7892784f5c add httpCode column to hostPageSnapDownload table ghost 2023-06-12 13:34:25 +0300
  • 20726fca45 update readme ghost 2023-06-10 00:20:39 +0300
  • dd736c7923 crontab schedule optimization ghost 2023-06-10 00:19:27 +0300
  • a79993a94b add an mk ghost 2023-06-08 00:11:02 +0300
  • edec590e09 fix MAYBE filter in the default search mode ghost 2023-06-06 00:36:13 +0300
  • e1fb7f8c17 change query separators to the MAYBE operator in default search mode ghost 2023-06-05 23:33:07 +0300
  • 9379809261 colorize meow ghost 2023-06-05 23:08:29 +0300
  • 0af5d165d3 remove logCrawler column not in use ghost 2023-06-05 22:06:55 +0300
  • 4b16b41440 make transaction for each item in crawl queue ghost 2023-06-05 22:01:22 +0300
  • b585b16d31 fix datatype error detection ghost 2023-06-05 21:02:18 +0300
  • 8726512cf0 change morphology from stem_enru to lemmatize_ru_all/lemmatize_en_all ghost 2023-06-05 18:20:49 +0300
  • 8bc8a943e7 add lemmatize_de_all ghost 2023-06-05 18:13:31 +0300
  • 3d3fcdda87 update readme ghost 2023-06-05 13:44:23 +0300
  • a1249859a8 update readme ghost 2023-06-05 13:42:19 +0300
  • eef23cc830 update readme ghost 2023-06-05 13:40:35 +0300
  • 17f69b9661 add min_word_len, min_prefix_len, html_strip, index_exact_words presets example ghost 2023-06-05 13:36:15 +0300
  • 1e2736d67b skip empty mime type index ghost 2023-06-04 18:10:59 +0300
  • c5e25d17fb prevent page ban when it MIME in the whitelist, skip steps below only (make multimedia/streaming resources visible in search results) ghost 2023-06-04 17:44:09 +0300
  • 4fa33afe40 prevent infinitive connection on streaming resources detected ghost 2023-06-04 17:02:32 +0300
  • 345c59b5f4 collect target location links on page redirect available ghost 2023-06-04 14:58:33 +0300
  • 5d7f2bf68c fix snap foreign keys deletion ghost 2023-06-04 13:39:47 +0300
  • 242e0abd86 ban pages only on data type error codes only ghost 2023-06-04 13:10:32 +0300
  • 62a4f33b53 load missed dependency ghost 2023-06-04 12:27:20 +0300
  • 512bd56056 ban page that throws the error and stuck the crawl queue ghost 2023-06-04 12:04:41 +0300
  • 5a47c66e55 fix readme description ghost 2023-06-04 11:49:25 +0300
  • f49076bb0c index homepages and shorter URL with higher priority ghost 2023-06-04 11:38:56 +0300
  • 3b1590cf7b update readme ghost 2023-06-01 00:07:11 +0300
  • 69eb8d4ed4 update readme ghost 2023-05-31 16:30:05 +0300
  • 52bd900ad4 update readme ghost 2023-05-31 16:28:58 +0300
  • 982be2a949 add the description text source ghost 2023-05-30 21:46:52 +0300
  • cb60d52a0b update documentation ghost 2023-05-29 22:36:13 +0300
  • 45c4f7b7b0 add database optimization settings ghost 2023-05-29 22:13:41 +0300
  • fc687f7f2c update readme ghost 2023-05-18 14:13:00 +0300
  • 3113057a36 update readme ghost 2023-05-18 14:11:42 +0300
  • a2b1dc4aa7 update meow ghost 2023-05-17 14:25:31 +0300
  • 6e45c85ce4 update readme ghost 2023-05-17 13:52:50 +0300
  • d88b30925c update readme ghost 2023-05-17 13:50:04 +0300
  • 3d100568d7 update readme ghost 2023-05-17 13:48:56 +0300
  • 940d5b0042 update readme ghost 2023-05-15 21:54:37 +0300
  • 56a706948e update readme ghost 2023-05-15 21:25:54 +0300
  • 35acabeab9 update readme ghost 2023-05-15 21:18:30 +0300
  • 2853db6207 fix mimes separator ghost 2023-05-15 17:18:33 +0300
  • a69270034a update robots.txt ghost 2023-05-15 17:08:15 +0300
  • f827c37691 add MEGAcmd/FTP launch examples ghost 2023-05-15 11:51:27 +0300
  • ae7467ad75 update readme ghost 2023-05-15 09:34:36 +0300
  • df195249df update readme ghost 2023-05-15 09:33:16 +0300
  • 81f7ea1e1e implement multi-storage snap downloads ghost 2023-05-15 09:18:18 +0300
  • 653be0b79d add missed snap folder ghost 2023-05-14 21:09:56 +0300
  • d277610b49 update .gitignore ghost 2023-05-14 19:44:20 +0300
  • afb014cdb2 update readme ghost 2023-05-14 19:42:55 +0300
  • 1969707eeb integrate optional MEGA/cmd snap storage ghost 2023-05-14 19:41:20 +0300
  • f55a2dd26a remove subject link shortener on explore page view ghost 2023-05-14 10:43:33 +0300
  • cdb9160c79 decrease max link preview length to 32 chars ghost 2023-05-14 10:37:27 +0300
  • bd99dcb023 add leading zero to mkdir access code ghost 2023-05-14 05:43:03 +0300
  • 5c1d8faa93 remove init file ghost 2023-05-14 05:41:50 +0300
  • 48664f0caf fix zip close, loop brake condition ghost 2023-05-14 04:33:35 +0300
  • 50c9066f62 add tables optimization to the cron/cleaner task ghost 2023-05-14 02:39:32 +0300
  • 134b70e130 update readme ghost 2023-05-14 01:51:30 +0300
  • 0d19004e86 make local snap storage optimization ghost 2023-05-14 01:45:55 +0300
  • 8a3b25b31c update .gitignore ghost 2023-05-13 11:07:29 +0300
  • efc66d5dab update local snap storage paths ghost 2023-05-13 11:06:40 +0300
  • 375a94a510 update readme ghost 2023-05-13 10:18:05 +0300
  • 2f7d99079d implement local snaps ghost 2023-05-13 10:15:07 +0300
  • d98b8f5c94 remove `hostPageToHostPage`.`quantity` field because of implements wrong duplicates counting on reindex ghost 2023-05-13 06:30:40 +0300
  • eeeb3dceac implement index explorer ghost 2023-05-13 05:54:15 +0300
  • 377b519a2c implement host page info mode ghost 2023-05-13 03:51:34 +0300
  • 371670fadf add media referrers info ghost 2023-05-13 03:01:00 +0300
  • 9477d87b2e change strpos to stripos ghost 2023-05-13 01:28:50 +0300