YGGverse/YGGo
1
0
Fork 0
mirror of https://github.com/YGGverse/YGGo.git synced 2025-02-03 10:25:52 +00:00
Code Issues Projects Releases Wiki Activity

Commit Graph

  • bbaeb50f60 update readme main ghost 2023-11-25 19:37:56 +02:00
  • ca92db8826 add transliteration support in search requests #13 ghost 2023-10-28 01:00:00 +03:00
  • 57332b1449 fix word-break ghost 2023-10-28 00:53:21 +03:00
  • 91dcf2d2e9 fix status code container #10 ghost 2023-10-25 18:40:13 +03:00
  • 400233bfe6 allow non 200 codes in search results (page could be available in explore mode / snap cache) #10 ghost 2023-10-25 18:33:04 +03:00
  • 92a93e611e lowercase 'search' button ghost 2023-10-18 20:33:19 +03:00
  • b23c663023 update paddings ghost 2023-10-18 20:31:23 +03:00
  • 77e9a0294f cache getTopHostPages method results for 24 hours ghost 2023-10-18 20:27:12 +03:00
  • 9e71a4538b add number_format for pages total ghost 2023-10-18 20:17:50 +03:00
  • 3563580fa9 update font size ghost 2023-10-18 18:11:56 +03:00
  • ea7d1a41ed update paddings ghost 2023-10-18 18:00:32 +03:00
  • de712181a4 quote ipv6 url ghost 2023-10-18 17:48:07 +03:00
  • 1ac3c18a26 update readme ghost 2023-10-17 18:50:14 +03:00
  • 377fd5a941 update readme ghost 2023-10-17 18:49:19 +03:00
  • 7c97b32dd5 update readme ghost 2023-10-17 18:45:11 +03:00
  • 71c16c9c19 add database snaps link ghost 2023-10-16 22:05:39 +03:00
  • 669743592e update readme ghost 2023-10-16 22:03:17 +03:00
  • 16a99347db update readme ghost 2023-10-16 22:01:41 +03:00
  • e218021ccc update readme ghost 2023-09-07 21:19:45 +03:00
  • ece0f03385 relate exception processing with #11 ghost 2023-09-06 14:32:21 +03:00
  • a1e2721849 skip links collect with rel=nofollow attribute ghost 2023-09-06 00:34:59 +03:00
  • e576cb69db update readme ghost 2023-09-02 16:32:17 +03:00
  • 0186d8705b change identicon library to jidenticon ghost 2023-09-02 16:31:00 +03:00
  • ebe42dfe18 update readme ghost 2023-08-30 12:06:55 +03:00
  • f26edf5af6 update readme ghost 2023-08-30 12:06:15 +03:00
  • f9cf414901 reduce quantity of http requests for each page in queue by CRAWL_HOST_PAGE_SECONDS_DELAY setting ghost 2023-08-17 18:56:29 +03:00
  • 468ef50ee3 delete deprecated constructions ghost 2023-08-17 18:43:19 +03:00
  • eccb7ea241 refactor hostPageDom tables, add multiple selectors and children values support ghost 2023-08-17 18:32:48 +03:00
  • 42b34d0783 fix settings procesing, remove unused variables ghost 2023-08-17 15:09:56 +03:00
  • b1bfd79b80 change DEFAULT_HOST_URL_REGEXP check from host to page URL ghost 2023-08-17 14:59:00 +03:00
  • a8ffe14349 implement 'hostPage add' CLI method ghost 2023-08-17 14:58:06 +03:00
  • 56c376474f fix foreach continue level ghost 2023-08-17 14:10:17 +03:00
  • 1012759c65 update config example ghost 2023-08-17 14:02:11 +03:00
  • 88d2b16699 implement hostPageDom delete action ghost 2023-08-17 13:43:36 +03:00
  • 3a9d78b7c4 implement hostPageDom delete action ghost 2023-08-17 13:43:27 +03:00
  • 0f127ddb91 upgrade hostPageDom crawler to Symfony\Component\DomCrawler ghost 2023-08-17 13:28:50 +03:00
  • 055b15333e fix variable name ghost 2023-08-17 13:16:00 +03:00
  • ec3fc1e15d remove debug constructions ghost 2023-08-17 13:13:20 +03:00
  • 37d01013db add semaphores namespace ghost 2023-08-17 12:59:13 +03:00
  • 8fd422b5c2 generate hostPageDom target value based on source selector ghost 2023-08-17 12:58:38 +03:00
  • d1b115d11c add semaphores namespace ghost 2023-08-17 12:55:42 +03:00
  • 0638bc6742 update DEFAULT_HOST_PAGES_DOM_SELECTORS format ghost 2023-08-17 12:55:15 +03:00
  • 175209813f add findLastHostPageDomBySelector method ghost 2023-08-17 11:04:28 +03:00
  • 0b4abd2b50 update DEFAULT_HOST_PAGES_DOM_SELECTORS syntax ghost 2023-08-17 11:04:09 +03:00
  • e3138faeac delete deprecated method ghost 2023-08-17 10:09:31 +03:00
  • 2b49ff5f6a move hostPageDescription.data field data to hostPageDom.value ghost 2023-08-16 23:25:45 +03:00
  • 665563e0b8 update setting options ghost 2023-08-16 22:30:55 +03:00
  • 70db9620ec replace simple_html_dom library with Symfony\Component\DomCrawler ghost 2023-08-16 22:01:10 +03:00
  • caa0df67ee update readme ghost 2023-08-16 12:47:09 +03:00
  • 644270ee11 update readme ghost 2023-08-15 11:50:18 +03:00
  • c081f27766 update readme ghost 2023-08-15 11:39:04 +03:00
  • a27cb61f69 replace memcached to Yggverse\Cache\Memory API ghost 2023-08-15 11:16:11 +03:00
  • 30520f6047 search page speed optimization, yggverse/cache library integration begin ghost 2023-08-15 10:24:37 +03:00
  • dc1b3a169c add peak memory usage debug ghost 2023-08-15 09:35:31 +03:00
  • e7201c33de add memory usage debug ghost 2023-08-15 09:21:43 +03:00
  • c9a354e4ba implement hostSetting set/get methods ghost 2023-08-14 12:22:54 +03:00
  • b2d7fb2fef fix line break return ghost 2023-08-14 12:08:05 +03:00
  • 6085677e67 upgrade yggstate db query ghost 2023-08-11 00:35:03 +03:00
  • ab0391e29e fix url parser path ghost 2023-08-07 14:14:12 +03:00
  • f8845c620f update installation/setup guide ghost 2023-08-07 14:04:32 +03:00
  • 183ae91ccc add composer support, refactor FS tree to psr-4 ghost 2023-08-07 14:00:13 +03:00
  • 7bb1eb5b61 add class deprecation notice ghost 2023-08-07 13:22:24 +03:00
  • 034a683df7 add YGGstate DB crawl integration ghost 2023-08-07 00:13:04 +03:00
  • 3d9db381e8 fix CRAWL_MANIFEST_API_VERSION ghost 2023-08-06 21:27:56 +03:00
  • 3c3443b3fd freeze crawl on remote storage connection lost, infinitely repeat new attempt after 60 seconds until storage connected again ghost 2023-08-06 17:57:42 +03:00
  • 872ea25d00 remove deprecated condition ghost 2023-08-05 22:00:26 +03:00
  • fff75d4d86 update debug message ghost 2023-08-05 21:58:18 +03:00
  • 6eefd9b762 fix undefined variable ghost 2023-08-05 21:57:11 +03:00
  • e953c01eaa update debug message ghost 2023-08-05 21:55:37 +03:00
  • bd212edb97 update debug message ghost 2023-08-05 21:52:26 +03:00
  • 1b287c8d28 update debug message ghost 2023-08-05 21:40:59 +03:00
  • 562b97ba8f update debug message ghost 2023-08-05 21:39:44 +03:00
  • c5ae6974bd fix PDO calls ghost 2023-08-05 21:36:28 +03:00
  • b3ec1d42a7 fix empty URI processing ghost 2023-08-05 21:31:33 +03:00
  • 7ddb47619a update debug message ghost 2023-08-05 21:17:05 +03:00
  • 9fe33a3b2c update CLI roadmap ghost 2023-08-05 21:16:09 +03:00
  • 6e069a86e5 update readme ghost 2023-08-05 21:11:40 +03:00
  • 513addc7af add query totals counting, update crawler debug ghost 2023-08-05 21:03:45 +03:00
  • 6e03a76ed8 add CURLOPT_SSL_VERIFYHOST/CURLOPT_SSL_VERIFYPEER options ghost 2023-08-05 20:24:47 +03:00
  • 004a5336de remove htmls pages ban on title tag not available ghost 2023-08-05 20:01:31 +03:00
  • f9774f2431 add innodb_buffer_pool_size default value ghost 2023-08-05 19:51:30 +03:00
  • de28d85a71 add connection exceptions ghost 2023-08-05 19:39:49 +03:00
  • 142d496108 fix SQL syntax error ghost 2023-08-05 19:31:29 +03:00
  • d46c4921c5 add page break ghost 2023-08-05 19:24:32 +03:00
  • 80b33f619c fix PAGES_LIMIT condition ghost 2023-08-05 19:24:21 +03:00
  • d024ffd770 implement unlimited settings customization for each host ghost 2023-08-05 19:06:39 +03:00
  • ab6c0379c8 implement hosts crawl queue, move robots, sitemaps, manifests to this task ghost 2023-08-04 09:32:12 +03:00
  • 6ee5e53ef4 show sitemaps processed debug ghost 2023-08-04 09:07:46 +03:00
  • 71724ae33f refactor manifest crawling ghost 2023-08-04 09:00:03 +03:00
  • cb37c57bc4 rename example files ghost 2023-08-03 18:49:29 +03:00
  • 68d5820f30 reserve one hour for huge load operations ghost 2023-08-03 18:47:39 +03:00
  • efbbf19601 fix multimedia snaps ghost 2023-08-03 17:41:55 +03:00
  • 6862fb35cd update readme ghost 2023-08-03 15:33:34 +03:00
  • 282a6d609d update manifest API ghost 2023-08-03 15:31:57 +03:00
  • b24d31f360 refactor cleaner, delegate tasks to crawler, init hostSetting table ghost 2023-08-03 15:25:38 +03:00
  • fd90e2d517 keep banned pages data ghost 2023-08-03 14:31:06 +03:00
  • ab8b6f6315 rename variables ghost 2023-08-03 14:24:37 +03:00
  • 02612d098b delete getFoundHostPage method, update API version ghost 2023-08-03 14:08:45 +03:00
  • 11e02da66d memory usage optimization, rename methods, remove memchached dependency from the model ghost 2023-08-03 10:48:27 +03:00
  • cbabea595b rename method name ghost 2023-08-03 10:26:37 +03:00