Commit Graph

555 Commits

Author SHA1 Message Date
ghost
0f127ddb91 upgrade hostPageDom crawler to Symfony\Component\DomCrawler 2023-08-17 13:28:50 +03:00
ghost
055b15333e fix variable name 2023-08-17 13:16:00 +03:00
ghost
ec3fc1e15d remove debug constructions 2023-08-17 13:13:20 +03:00
ghost
37d01013db add semaphores namespace 2023-08-17 12:59:13 +03:00
ghost
8fd422b5c2 generate hostPageDom target value based on source selector 2023-08-17 12:58:38 +03:00
ghost
d1b115d11c add semaphores namespace 2023-08-17 12:55:42 +03:00
ghost
0638bc6742 update DEFAULT_HOST_PAGES_DOM_SELECTORS format 2023-08-17 12:55:15 +03:00
ghost
175209813f add findLastHostPageDomBySelector method 2023-08-17 11:04:28 +03:00
ghost
0b4abd2b50 update DEFAULT_HOST_PAGES_DOM_SELECTORS syntax 2023-08-17 11:04:09 +03:00
ghost
e3138faeac delete deprecated method 2023-08-17 10:09:31 +03:00
ghost
2b49ff5f6a move hostPageDescription.data field data to hostPageDom.value 2023-08-16 23:25:45 +03:00
ghost
665563e0b8 update setting options 2023-08-16 22:30:55 +03:00
ghost
70db9620ec replace simple_html_dom library with Symfony\Component\DomCrawler 2023-08-16 22:01:10 +03:00
ghost
caa0df67ee update readme 2023-08-16 12:47:09 +03:00
ghost
644270ee11 update readme 2023-08-15 11:50:18 +03:00
ghost
c081f27766 update readme 2023-08-15 11:39:04 +03:00
ghost
a27cb61f69 replace memcached to Yggverse\Cache\Memory API 2023-08-15 11:16:11 +03:00
ghost
30520f6047 search page speed optimization, yggverse/cache library integration begin 2023-08-15 10:24:37 +03:00
ghost
dc1b3a169c add peak memory usage debug 2023-08-15 09:35:31 +03:00
ghost
e7201c33de add memory usage debug 2023-08-15 09:21:43 +03:00
ghost
c9a354e4ba implement hostSetting set/get methods 2023-08-14 12:22:54 +03:00
ghost
b2d7fb2fef fix line break return 2023-08-14 12:08:05 +03:00
ghost
6085677e67 upgrade yggstate db query 2023-08-11 00:35:03 +03:00
ghost
ab0391e29e fix url parser path 2023-08-07 14:14:12 +03:00
ghost
f8845c620f update installation/setup guide 2023-08-07 14:04:32 +03:00
ghost
183ae91ccc add composer support, refactor FS tree to psr-4 2023-08-07 14:00:13 +03:00
ghost
7bb1eb5b61 add class deprecation notice 2023-08-07 13:22:24 +03:00
ghost
034a683df7 add YGGstate DB crawl integration 2023-08-07 00:13:04 +03:00
ghost
3d9db381e8 fix CRAWL_MANIFEST_API_VERSION 2023-08-06 21:27:56 +03:00
ghost
3c3443b3fd freeze crawl on remote storage connection lost, infinitely repeat new attempt after 60 seconds until storage connected again 2023-08-06 17:57:42 +03:00
ghost
872ea25d00 remove deprecated condition 2023-08-05 22:00:26 +03:00
ghost
fff75d4d86 update debug message 2023-08-05 21:58:18 +03:00
ghost
6eefd9b762 fix undefined variable 2023-08-05 21:57:11 +03:00
ghost
e953c01eaa update debug message 2023-08-05 21:55:37 +03:00
ghost
bd212edb97 update debug message 2023-08-05 21:52:26 +03:00
ghost
1b287c8d28 update debug message 2023-08-05 21:40:59 +03:00
ghost
562b97ba8f update debug message 2023-08-05 21:39:44 +03:00
ghost
c5ae6974bd fix PDO calls 2023-08-05 21:36:28 +03:00
ghost
b3ec1d42a7 fix empty URI processing 2023-08-05 21:31:33 +03:00
ghost
7ddb47619a update debug message 2023-08-05 21:17:05 +03:00
ghost
9fe33a3b2c update CLI roadmap 2023-08-05 21:16:09 +03:00
ghost
6e069a86e5 update readme 2023-08-05 21:11:40 +03:00
ghost
513addc7af add query totals counting, update crawler debug 2023-08-05 21:03:45 +03:00
ghost
6e03a76ed8 add CURLOPT_SSL_VERIFYHOST/CURLOPT_SSL_VERIFYPEER options 2023-08-05 20:24:47 +03:00
ghost
004a5336de remove htmls pages ban on title tag not available 2023-08-05 20:01:31 +03:00
ghost
f9774f2431 add innodb_buffer_pool_size default value 2023-08-05 19:51:30 +03:00
ghost
de28d85a71 add connection exceptions 2023-08-05 19:39:49 +03:00
ghost
142d496108 fix SQL syntax error 2023-08-05 19:31:29 +03:00
ghost
d46c4921c5 add page break 2023-08-05 19:24:32 +03:00
ghost
80b33f619c fix PAGES_LIMIT condition 2023-08-05 19:24:21 +03:00