Commit Graph

581 Commits

Author SHA1 Message Date
ghost
3563580fa9 update font size 2023-10-18 18:11:56 +03:00
ghost
ea7d1a41ed update paddings 2023-10-18 18:00:32 +03:00
ghost
de712181a4 quote ipv6 url 2023-10-18 17:48:07 +03:00
ghost
1ac3c18a26 update readme 2023-10-17 18:50:14 +03:00
ghost
377fd5a941 update readme 2023-10-17 18:49:19 +03:00
ghost
7c97b32dd5 update readme 2023-10-17 18:45:11 +03:00
ghost
71c16c9c19 add database snaps link 2023-10-16 22:05:39 +03:00
ghost
669743592e update readme 2023-10-16 22:03:17 +03:00
ghost
16a99347db update readme 2023-10-16 22:01:41 +03:00
ghost
e218021ccc update readme 2023-09-07 21:19:45 +03:00
ghost
ece0f03385 relate exception processing with #11 2023-09-06 14:32:21 +03:00
ghost
a1e2721849 skip links collect with rel=nofollow attribute 2023-09-06 00:34:59 +03:00
ghost
e576cb69db update readme 2023-09-02 16:32:17 +03:00
ghost
0186d8705b change identicon library to jidenticon 2023-09-02 16:31:00 +03:00
ghost
ebe42dfe18 update readme 2023-08-30 12:06:55 +03:00
ghost
f26edf5af6 update readme 2023-08-30 12:06:15 +03:00
ghost
f9cf414901 reduce quantity of http requests for each page in queue by CRAWL_HOST_PAGE_SECONDS_DELAY setting 2023-08-17 18:56:29 +03:00
ghost
468ef50ee3 delete deprecated constructions 2023-08-17 18:43:19 +03:00
ghost
eccb7ea241 refactor hostPageDom tables, add multiple selectors and children values support 2023-08-17 18:32:48 +03:00
ghost
42b34d0783 fix settings procesing, remove unused variables 2023-08-17 15:09:56 +03:00
ghost
b1bfd79b80 change DEFAULT_HOST_URL_REGEXP check from host to page URL 2023-08-17 14:59:00 +03:00
ghost
a8ffe14349 implement 'hostPage add' CLI method 2023-08-17 14:58:06 +03:00
ghost
56c376474f fix foreach continue level 2023-08-17 14:10:17 +03:00
ghost
1012759c65 update config example 2023-08-17 14:02:11 +03:00
ghost
88d2b16699 implement hostPageDom delete action 2023-08-17 13:43:36 +03:00
ghost
3a9d78b7c4 implement hostPageDom delete action 2023-08-17 13:43:27 +03:00
ghost
0f127ddb91 upgrade hostPageDom crawler to Symfony\Component\DomCrawler 2023-08-17 13:28:50 +03:00
ghost
055b15333e fix variable name 2023-08-17 13:16:00 +03:00
ghost
ec3fc1e15d remove debug constructions 2023-08-17 13:13:20 +03:00
ghost
37d01013db add semaphores namespace 2023-08-17 12:59:13 +03:00
ghost
8fd422b5c2 generate hostPageDom target value based on source selector 2023-08-17 12:58:38 +03:00
ghost
d1b115d11c add semaphores namespace 2023-08-17 12:55:42 +03:00
ghost
0638bc6742 update DEFAULT_HOST_PAGES_DOM_SELECTORS format 2023-08-17 12:55:15 +03:00
ghost
175209813f add findLastHostPageDomBySelector method 2023-08-17 11:04:28 +03:00
ghost
0b4abd2b50 update DEFAULT_HOST_PAGES_DOM_SELECTORS syntax 2023-08-17 11:04:09 +03:00
ghost
e3138faeac delete deprecated method 2023-08-17 10:09:31 +03:00
ghost
2b49ff5f6a move hostPageDescription.data field data to hostPageDom.value 2023-08-16 23:25:45 +03:00
ghost
665563e0b8 update setting options 2023-08-16 22:30:55 +03:00
ghost
70db9620ec replace simple_html_dom library with Symfony\Component\DomCrawler 2023-08-16 22:01:10 +03:00
ghost
caa0df67ee update readme 2023-08-16 12:47:09 +03:00
ghost
644270ee11 update readme 2023-08-15 11:50:18 +03:00
ghost
c081f27766 update readme 2023-08-15 11:39:04 +03:00
ghost
a27cb61f69 replace memcached to Yggverse\Cache\Memory API 2023-08-15 11:16:11 +03:00
ghost
30520f6047 search page speed optimization, yggverse/cache library integration begin 2023-08-15 10:24:37 +03:00
ghost
dc1b3a169c add peak memory usage debug 2023-08-15 09:35:31 +03:00
ghost
e7201c33de add memory usage debug 2023-08-15 09:21:43 +03:00
ghost
c9a354e4ba implement hostSetting set/get methods 2023-08-14 12:22:54 +03:00
ghost
b2d7fb2fef fix line break return 2023-08-14 12:08:05 +03:00
ghost
6085677e67 upgrade yggstate db query 2023-08-11 00:35:03 +03:00
ghost
ab0391e29e fix url parser path 2023-08-07 14:14:12 +03:00