547 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
ghost	0b4abd2b50	update DEFAULT_HOST_PAGES_DOM_SELECTORS syntax	2023-08-17 11:04:09 +03:00
ghost	e3138faeac	delete deprecated method	2023-08-17 10:09:31 +03:00
ghost	2b49ff5f6a	move hostPageDescription.data field data to hostPageDom.value	2023-08-16 23:25:45 +03:00
ghost	665563e0b8	update setting options	2023-08-16 22:30:55 +03:00
ghost	70db9620ec	replace simple_html_dom library with Symfony\Component\DomCrawler	2023-08-16 22:01:10 +03:00
ghost	caa0df67ee	update readme	2023-08-16 12:47:09 +03:00
ghost	644270ee11	update readme	2023-08-15 11:50:18 +03:00
ghost	c081f27766	update readme	2023-08-15 11:39:04 +03:00
ghost	a27cb61f69	replace memcached to Yggverse\Cache\Memory API	2023-08-15 11:16:11 +03:00
ghost	30520f6047	search page speed optimization, yggverse/cache library integration begin	2023-08-15 10:24:37 +03:00
ghost	dc1b3a169c	add peak memory usage debug	2023-08-15 09:35:31 +03:00
ghost	e7201c33de	add memory usage debug	2023-08-15 09:21:43 +03:00
ghost	c9a354e4ba	implement hostSetting set/get methods	2023-08-14 12:22:54 +03:00
ghost	b2d7fb2fef	fix line break return	2023-08-14 12:08:05 +03:00
ghost	6085677e67	upgrade yggstate db query	2023-08-11 00:35:03 +03:00
ghost	ab0391e29e	fix url parser path	2023-08-07 14:14:12 +03:00
ghost	f8845c620f	update installation/setup guide	2023-08-07 14:04:32 +03:00
ghost	183ae91ccc	add composer support, refactor FS tree to psr-4	2023-08-07 14:00:13 +03:00
ghost	7bb1eb5b61	add class deprecation notice	2023-08-07 13:22:24 +03:00
ghost	034a683df7	add YGGstate DB crawl integration	2023-08-07 00:13:04 +03:00
ghost	3d9db381e8	fix CRAWL_MANIFEST_API_VERSION	2023-08-06 21:27:56 +03:00
ghost	3c3443b3fd	freeze crawl on remote storage connection lost, infinitely repeat new attempt after 60 seconds until storage connected again	2023-08-06 17:57:42 +03:00
ghost	872ea25d00	remove deprecated condition	2023-08-05 22:00:26 +03:00
ghost	fff75d4d86	update debug message	2023-08-05 21:58:18 +03:00
ghost	6eefd9b762	fix undefined variable	2023-08-05 21:57:11 +03:00
ghost	e953c01eaa	update debug message	2023-08-05 21:55:37 +03:00
ghost	bd212edb97	update debug message	2023-08-05 21:52:26 +03:00
ghost	1b287c8d28	update debug message	2023-08-05 21:40:59 +03:00
ghost	562b97ba8f	update debug message	2023-08-05 21:39:44 +03:00
ghost	c5ae6974bd	fix PDO calls	2023-08-05 21:36:28 +03:00
ghost	b3ec1d42a7	fix empty URI processing	2023-08-05 21:31:33 +03:00
ghost	7ddb47619a	update debug message	2023-08-05 21:17:05 +03:00
ghost	9fe33a3b2c	update CLI roadmap	2023-08-05 21:16:09 +03:00
ghost	6e069a86e5	update readme	2023-08-05 21:11:40 +03:00
ghost	513addc7af	add query totals counting, update crawler debug	2023-08-05 21:03:45 +03:00
ghost	6e03a76ed8	add CURLOPT_SSL_VERIFYHOST/CURLOPT_SSL_VERIFYPEER options	2023-08-05 20:24:47 +03:00
ghost	004a5336de	remove htmls pages ban on title tag not available	2023-08-05 20:01:31 +03:00
ghost	f9774f2431	add innodb_buffer_pool_size default value	2023-08-05 19:51:30 +03:00
ghost	de28d85a71	add connection exceptions	2023-08-05 19:39:49 +03:00
ghost	142d496108	fix SQL syntax error	2023-08-05 19:31:29 +03:00
ghost	d46c4921c5	add page break	2023-08-05 19:24:32 +03:00
ghost	80b33f619c	fix PAGES_LIMIT condition	2023-08-05 19:24:21 +03:00
ghost	d024ffd770	implement unlimited settings customization for each host	2023-08-05 19:06:39 +03:00
ghost	ab6c0379c8	implement hosts crawl queue, move robots, sitemaps, manifests to this task	2023-08-04 09:32:12 +03:00
ghost	6ee5e53ef4	show sitemaps processed debug	2023-08-04 09:07:46 +03:00
ghost	71724ae33f	refactor manifest crawling	2023-08-04 09:00:03 +03:00
ghost	cb37c57bc4	rename example files	2023-08-03 18:49:29 +03:00
ghost	68d5820f30	reserve one hour for huge load operations	2023-08-03 18:47:39 +03:00
ghost	efbbf19601	fix multimedia snaps	2023-08-03 17:41:55 +03:00
ghost	6862fb35cd	update readme	2023-08-03 15:33:34 +03:00