From 0b60445fea7f220ac6d5947bbbb3d30ce765a7fe Mon Sep 17 00:00:00 2001 From: ghost Date: Fri, 24 Nov 2023 18:00:15 +0200 Subject: [PATCH] update readme --- README.md | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 1995032..b0c11ed 100644 --- a/README.md +++ b/README.md @@ -1,29 +1,33 @@ -# Yo! - -Micro Web Crawler in PHP & Manticore +# Yo! Micro Web Crawler in PHP & Manticore Yo! is the next generation of [YGGo!](https://github.com/YGGverse/YGGo) project with goal to reduce server requirements and make deployment process simpler. +Index model changed to the distributed clustering model, and oriented to aggregate search results from different instances trough API. + +## Implementation + Engine written in PHP and uses [Manticore](https://github.com/manticoresoftware) search engine on backend. -Default build adapted for [Yggdrasil](https://github.com/yggdrasil-network) eco-system but could be used to make own search project. +Default build inspired and adapted for [Yggdrasil](https://github.com/yggdrasil-network) eco-system but could be used to make own search project. -Project contain: +## Components * CLI tools for index operations * JS-less frontend to make search web portal * API tools to make search index distributed -Features: +### Features * MIME-based crawler with flexible filter settings * Page snap history with local and remote mirrors support -## CLI +### Documentation + +#### CLI -### Index +##### Index -#### Init +###### Init Create initial index @@ -32,22 +36,22 @@ php src/cli/index/init.php [reset] ``` * `reset` - optional, reset existing index -### Document +##### Document -#### Add +###### Add ``` php src/cli/document/add.php URL ``` * `URL` - add new URL to the crawl queue -#### Crawl +###### Crawl ``` php src/cli/document/crawl.php ``` -#### Search +###### Search ``` php src/cli/document/search.php '@title "*"' [limit]