Browse Source

update readme

main
ghost 1 year ago
parent
commit
0b60445fea
  1. 30
      README.md

30
README.md

@ -1,29 +1,33 @@ @@ -1,29 +1,33 @@
# Yo!
Micro Web Crawler in PHP & Manticore
# Yo! Micro Web Crawler in PHP & Manticore
Yo! is the next generation of [YGGo!](https://github.com/YGGverse/YGGo) project with goal to reduce server requirements and make deployment process simpler.
Index model changed to the distributed clustering model, and oriented to aggregate search results from different instances trough API.
## Implementation
Engine written in PHP and uses [Manticore](https://github.com/manticoresoftware) search engine on backend.
Default build adapted for [Yggdrasil](https://github.com/yggdrasil-network) eco-system but could be used to make own search project.
Default build inspired and adapted for [Yggdrasil](https://github.com/yggdrasil-network) eco-system but could be used to make own search project.
Project contain:
## Components
* CLI tools for index operations
* JS-less frontend to make search web portal
* API tools to make search index distributed
Features:
### Features
* MIME-based crawler with flexible filter settings
* Page snap history with local and remote mirrors support
## CLI
### Documentation
#### CLI
### Index
##### Index
#### Init
###### Init
Create initial index
@ -32,22 +36,22 @@ php src/cli/index/init.php [reset] @@ -32,22 +36,22 @@ php src/cli/index/init.php [reset]
```
* `reset` - optional, reset existing index
### Document
##### Document
#### Add
###### Add
```
php src/cli/document/add.php URL
```
* `URL` - add new URL to the crawl queue
#### Crawl
###### Crawl
```
php src/cli/document/crawl.php
```
#### Search
###### Search
```
php src/cli/document/search.php '@title "*"' [limit]

Loading…
Cancel
Save