|
|
@ -2,9 +2,10 @@ |
|
|
|
|
|
|
|
|
|
|
|
Next generation of [YGGo!](https://github.com/YGGverse/YGGo) project with goal to reduce server requirements and make deployment process simpler. |
|
|
|
Next generation of [YGGo!](https://github.com/YGGverse/YGGo) project with goal to reduce server requirements and make deployment process simpler. |
|
|
|
|
|
|
|
|
|
|
|
Index model changed to the distributed cluster model, and oriented to aggregate search results from different instances trough API. |
|
|
|
- Index model changed to the distributed cluster model, and oriented to aggregate search results from different instances trough API. |
|
|
|
|
|
|
|
- Refactored data exchange model with drop all primary keys dependencies |
|
|
|
Codebase following minimalism such as possible. |
|
|
|
- Snaps now using tar.gz compression to reduce storage requirements and still supporting remote mirrors, FTP including |
|
|
|
|
|
|
|
- Codebase following minimalism everywhere as possible. |
|
|
|
|
|
|
|
|
|
|
|
## Implementation |
|
|
|
## Implementation |
|
|
|
|
|
|
|
|
|
|
@ -18,18 +19,18 @@ Default build inspired and adapted for [Yggdrasil](https://github.com/yggdrasil- |
|
|
|
* JS-less frontend to make search web portal |
|
|
|
* JS-less frontend to make search web portal |
|
|
|
* API tools to make search index distributed |
|
|
|
* API tools to make search index distributed |
|
|
|
|
|
|
|
|
|
|
|
### Features |
|
|
|
## Features |
|
|
|
|
|
|
|
|
|
|
|
* MIME-based crawler with flexible filter settings |
|
|
|
* MIME-based crawler with flexible filter settings |
|
|
|
* Page snap history with local and remote mirrors support |
|
|
|
* Page snap history with local and remote mirrors support |
|
|
|
|
|
|
|
|
|
|
|
### Documentation |
|
|
|
## Documentation |
|
|
|
|
|
|
|
|
|
|
|
#### CLI |
|
|
|
### CLI |
|
|
|
|
|
|
|
|
|
|
|
##### Index |
|
|
|
#### Index |
|
|
|
|
|
|
|
|
|
|
|
###### Init |
|
|
|
##### Init |
|
|
|
|
|
|
|
|
|
|
|
Create initial index |
|
|
|
Create initial index |
|
|
|
|
|
|
|
|
|
|
@ -38,22 +39,22 @@ php src/cli/index/init.php [reset] |
|
|
|
``` |
|
|
|
``` |
|
|
|
* `reset` - optional, reset existing index |
|
|
|
* `reset` - optional, reset existing index |
|
|
|
|
|
|
|
|
|
|
|
##### Document |
|
|
|
#### Document |
|
|
|
|
|
|
|
|
|
|
|
###### Add |
|
|
|
##### Add |
|
|
|
|
|
|
|
|
|
|
|
``` |
|
|
|
``` |
|
|
|
php src/cli/document/add.php URL |
|
|
|
php src/cli/document/add.php URL |
|
|
|
``` |
|
|
|
``` |
|
|
|
* `URL` - add new URL to the crawl queue |
|
|
|
* `URL` - add new URL to the crawl queue |
|
|
|
|
|
|
|
|
|
|
|
###### Crawl |
|
|
|
##### Crawl |
|
|
|
|
|
|
|
|
|
|
|
``` |
|
|
|
``` |
|
|
|
php src/cli/document/crawl.php |
|
|
|
php src/cli/document/crawl.php |
|
|
|
``` |
|
|
|
``` |
|
|
|
|
|
|
|
|
|
|
|
###### Search |
|
|
|
##### Search |
|
|
|
|
|
|
|
|
|
|
|
``` |
|
|
|
``` |
|
|
|
php src/cli/document/search.php '@title "*"' [limit] |
|
|
|
php src/cli/document/search.php '@title "*"' [limit] |
|
|
|