YGGverse/YGGo
1
0
Fork 0
mirror of https://github.com/YGGverse/YGGo.git synced 2025-01-08 22:07:56 +00:00
Code Issues Projects Releases Wiki Activity

add crontab configuration example, check roadmap item

Browse Source
This commit is contained in:
ghost 2023-04-09 00:07:06 +03:00
parent 8e8d89db0e
commit bee5086f22
1 changed files with 10 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
README.md
View File

@ -42,7 +42,14 @@ sphinxsearch
#### Configuration #### Configuration
todo ##### Crontab
```
0 * * * * indexer --all --rotate
0 0 * * * cd /YGGo/crontab && php cleaner.php > /dev/null 2>&1
* * * * * cd /YGGo/crontab && php crawler.php > /dev/null 2>&1
```
#### Roadmap / ideas #### Roadmap / ideas
@ -53,10 +60,10 @@ todo
* [ ] Improve yggdrasil links detection, add .ygg domain zone support * [ ] Improve yggdrasil links detection, add .ygg domain zone support
* [ ] Make page description visible - based on the cached content dump, when website description tag not available, add condition highlights * [ ] Make page description visible - based on the cached content dump, when website description tag not available, add condition highlights
* [ ] Images search (basically implemented but requires testing and some performance optimization) * [ ] Images search (basically implemented but requires testing and some performance optimization)
* [ ] Deprecated pages index cleaner (404 http codes etc) * [x] Index cleaner
* [ ] Crawl queue balancer, that depends from CPU available * [ ] Crawl queue balancer, that depends from CPU available
* [ ] Implement smart queue algorithm that indexing new sites homepage in higher priority * [ ] Implement smart queue algorithm that indexing new sites homepage in higher priority
* [ ] Implement database autobackup on crawl process completting * [ ] Implement database auto backup on crawl process completing
* [x] Add transactions to prevent data loss on DB crashes * [x] Add transactions to prevent data loss on DB crashes
* [ ] Distributed index data sharing between the nodes trough service API * [ ] Distributed index data sharing between the nodes trough service API
* [x] An idea to make unique gravatars for sites without favicons, because simpler to ident, comparing to ipv6 * [x] An idea to make unique gravatars for sites without favicons, because simpler to ident, comparing to ipv6