focus for this week: my research helpers
feedback? this wiki is running since 2009

Twister experience

From I ask questions
Jump to: navigation, search

why I like twister

2016 March: I'm using now ZeroNet:

  • see my blog post there: "Will ZeroNet make me leave twister ?" (2016 March 5)


since my original report below:

more from a chat (June 10, 2014):

my goal was to get people to install SN/GS for themselves, but that goal fails/failed (only few are doing that)
but in twister:
everyone I talk to is self-hosting their twister
I don't want people making beehives bigger (they "get away" from e.g. twitter just to make another beehive)
they should run things themselves


e.g. (the company) twitter: I am sure one day they'll get controlled on their practice(s),
but who will do that for a private person hosting the beehive?
it's a trust issue
you trust, that the private admin where you are, is good
you trust, that such an admin is prepared to protect you against attackers (where e.g. twitter would have resources for sure). see also easier access of your data?
instead of that:
be sure + run yourself


btw: SN/GS can also be run from your home (not a VPS, remote server)
but it's much harder (e.g. pagekite, dydns...)
e.g. I run 2 instances of SN/GS right now, I ran friendica for years...
but twister (or Freenet): just works
  • chat from 2015 August 14 here

improvement potential for twister (aka: why I don't like twister)

2015 July 24

joined twister again after some break:

  • first I ran into this issue:
    • my port 29333 is blocked, which is needed by twister
    • I am not the owner of the router, so bad for me, I guess
      • solution was and still is: to use tor
  • I missed these 3 things immediately:
    • >140 chars per posting
      • as it explains there: the change could also be done in the client, not necessarily the "whole twisterd" needs to be changed with possible side effects
    • favouriting posts
    • viewing top10 posts (over e.g. 2 months)
  • These I missed a little later:
    • referencing a twist (via e.g. url)
    • the search doesn't yield all past results (so, one has to use search engines like twistnik, but then you leave your IP trace in the web (site))

2015 Feb 23

50 mins until private message is received
yesterday we exchanged private messages, but it took too long :-(
  • situation:
    • my twister 0.9.28 runs over tor
      • I have 6-8 connections AND all of them are active DHT nodes also
    • user K's twister is not over tor
  • results:
    • user K gets my DM immediately
    • it took 50 mins until I received the DMs by K :-(
    • we got the exchange done via tox ;-)
      • I also used there tor

2015 Feb 22

char limit + no like-button
well, I have been using and also not using twister. some things why I don't think twister will change fast is:
the char limit and not having a like/favourite button
if it takes so long to not even have that, I don't think other things will come sooner
having built it on 140 chars is already a design flaw in mind

my first experience(s) with twister

intro

I used twister 2 times so far:

observations

the following are observations of this second usage:

  • finding contacts:
    • I knew users from the !fediverse so it was easy for me,
      • not sure if the "Who to follow" would have lead to a satisfying experience though :-(
      • one has to click through tags or others following/followers list to find people
      • bugtracker
    • I was reminded of the times when social systems just started (tent, pump.io,...):
      • not many there yet and one has to do extra effort to find people/info :-(
  • you may not see posts until you visit another's profile :-( bug
    • "DHT is based on UDP, that is, it is an unreliable protocol"
    • possible solution:
      • "a retry mechanism: mentions would be temporarily stored in local computer until it confirms it got accepted into the DHT."
    • see also DHT traffic may be TCP-tunneled
  • Why is the 140 char limit not yet gone?
    • make a system that's better than e.g. twitter, other distributed social systems can do that already
    • my feeling
      • feeling changed after 4 days:
        • "if a twister client would have hierarchical threading, perhaps I'd not want to surpass the 140 char limit that much?"
    • e.g. I could feed in my posts from other networks, but then I'd need to use url shortener
      • I can use a self-hosted one, but still: I trust links only when I see their final destination
    • temp. solution: activate in options "split long posts"
  • I wonder what costs more energy?
    • running twister on home devices (not adjusted for minimal power consumption) or using a VPS (with e.g. GNUsocial)?
      • it's not directly comparable, I know
      • but e.g. here on the laptop through the cpu load I have less resources for other tasks now
      • I let twister run for half a day and when I stopped it: I gained 900 MB RAM back (block generation was deactive also)
        • this machine has in total 2 GB RAM only
        • see bugtracker
      • how to disable "block generation"?
        • in your settings: make sure you have "block generation" disabled
        • other things that show it's disabled: Hash rate must be Zero when no blocks are generated
        • or check via command line: twisterd getmininginfo
    • this just shows that one has to pay a cost for (a certain) freedom[1]
      • not sure everyone wants to do that
  • I was missing info about twister's usage on my system
    • e.g. size of downloaded chains, memory, cpu usage and show stuff like warnings when limit is in reach
      • I can gather these myself, but it's still effort
      • bugtracker
    • see also Twister#twister stats
  • clients:
  • I wondered if twister messages have a URL?
    • so I could post them outside of the system and bring interested people to there
    • see bugtracker
  • comparison: now and (13 Jan):
    • block chains: 34548 (18977),
    • peers: 6929 (768), but: last block time: May 6 2014, so I guess needs to be still downloaded
    • see also Twister#twister stats
  • one can't delete a message anymore [3]
  • I wonder how many messages (public, private) have been written so far in total in twister?

in progress

2015-09-13

group chat

see feature here


  • what we know:
    • any/only group members can invite new group members. non group members can NOT join a group otherwise
    • encryption used
    • history viewable
      • new group members can see group posts from before they joined
        • but if the chat was too long, it doesn't show all history
        • very likely this is "only" a syncing issue: until twister loads all history
  • questions:
    • groups are decentral ?
      • I assume the group one creates on twister instance A is also available decentral.
        • e.g. when the twisterd on A isn't running: group members can still use the group chat ?
          • I think this is true since for each group member the *something is different
      • "twister's group chat is a special case of direct messages using a private key pair"
    • a few group members had problems:
      • creating own groups
        • > Error: can't invite 1 to foo group — value is type int, expected array
        • but, later: @talex "all those groups with the error got created. they all arrived suddenly in my menu. Lots of async things happening here"
      • inviting others to groups
    • will group messages move with me, when I switch the pc (since I'd use only secret key on new pc)?
      • yes, one has to use the command: rescandirectmsgs
      • related: what happens when all group members leave a group:
        • all messages can't be accessible, since encrypted.
        • So, is there "waste" lying around?
          • "twister's group chat is a special case of direct messages using a private key pair"
  • wishes:
    • want to change/upload avatar
    • @mrb addressing a group directly from the post-interface would be nice (thinking along lines of !group syntax)
    • Green tick.svg Done works now in original theme twister-html: @aqeeliz There are no names with posts, just avatars, could be confusing for people who don't know everyone in group.

2015-09-10

comparing twister with other platforms

2015-08-17

RAM increase
info

We're currently only collecting RAM values from different people/versions to see if there's a leak in twisterd.

(This doesn't involve yet deeper things like:

  • what actions are being done in twisterd
  • how many peers/DHT nodes
  • ...

Perhaps all above is not needed, when we can see an increase on all people's twisterd)



The below values can also be used to compare future twisterd versions

RAM observations

a decrease is marked in bold

twisterd version from fresh start generates
blocks?
comment
0.9.30 erkan 70 MB 91 MB
(13h)
101 MB
(48h)
109 MB
(62h)
110.4 MB
(74h)
107.4 MB
(85h)
110 MB
(110h)
135 MB
(172h)
152.1 MB
(224.5h)
158.9 MB
(268h)
+56% in 3 days
0.9.31 81.8 MB
(after 12h)
with tor using
additionally 23.7 MB RAM
28 peers
0.9.30 mrvdb 90 MB
(about 1h running)
120 MB
(36h)
129 MB
(50h)
132 MB
(73h)
+47% in 3 days
77.3 MB 98 MB
(after 34 mins)
0.9.31 157 MB
(after 12h)
0.9.31 186 MB
(after 66h)
0.9.31 111 MB
(after 6h)
0.9.30 vegos 101 MB
(63h)
121.8 MB
(73h)
x
+21% in 10h
0.9.30 47.4 MB
(after 10 mins)
55.5 MB
(after 30 mins)
60.2 MB
(after 1h)
64.9 MB
(after 5h)
65.3 MB
(after 10h)
x
+18% in 9.5h
0.9.30 69.6 MB
(after 30 mins)
82.5 MB
(after 10h)
84.9 MB
(after 12.5h)
x
+22% in 10h
0.9.31 60.7 MB
(after 15 mins)
72 MB
(after 30 mins)
73.5 MB
(after 60 mins)
75.7 MB
(after 2h)
93.1 MB
(after 8h)
94.1 MB
(after 12h)
x
0.9.31 26.6 MB 41.4 MB
(after 10 mins)
66.9 MB
(after 12h)
x
0.9.31 65.9 MB
(after 3.5h)
67.1 MB
(after 5h)
69.8 MB
(after 10h)
72.3 MB
(after 36h)
93.3 MB
(after 48h)
97.5 MB
(after 108h)
x
0.9.31 testtwister0123 62.1 MB 79.4 MB
(after 84 mins)
81 MB
(after 135 mins)
80.5 MB
(after 3h)
x
0.9.31 48 MB 68.3 MB
(after 1h)
82.3 MB
(after 2h)
87.3 MB
(after 9h)
x

2015-07-31

dhtget: takes too long to receive (A sends to B, on same pc)

The reason for this phenomenon is:

  • "dht content has to be stored by some of the 8 random nodes near the hash. and then retrieved."



2015-09-14: up to 3 weeks delay, see here
2015-08-17: up to 9 days delay, see here

details (from 2015-07-31):


Observed over last days:

  • The case below has now the situation where A + B are on the same pc, but:
    • I have also seen this when C (somewhere else) sends to B
  • I can understand that twister being fully decentralized P2P and so on, X may take longer to arrive
    • but for the human mind: A + B at the same place
      • A + B should have a "direct line" at least




A + B on same pc (twister over tor, runs now for 41h, 24 connections, with 22 active DHT nodes):
  • A sends post X to B
    • OK: A and B can see X in the UI
    • BUT: B doesn't get X via dhtget
      • nothing in 36 mins, B has made via dhtget 12x a request[2]
      • nothing in 75 mins
      • still nothing (but after 100 mins: I see that B (my bot) replied to 2 other user's posts. So, it's not broken completely)
  • I am also observing different results in those 12 requests.
  • Must I use another call?
  • or does the dhtget method ask the "crowd"
    • instead of using "something" already being here? Here = same pc, where A + B are on
  • I notice that X doesn't yet appear here also in 36 mins
    • appears here after 82 mins
      • not sure when exactly in the time frame [36 mins; 82 mins] it appeared there
  • but X has no sandclock on the twist either,
    • so: >2 peers have accepted the post
      • since I am over tor: is the distribution that slow ? :-(
  • I've verified with 2 other users that we share same connecting peers, so I should be connected over them and they are connected to me
  • todo:



see also:

2015-07-28

FIXED: sandclock on promoted post

Green tick.svg Done FIXED on the same day!

  • OK: block was mined; the promoted post was finished on: Mon Jul 27 2015 05:37:14
  • but: it shows the ⌛ (see pic above)
    • the sandclock tends to appear when something is blocked (e.g. here the peers had to accept the msg for the sandclock to disappear. the port 29333 was blocked, but when going over tor it was solved)
  • while my normal posts/twists don't have a sandclock

It's not that I worry that my block wasn't accepted, because:

  • after 7h I see 34 confirmations (see getblock output below)
  • also the @blockhash bot has seen it

But I want to know if something needs to be done more, e.g.

  • blocks need another port to be open?
  • block must be confirmed by more than 34 twister instances ?
./twisterd getblock afb20f40aa00d74f7b1910f96d14a96070fa4944fcecc98d154526fb7ceefb73
{
   "hash" : "afb20f40aa00d74f7b1910f96d14a96070fa4944fcecc98d154526fb7ceefb73",
   "confirmations" : 34,
   "size" : 237,
   "height" : 97303,
   "version" : 2,
   "merkleroot" : "499045795c0d12bc19c1667f98b6e3014ac291a9f57dfecf14fe9c23f5032799",
   "tx" : [
       "499045795c0d12bc19c1667f98b6e3014ac291a9f57dfecf14fe9c23f5032799"
   ],
   "spamMessage" : "@erkan_yilmaz is back in #twister - oh, really?",
   "spamUser" : "erkan_yilmaz",
   "userhashes" : [
   ],
   "usernames" : [
   ],
   "time" : 1437968234,
   "nonce" : 62167,
   "bits" : "1d2e5908",
   "difficulty" : 0.02157568,
   "previousblockhash" : "1583e1f086e956db4d2c44b3f51798f9e47f5840d78a56a4483031e02ef97590",
   "nextblockhash" : "004e85fe8a00893e0e4ee65d09b149d0867c1b261965ade76e5097ef2927e743"
}

2014

FIXED: daemons wouldn't exit, needed reboot

this is probably fixed (with the mem leak fix in version 0.9.21), didn't have it anymore since then:

  • OK: I started a few times successfully the daemon (didn't want RAM/CPU wasted here) during the day
  • BUT then:
    • Error: Cannot obtain a lock on data directory /home/YOUR_USER/.twister. Twister is probably already running.
    • couldn't kill the process, at all
  • so, reboot
  • trying to start twisterd but:
    • Speicherzugriffsfehler (Speicherabzug geschrieben)

debug.log says:

Error: Cannot obtain a lock on data directory /home/YOUR_USER/.twister. Twister is probably already running.
libtorrent + dht stopped
Flush(false) db not started
StopNode()
Flushed 0 addresses to peers.dat  89ms
Flush(true) db not started
Error: Cannot obtain a lock on data directory /home/YOUR_USER/.twister. Twister is probably already running.
libtorrent + dht stopped
Flush(false) db not started
StopNode()
Flushed 0 addresses to peers.dat  43ms
Flush(true) db not started

notes

  1. Jump up I'm currently on a low-end hardware and can NOT start twister because I don't have the resources available. ON another system I could just open the browser and communicate. Not here :-(
  2. Jump up 14:22, 14:25, 14:27, 14:32, 14:36, 14:39, 14:41, 14:45, 14:49, 14:50, 14:52, 14:56

see also