Basic command line utilities

README.md

@@ -57,14 +57,100 @@ Perhaps you could trade this information with their adversaries? The possibiliti
 
 ### How to play
 
-More info soon, meanwhile here's how to become a player:
+#### First you need to join
 
 * Every player should have a [twister](http://twister.net.co) account
 * <del>If you know how, add a line about yourself to [players.csv](https://github.com/Knights-of-Redact/DarkenedAges/blob/master/players.csv),
 (as a pull-request or something).</del> Bugger it. Just twist `@darkenedages I want to play #DarkenedAges` ;)
-* It is recommended to follow `@darkenedages` and have `#DarkenedAges` in your profile, but the formal definition of "player" is
+:* It is recommended to follow `@darkenedages` and have `#DarkenedAges` in your profile, but the formal definition of "player" is
 "one who appears at [players.csv](https://github.com/Knights-of-Redact/DarkenedAges/blob/master/players.csv)" ;)
 
+#### Sending a message
+In this example we send from and to fictional characters (NPCs) [the trustees are always "real" players].
+
+    $ ./dasend.py chuck flava -s "Can't truss it" < msg.txt 
+    == Publicly twist:
+    #Darkages #DA14342254974005 public: https://pastee.org/69f38#V4F8BjwgSfzkaentZlcBsacuLNM=
+    [No need to DM NPC] flava:
+    #Darkages #DA14342254974005 full https://pastee.org/4bsd6#QxCkPf7+G3qt4u+Ibj1pQVmxdvU=
+    == DM @thedod:
+    #Darkages #DA14342254974005 trustee broyo: https://pastee.org/hzrk2#DopLVbR1IWiROpQ00u9ncZH5+RA=
+    == DM @sandyclaws:
+    #Darkages #DA14342254974005 trustee sandyclaws: https://pastee.org/yt3dd#UWIbch+CM7+W7CMKYAHEIwsd93Y=
+    == DM @forth:
+    #Darkages #DA14342254974005 trustee forth: https://pastee.org/783rv#tbsqCly/fu4uKwzwAJ73gNbGC+I=
+    
+As we see, `dasend.py` tells us what to twist and DM (maybe one day this will be integrated, no rush).
+
+#### Receiving a message
+
+If this was really happening on twister, and I was @thedod, I'd only know the `public:` and `trustee broyo:` pastes.
+What I'd do would be:
+
+    $ ./daget.py https://pastee.org/69f38#V4F8BjwgSfzkaentZlcBsacuLNM=
+    ### getting https://pastee.org/69f38#V4F8BjwgSfzkaentZlcBsacuLNM=
+    # getting paste 69f38
+    # Hash Matches
+    # Wrote file: darkive/DA14342254974005/69f38.json
+    $ ./daget.py https://pastee.org/hzrk2#DopLVbR1IWiROpQ00u9ncZH5+RA=
+    ### getting https://pastee.org/hzrk2#DopLVbR1IWiROpQ00u9ncZH5+RA=
+    # getting paste hzrk2
+    # Hash Matches
+    # Wrote file: darkive/DA14342254974005/hzrk2.json
+
+The folder `darkive/` gets created if it doesn't exist.
+
+#### Unredacting a message
+
+    $ ./unredact.py darkive/DA14342254974005/
+    # Scanning folder darkive/DA14342254974005
+    msgid: DA14342254974005
+    sender: chuck
+    recipients: ['flava']
+    trustees: ['broyo', 'forth', 'sandyclaws']
+    subject: Can't truss it
+    
+    ████ ███ ██████ ████████ ███ a ███ ████
+    Because ██ that now █ ████ my █████
+    So ██████ █ ████ to the strong
+    █████ ███ █████ ██ the █████
+    and the smile ████ █████ ████ ████
+
+Note that I could also do `./unredact.py DA14342254974005` but you
+can prefix it with `darkive` and append `/` if that's what autocomplete
+tempts you to do ;)
+
+If `flava` was a real player, he'd do:
+
+    $ ./daget.py https://pastee.org/4bsd6#QxCkPf7+G3qt4u+Ibj1pQVmxdvU=
+    ### getting https://pastee.org/4bsd6#QxCkPf7+G3qt4u+Ibj1pQVmxdvU=
+    # getting paste 4bsd6
+    # Hash Matches
+    # Wrote file: darkive/DA14342254974005/4bsd6.json
+
+He could then simulate what `forth` and `sandyclaws` could do if
+they shared the keys between them
+
+    $ ./unredact.py darkive/DA14342254974005/ forth sandyclaws
+    # Scanning folder darkive/DA14342254974005
+    msgid: DA14342254974005
+    sender: chuck
+    recipients: ['flava']
+    trustees: ['broyo', 'forth', 'sandyclaws']
+    subject: Can't truss it
+    
+    King and chief, probably had █ big beef
+    ███████ of ████ ███ I grit ██ teeth
+    ██ here's a song ██ ███ ██████
+    'Bout the shake of ███ snake
+    ███ ███ █████ went along With that
+
+#### Todo:
+
+A utility to search the darkive by from/to/subject.
+
+#### The `darkened.py` library [under the hood]
+
 The file [output.txt](https://raw.githubusercontent.com/Knights-of-Redact/DarkenedAges/master/output.txt)
 was produced with `python darkened.py > output.txt`. You should get a similar output if everything works well.
 

daget.py

@@ -0,0 +1,60 @@
+#!/usr/bin/env python3
+import sys
+import os
+import json
+import darkened
+import pastee3
+
+__version__ = (0, 1, 0)
+
+pasteclient = pastee3.PasteClient()
+
+def todarkive(s,folder,filename):
+    """Store a string as a file in the "darkive", and notify the user"""
+    folder = os.path.join('darkive',folder)
+    os.makedirs(folder,0o700,True)
+    filename = os.path.join(folder,filename)
+    if os.path.exists(filename):
+        sys.stderr.write('# Skipping existing file! {}\n'.format(filename))
+    else:
+        open(filename,'w').write(s)
+        sys.stderr.write('# Wrote file: {}\n'.format(filename))
+
+def daget(pasteid):
+    pasteid = pasteid.split('/')[-1] # in case it's a full url
+    if '#' in pasteid:
+        p,needhash = pasteid.split('#')
+    else:
+        p,needhash = pasteid,None
+    sys.stderr.write('# getting paste {}\n'.format(p))
+    payload = pasteclient.sloppy_get(p)
+    if not payload:
+        sys.stderr.write('# bad or missing pastee!\n')
+        return False
+    if needhash:
+        gothash = darkened.hash64(bytes(payload.strip(),'ascii','replace'))
+        if gothash == needhash:
+            sys.stderr.write('# Hash Matches\n')
+            d = json.loads(payload)
+            todarkive(payload,d['msgid'],'{}.json'.format(p))
+            return True
+        else:
+            sys.stderr.write('# Hash mismatch! need {}, got {}. Corrupt paste?\n'.format(repr(needhash),repr(gothash)))
+            todarkive(payload,'corrupt','{}.corrupt.json'.format(p))
+            return False
+    else:
+        sys.stderr.write('# Not checking hash(!)\n')
+        d = json.loads(payload)
+        todarkive(payload,d['msgid'],'{}.unverified.json'.format(p))
+        return True
+
+if __name__=='__main__':
+    if len(sys.argv)<2:
+        sys.stderr.write("""Usage: {} pasteeurl[#hash] ...
+E.g. {} https://pastee.org/69f38#V4F8BjwgSfzkaentZlcBsacuLNM= hzrk2
+(second argument is in the shortest form: only id, no hash)
+""")
+        sys.exit(1)
+    for pasteid in sys.argv[1:]:
+        sys.stderr.write('### getting {}\n'.format(pasteid))
+        daget(pasteid)

dasend.py

@@ -0,0 +1,51 @@
+#!/usr/bin/env python3
+import sys
+import optparse
+import json
+import darkened
+import pastee3
+
+__version__ = (0, 1, 0)
+
+pasteclient = pastee3.PasteClient()
+
+def paste(value, indent=None):
+    b = bytes(json.dumps(value, indent=indent).strip(), 'ascii')
+    h = darkened.hash64(b)
+    return '#'.join((pasteclient.paste(b, ttl=365), h))
+
+def main():
+    parser = optparse.OptionParser(
+        usage='%prog [options] From To [To ...]',
+        epilog='This will create some pastee.org pastes, '
+            'and write stuff you need to copy/paste and tweet/DM')
+    parser.add_option("-s", "--subject", default="(untitled)",
+                      help=("Subject"))
+    parser.add_option("-d", "--debug", action="store_true",
+                      help=("Debug: don't crate pastes, dump as json to stdout instead"))
+    (options, args) = parser.parse_args()
+    if len(args)<2:
+        parser.print_help()
+        exit(1)
+    redaction = darkened.redact(sys.stdin.read(), sender=args.pop(0), recipients=args, subject=options.subject)
+    if options.debug:
+        json.dump(redaction, sys.stdout, indent=4)
+    else:
+        players = darkened.getplayers()
+        payload = redaction.pop('__public__')
+        pasteurl = paste(payload, indent=4)
+        print('== Publicly twist:\n#Darkages #{} public: {}'.format(payload.get('msgid', 'bug!!!'), pasteurl))
+        payload = redaction.pop('__to__')
+        pasteurl = paste(payload, indent=4)
+        for r in payload['recipients']:
+            if r in players:
+                print('== DM @{}:\n#Darkages #{} for {}: {}'.format(players[r]['twister'], payload.get('msgid', 'bug!!!'), pasteurl))
+            else:
+                print('[No need to DM NPC] {}:\n#Darkages #{} full: {}'.format(r, payload.get('msgid', 'bug!!!'), pasteurl))
+        for t in redaction:  # Only trustees left after popping those two
+            payload = redaction[t]
+            pasteurl = paste(payload, indent=4)
+            print('== DM @{}:\n#Darkages #{} trustee {}: {}'.format(players[t]['twister'], payload.get('msgid', 'bug!!!'), t, pasteurl))
+
+if __name__ == "__main__":
+  main()

unredact.py

@@ -0,0 +1,51 @@
+#!/usr/bin/env python3
+import sys
+import os
+import json
+import darkened
+import pastee3
+
+__version__ = (0, 1, 0)
+
+pasteclient = pastee3.PasteClient()
+
+def fromdarkive(msgid):
+    """Retrieve all files in a "darkive" folder, and returns cipher and consolidated pads"""
+    folder = os.path.join('darkive',msgid)
+    if not os.path.isdir(folder):
+        sys.stderr.write('# Folder does not exist! {}\n'.format(folder))
+        return None, None
+    sys.stderr.write('# Scanning folder {}\n'.format(folder))
+    cipher = None
+    pads = None
+    for f in os.listdir(folder):
+        # sys.stderr.write('Reading {}\n'.format(f))
+        try:
+            d=json.load(open(os.path.join(folder,f)))
+        except Exception as e:
+            sys.stderr.write('Error reading {}! {}\n'.format(os.path.join(folder,f),e))
+            continue
+        if 'cipher' in d:
+            cipher = d
+        else:
+            if pads:
+                pads['pads'].update(d['pads'])  # Todo: check conflicts?
+            else:
+                pads = d
+    return cipher, pads
+
+if __name__=='__main__':
+    if len(sys.argv)<2:
+        sys.stderr.write("Usage: {} msgid [trustee ...]\n")
+        sys.exit(1)
+    # Trick to allow e.g. 'darkive/DA14342254974005/' (like autocomplete does)
+    msgid = list(filter(None,sys.argv[1].split('/')))[-1]
+    cipher,pads = fromdarkive(msgid)
+    if cipher or pads:
+        d = cipher or pads
+        for k in ['msgid', 'sender', 'recipients', 'trustees', 'subject']:
+            if k in d:
+                print('{}: {}'.format(k,d[k]))
+    if cipher and pads:
+        print()
+        print(darkened.unredact(cipher,pads,trustees=sys.argv[2:]))