KHJK

KHJK http://www.khjk.org/ Karl Hans Janke Kollaborativ de sm@khjk.org (Sven Moritz Hallberg) 25 Jul 2010 00:00 GMT 25 Jul 2010 00:00 GMT Command line flags for Haskell scripts, Really Cheap & Simple (TM) http://www.khjk.org/log/2010/jul/getflag.html How often do you find yourself in the following situation? Having written a nice little program, you just want to add one little command line option. Typical example: -d — <q>print debugging output</q >. So all you want to say (in code) is something like this: <blockquote><pre><code>If we were given the '-d' flag on the command line, ... [print/enable debug output] </code ></pre ></blockquote >But what you actually have to do is usually this: <ul><li>Get a hold of your favourite <q>getopt</q > library. If it's not a standard library, make sure to note that dependency somewhere! </li ><li>Find the documentation and look up basic usage. </li ><li>Specify to the library exactly which options you want to accept, whether they take any arguments, and provide a usage note for the autogenerated help text. </li ><li>Make a loop in your main routine for <q>parsing all options</q >. </li ><li>Finally, in that loop, state that -d enables debugging output. </li ></ul >Seem a bit much? Not that there is anything wrong with the above in principle. These libraries are very useful for writing compilers and faithful reproductions of historic UNIX utilities. But for your everyday script? I think it's annoying. So, speaking of returns to simplicity. These are for pasting into your next Haskell script: <pre><code>import System.Environment -- pesco's really cheap and simple flags and options (tm) clparts = getArgs >>= return . (\(a,b) -> (a,drop 1 b)) . break (=="--") getargs = clparts >>= \(a,b)-> return ([h:t| h:t<-a, h/='-' || null t] ++ b) getflags = clparts >>= \(a,_)-> return (concat [t| '-':t <- a]) getflag x = getflags >>= return . elem x getenv f v x = catch (getEnv v >>= return . f) (\_ -> return x) </code ></pre >Here, have some type signatures, too: <pre><code>getargs :: IO [String] getflags :: IO [Char] getflag :: Char -> IO Bool getenv :: (String -> a) -> String -> a -> IO a </code ></pre >Usage of the functions is pretty apparent from the types. Don't look anything up. Please! From five lines you get: <ul><li>Boolean flags: Anywhere in your (IO) code, ask whether a certain flag has been specified: <pre><code>d <- getflag 'd' when d $ putStrLn "debug on!" </code ></pre >Alternatively, get the list of flags once and pass it around. <pre><code>flags <- getflags when all (`elem` flags) "nlp" $ putStrLn "my favourite options!" </code ></pre ></li ><li>Condensed flags: Throw any number of flag characters behind a single dash <q>-</q >: <pre><code>$ myps -aux </code ></pre ></li ><li>Options with defaults: This might feel strange, but seriously: Use environment variables. To query, complete with reader/parser and default value: <pre><code>n <- getenv read "n" 10 -- think "tail -n" c <- getenv read "C" 0 -- think "grep -C" -- default comes in from another IO action today <- getenv parsedate "today" =<< getcurdate -- cascading defaults plan <- getenv id "PLAN" =<< getenv (++ "/.plan") "HOME" =<< return "-" </code ></pre >BTW, don't think that setting an env var is more work than a command line argument! Compare: <pre><code>$ today=2010-07-26 watnu </code ></pre >vs. <pre><code>$ watnu --today=2010-07-26 </code ></pre >The so-called <q>keyword parameter</q > syntax of the former has been supported since the original Bourne Shell. </li ><li>Retrieve any non-flag arguments via the getargs routine. </li ><li>Arguments that begin with a dash: Anything after an argument of <q>--</q > is not a flag. A single dash <q>-</q > is also not a flag. </li ></ul >You buy simplicity at the expense of comprehensiveness. What you don't get: <ul><li>Long flags (e.g. <q>--debug</q >). You have 52 latin letters (upper- and lower-case) available. Plus any digits and special characters you can sensibly make use of. When you need more, use a real command line option parser. You're writing a goddamn compiler or something. </li ><li>Complaints about unrecognized flags or options. Defining the following function is left as the proverbial exercise to the reader: <pre><code>allowflags :: [Char] -> IO () </code ></pre ></li ><li>Command line options with arguments. Use environment variables. </li ><li>Autogenerated help and usage messages. Type them yourself, they're going to be so much prettier. Come on, add some ASCII art. When it really is too bothersome, feel free to define the following function: <pre><code>helpscreen :: String -- textual command description -> [(Char, String)] -- recognized flags with desc. -> [(String, String)] -- recognized env vars with desc. -> String </code ></pre ></li ><li>Elaborate error messages. Obviously there's nothing like <q>command line option --foo requires Frob argument, but got Twizzle</q > in those five lines. Then again, nothing is to stop you from putting arbitrarily complicated error checking in the reader you pass to getenv or validate the order and number of things in the result of getflags, etc.. </li ></ul >PS. I did make an elaborate command line parsing library a few years ago. Suitable for building big compilers and stuff! You might still find it in the caches of the intarweb if you're interested. Reading the original abstract, the goal was similar then: Make it really easy to get command line options into programs. Alas, using that library still suffered from problems outlined at the top of this post: Find it, look through the docs, add a big bunch of code to your project, use an elaborate API. I think it was a very good replacement for the standard GetOpt; considerably nicer, with many fancy features. But probably overkill for everyday use in scripts or small programs. pesco@khjk.org (pesco) http://www.khjk.org/log/2010/jul/getflag.html 25 Jul 2010 00:00 GMT A command-line organizer for <q>getting things done</q > http://www.khjk.org/log/2010/jul/watnu.html In case anyone hasn't noticed: I have a thing for simple solutions. So here is the third iteration of my personal to-do tracker. Previous versions held each task in a separate file, possibly along with a verbose description and several metadata fields. This one is a return to simplicity. As a special note, I am particularly proud of defaulting to the venerable <a href="http://catb.org/jargon/html/P/plan-file.html">.plan</a > file to hold the task list. I've always wished I had a proper use for it! From the <a href="http://code.khjk.org/watnu/README">README</a >: <blockquote><h2>basic operation </h2 ><ul><li>keep a list of tasks in ~/.plan, one per line. example: <pre><code> do something do something else </code ></pre ></li ><li>annotate them with due dates as appropriate <pre><code>12.7.2010! do something do something else </code ></pre ></li ><li>arrange roughly in order of priority </li ><li>group tasks by context, i.e. what could be done together <pre><code>do laundry scrub floor visit parents do something on way to parents house do something at parents' house </code ></pre ></li ><li>mark current tasks / <q>next actions</q > <pre><code>> laundromat scrub floor > read chapter on poly types </code ></pre ></li ><li>stall tasks until a certain date <pre><code>2010-08-20> bday present for dad 2010-08-25> bday present for sis </code ></pre ></li ><li>use hashtags to name projects <pre><code>code up first prototype of new #watnu blog about #watnu on #khjk </code ></pre ></li ><li>watnu will warn when a project has no tasks active or scheduled. </li ><li>use generic tasks to keep projects <q>on plan</q >: <pre><code>keep blogging on #khjk keep coding: #watnu #bitlbeeotr #noooo #home #friends #school </code ></pre ></li ><li>multiple date formats allowed. examples (all 1st of may): <pre><code>2010-05-01! iso date 1.5.2010! german 5/1/2010! us 1.5.! german w/o year 5/1! us w/o year </code ></pre ></li ><li>delegate tasks to other people, schedule activation as reminder <pre><code>10.7.> [mom] do laundry [timmy] scrub floor </code ></pre ></li ><li>call watnu to get your current todo list. order and grouping will be preserved from input. </li ><li>set PLAN environment variable to use a different input file </li ><li>set PLAN="-" to read from stdin </li ></ul ></blockquote >The concepts are my take on ideas from David Allen's <a href="http://en.wikipedia.org/wiki/GTD"><q>GTD</q ></a >. These are some data points on the design: <ul><li>Contexts never really worked out for me. Instead of ordering tasks by priority, GTD advocates grouping them by what can be done together. While I agree with that, trying a priori to name a good set of contexts was annoyingly cumbersome. </li ><li>It was hard to keep an overview when tasks were scattered around their separate files. This made the weekly review unpleasant. </li ><li>Long task descriptions are superfluous. Sometimes there would be a note, e.g. opening hours of a store, but the vast majority of tasks are one-liners. Tack-on notes can be added as a separate feature later. </li ><li>Only some things have hard due dates. In general, it is more important when I plan to start something rather than until when I must have completed it. </li ><li>Tasks of the <q>someday/maybe</q > category need not be tracked. Keep them in a different file. </li ><li>There are always tasks that are definately <q>to do</q > but should not appear on my radar, yet. They might be scheduled for later or simply buried by more pressing matters. </li ></ul >The program is just short of 180 lines of Haskell code. See the README file for build instructions. NB. I have found it a surprisingly nice routine to print a fresh todo list on a <a href="http://www.pocketmod.com">PocketMod</a > each morning, so I'm throwing in a dirty shell script as a free bonus. Get it here: <a href="http://code.khjk.org/watnu/">http://code.khjk.org/watnu/</a > pesco@khjk.org (pesco) http://www.khjk.org/log/2010/jul/watnu.html 6 Jul 2010 23:36 GMT Fibers for Ruby 1.8 in 42 lines feat. call/cc http://www.khjk.org/log/2010/jun/fibr.html Actually, stripped of comments and stuff, it's even less than 42 lines. <q>Fibers</q > are a big new thing with Ruby 1.9. The name is supposed to suggest that they're the thinner things inside a thread. You create them with a block to execute but they won't run in parallel. Instead they are explicitly entered and left via resume and yield. The first call to resume enters at the top of the block. Calling yield from inside the block jumps back out and the next resume jumps back inside. Repeat as often as you like. <pre><code>f = Fiber.new { ... Fiber.yield 23 # returns 5 ... } f.resume # start it up; returns 23 ... # control transfers back here after "yield" f.resume 5 # run the rest </code ></pre >The two can pass arguments to each other like regular procedure calls. So it's indeed like threads as there are independent control flows. But execution switches only explicitly and intercommunication is much more direct. Today I found out what they are really useful for: To separate some very sequential business logic (do A, do B, do C, finish) from the event-centric tangle of a GUI toolkit. A fiber does A, B, and C in sequence. Where <q>does</q > means it starts the job in the background, registers an event handler, and immediately yields back to the GUI event loop. When the job finishes, the handler resumes the fiber and it goes on to the next step. Ruby 1.9 adds fibers as a primitive, but they are also easily implemented in terms of call/cc - <q>call with current continuation</q >, which Ruby has had for no idea how long. So here goes, a drop-in substitute for 1.9's Fiber class: <pre><code>class Fibr @@fs = [] # a stack of fibers corresponding to calls of 'resume' def initialize(&block) @k = lambda(&block) # lambda makes 'return' work as expected end def resume(*xs) @@fs.push(self) jump(xs) # jumping into fiber end def self.current @@fs.last end def self.yield(*xs) f = @@fs.pop f && f.send(:jump, xs) # jumping out of fiber end private def jump(xs) callcc { |k| destination = @k @k = k destination.call(*xs) @@fs.pop @k.call # return from the last 'resume' } end end Fiber = Fibr if RUBY_VERSION<"1.9" </code ></pre >Download: <a href="log/2010/jun/fibr.rb">fibr.rb</a > pesco@khjk.org (pesco) http://www.khjk.org/log/2010/jun/fibr.html 7 Jun 2010 22:15 GMT Really simple distributed DNS http://www.khjk.org/log/2010/feb/rsddns.html There is a project, originating within the CCC, to link hackerspaces around the world into a <a href="http://media.ccc.de/foo/bar">common VPN</a >. Within this, we would like to provide DNS; probably under a fake TLD, say .hack. Well, you know, we like decentralized systems… This post is about a prototype distributed system that, give or take a few design iterations, could actually be useful for decentralized name resolution. Desirable properties: <ul><li>require only a minimum of local knowledge to configure the peers </li ><li>distributed self-organization instead of central administration </li ><li>avoid central points of failure </li ></ul >At the conceptual level, the network's nodes represent any kind of communications medium. I.e. anything that messages can be sent <q>onto</q > with the expectation that all entities also <q>on the medium</q > will receive them. Concretely there are two kinds of nodes: <ul><li>Low-level network links. IP-port pairs or other kinds of peer-to-peer addresses. The idea is that these could also be Ethernet interfaces and thelike. They carry no global name and are thought of as only locally known to a specific station (host system). </li ><li><q>Mesh</q > nodes (for lack of a better name). Formed by connecting stations in a mesh. Has a globally unique ID. For each node it participates in, a station knows how to reach its neighbors within that node. Any other node known to the station can act as the medium for these connections. Putting other meshes in this list effectively creates a tunnel. </li ></ul >Messages also come in two flavors: <ul><li>Payload. In the example code, these consist of a single String containing a message to be printed. </li ><li>Tunnel messages. When a message is sent onto a mesh node, the station wraps it with the ID of that node before sending it down the neighbor links (i.e. nodes). </li ></ul >When a station receives a message on a low-level link, it starts <q>unwrapping</q > the tunneling headers and propagates the message to any neighbors it has within the respective nodes. Given the above behaviour, it is possible to construct networks of nodes interconnected in rather arbitrary ways. For the purpose of emulating DNS, and probably many others, it is useful to think of nodes as either representing hosts or being formed by joining (<q>federating</q >) a number of subnodes. The federations would correspond to DNS domains. Determining how each station's table of node neighbor connections should look is left as an exercise to the reader. ;) The attached code contains a number of such context tables for an example network, pictured below. <div class="float" style="float:none"><div class="floatcontent"><a href="log/2010/feb/rsddns-proto.medium.jpg"><img src="log/2010/feb/rsddns-proto.klein.jpg" alt="rsddns-proto.klein.jpg" /></a ></div ><div class="floatcaption">Diagram of a hypothetical network showing federations (circles with green labels), hosts (squares), and low-level links (lines). Shown in red is the path of an example message from within a hamburg subnode to a host in cologne. </div ></div >To watch the example in action, run each of the following commands in a separate terminal: <pre><code>$ ghc rsddns-proto.hs -e 'mainloop 2001 laptop_ct' $ ghc rsddns-proto.hs -e 'mainloop 2002 daddel_ct' $ ghc rsddns-proto.hs -e 'mainloop 2003 router_ct' $ ghc rsddns-proto.hs -e 'mainloop 2004 turing_ct' $ ghc rsddns-proto.hs -e 'mainloop 2005 b1_ct' $ ghc rsddns-proto.hs -e 'mainloop 2006 b2_ct' $ ghc rsddns-proto.hs -e 'mainloop 2007 k1_ct' </code ></pre >Then inject the test message from an interpreter prompt: <pre><code>$ ghci rsddns-proto.hs > s <- testsocket 5555 > testsend s "127.0.0.1" 2001 $ Mtun "pesco.hh.ccc.hack" $ Mtun "hh.ccc.hack" $ Mtun "ccc.hack" $ Mtun "koeln.ccc.hack" $ Mtun "k1.koeln.ccc.hack" $ Mstr "hello world!" </code ></pre >Note the DNS-like hierarchical node IDs and how the nesting of messages reflects the path through this hierarchy from pesco.hh.ccc.hack to k1.koeln.ccc.hack. There is a 1s delay after each line of output so it's easy to watch the messages propagate. The destination node only prints out the message in this instance, but it would be trivial to include a return address and have it send back a useful answer (e.g. the station's IP address). You will see a lot of backscatter of messages. This is due to the fact that the stations only perform a minimal check whether they have already seen a particular message. A proper implementation should avoid more. Appendix: implementation prototype (Haskell): <a href="log/2010/feb/rsddns-proto.hs">rsddns-proto.hs</a > pesco@khjk.org (pesco) http://www.khjk.org/log/2010/feb/rsddns.html 12 Feb 2010 18:50 GMT Closed algebraic data types for Ruby http://www.khjk.org/log/2009/dec/ruby-adt.html Today, a little detour off the path to world domination. Or maybe not; who knows. I use <a href="http://www.rubylang.org">Ruby</a > at work and I guess it's pointless to deny that I regularly cringe at it for its rude failure at being just like Haskell. Even though, I have to admit that it is rather flexible: Algebraic data types in Haskell look like this: <pre><code>data Expr = Lam String Expr | App Expr Expr | Var String </code ></pre >It says that there are three kinds of expressions, called lambdas, applications and variables, consisting of the given data fields (e.g. a variable name and a body for lambdas, and so on). Operations on such a type are defined by giving cases for each kind of argument. Off the shelf, there are no algebraic data types in Ruby. Google didn't find anything, so I thought about it. With the right plumbing, it's possible to do the following: <pre><code>class Expr ctor :lam, :var => String, :body => Expr ctor :app, :op => Expr, :arg => Expr ctor :var, :name => String end </code ></pre >Not much longer and only slightly uglier! \o/ And with type checks, too. The result is a class Expr with three singleton constructor methods, similar to the usual new. Each takes a single hash as its argument that assigns values to the relevant fields. All fields must be provided and their types must match the definition. For example: <pre><code>id = Expr.lam(:var=>"x", :body=>Expr.var(:name=>"x")) # (\x.x) </code ></pre >The ctor method defines accessors to the constructor <q>tag</q > and each field. A typical method definition on Exprs would look like this: <pre><code>class Expr def eval(env={}) if is_lam? then self elsif is_app? then f = op.eval(env) if f.is_lam? x = arg.eval(env) f.body.subst(f.var,x).eval else raise "operator is not a function" end elsif is_var? then env[name] || self end end end </code ></pre >Obviously, this is in contrast to the <q>object-oriented</q > way of distributing operations on different kinds of things over respective (sub-) class definitions. Aside from being a matter of taste, I prefer the above in this case, because the definitions of evaluation for different kinds of expressions have to fit together. Their interaction is easier to view when the definition is in one place instead of scattered across multiple classes. To the plumbing: Everything happens in ctor, which is singleton method of Class, so it can be used in a class definition (similar to convenience methods like attr_reader and such). The routine looks like this: <pre><code>class Class def ctor(ctorname, argtypes) # add singleton constructor method to self self_ = (class << self; self; end) self_.module_eval do define_method(ctorname) do |argvalues| x = allocate # check argument types # [...] # fill instance variables of x x.instance_eval do @ctor = ctorname @args = argvalues end x end # constructor end # singleton methods of our ADT # inspection methods attr_reader :ctor attr_reader :args # define accessors and convenience methods on self # [...] end end </code ></pre >The full source code along with a complete lambda evaluator is in the file attached below. The evaluator also supports destructive in-place updates, effectively yielding graph reduction. Note: The copyright notice states <a href="http://opensource.org/licenses/isc-license.txt"><q>isc license</q ></a >, meaning do whatever you want as long as you leave the notice intact. Attachment: <a href="log/2009/dec/adt.rb">adt.rb</a > PS. Happy Birthday, you know who you are. pesco@khjk.org (pesco) http://www.khjk.org/log/2009/dec/ruby-adt.html 11 Dec 2009 13:37 GMT Do you have a MUA ready to follow mailto: links? http://www.khjk.org/log/2009/nov/email-comments.html <div class="float" style="float:none"><div class="floatcontent"><a href="img/2009/stift-uhr.medium.jpg"><img src="img/2009/stift-uhr.klein.jpg" alt="stift-uhr.klein.jpg" /></a ></div ></div >Just wondering, because if you do, you can now leave comments on posts on this site. Of course, my solution is unusual and specifically excludes HTML forms, CGI scripts or any other webcrap. Honestly though, I think it's kind of elegant like this: <ul><li>Follow the <q>reply</q > link on one of the blog entries. Or copy and paste the address. It has the form: <pre><code>comment-<path.to.post>@khjk.org </code ></pre ></li ></ul ><ul><li>Put your comment text into the body of the email. The subject is ignored. You can use email-style <q>ASCII art markup</q > like *asterisks* for bold face, /slashes/ for emphasis, >-style quotes, etc. </li ></ul ><ul><li>A <q>little</q > <a href="submitcomment.hs">script</a > (actually, it got partly hideous with all the <a href="Email.hs">email parsing</a > and whatnot) receives the message via procmail and decides to forward it for moderation. </li ></ul ><ul><li>I read the email like all the others (i.e: on my phone or whatever) and if it's ok simply forward it back to the comment address. This leaves my PGP signature on it. </li ></ul ><ul><li>The little script does a few checks, recognizes the signature, and stores the included message somewhere the <a href="generate.hs">site generator</a > will find it. </li ></ul ><ul><li>The site's HTML content is regenerated. If the From: address contained a name, that will be shown as the comment's author. To remain anonymous, leave out your name (that's a slogan). The address itself will never be revealed, of course. </li ></ul ><ul><li>To post a reaction to a comment, I can reply to the OP's email, leaving the comment address in Cc (group reply). In theory, the reply will already carry my signature and go directly through the little script and appear as a comment on the site. Unfortunately, this doesn't work in the current reality: Little script needs the signature over the entire comment message (a mime part of type message/rfc822) to check the metadata for consistency. This is the case with forwarded messages, but in a <q>regular</q > signed email, the signature only extends across the text/plain body. So, if you want to do me a great favour, add an option to mutt to sign headers by using a message/rfc822 part: <pre><code>multipart/signed application/pgp-signature message/rfc822 -- includes headers AND body </code ></pre >vs. <pre><code>multipart/signed application/pgp-signature text/plain -- only the message body :( </code ></pre ></li ></ul ><ul><li>Obviously, this game can go on through the entire thread of discussion by just keeping the comment address in Cc at all times. </li ></ul >Anyway, with this yak now out of the way, back to important work. I've got a few crazy ideas on which I'd very much appreciate feedback. Stay tuned. pesco@khjk.org (pesco) http://www.khjk.org/log/2009/nov/email-comments.html 22 Nov 2009 11:22 GMT <q>telnetd an</q > http://www.khjk.org/log/2009/nov/telnetd.html <div class="float" style="float:none"><div class="floatcontent"><a href="img/2009/kabelstraenge.medium.jpg"><img src="img/2009/kabelstraenge.klein.jpg" alt="kabelstraenge.klein.jpg" /></a ></div ><div class="floatcaption">random foto: elaborately wired contraption (Technikmuseum Berlin) </div ></div >I bought a Fritz!Box 7050 for cheap on eBay the other day. It's a home router with built-in DSL modem, WLAN, and a POTS interface. The reason I got it was that I'd like to ditch my aging, bulky, and increasingly flaky ISDN phone in favour of using my mobile as a VOIP handset for the Fritz!Box. The device itself works very well, including my old phone, WLAN, and ADSL2+. Unfortunately, as it turns out, the 7050 doesn't support VOIP handsets in the stock firmware. Google says, however, that you can actually run full-blown Asterisk on it. Now, the reason for this post is my amusement over the method to get a shell on the thing. Connect a phone, dial #96*7*, et voila: <div class="float" style="float:none"><div class="floatcontent"><img src="img/2009/telnetd.klein.jpg" alt="telnetd.klein.jpg" /></div ><div class="floatcaption">Well, thanks! :) </div ></div >A lot of frustrating frickel later, however, I've decided to give up for today. I got Asterisk to run, dialing out from the PC via X-Lite actually worked, but SIP registration fails on my Nokia E51, for unknown reasons. For later reference, here's the list of the relevant links I found: <ul><li><a href="http://www.avm.de/de/Service/Service-Portale/Service-Portal/index.php?portal=FRITZ!Box_Fon_WLAN_7050">http://www.avm.de/de/Service/Service-Portale/Service-Portal/index.php?portal=FRITZ!Box_Fon_WLAN_7050</a > </li ><li><a href="http://www.juerging.net/projekte/Fritzbox-Asterisk/">http://www.juerging.net/projekte/Fritzbox-Asterisk/</a > </li ><li><a href="http://www.wehavemorefun.de/fritzbox/Hilfsprogramme_/_Tipps_&_Tricks#Asterisk_.28capi_intern.2Bextern.2Fiax2.2Fsip.29_auf_der_7050_.28ohne_Firmware-Mod.29">http://www.wehavemorefun.de/fritzbox/Hilfsprogramme_/_Tipps_&_Tricks#Asterisk_.28capi_intern.2Bextern.2Fiax2.2Fsip.29_auf_der_7050_.28ohne_Firmware-Mod.29</a > </li ><li><a href="http://www.asterisk-kompakt.de/artikel/45-asterisk-auf-fritzbox-phone.html">http://www.asterisk-kompakt.de/artikel/45-asterisk-auf-fritzbox-phone.html</a > </li ><li><a href="http://www.spblinux.de/fbox/info/asterisk/sip.conf.default">http://www.spblinux.de/fbox/info/asterisk/sip.conf.default</a > </li ><li><a href="http://nuxx.net/blog/2008/10/24/sip-via-asterisk-on-nokia-e51/">http://nuxx.net/blog/2008/10/24/sip-via-asterisk-on-nokia-e51/</a > </li ><li><a href="http://www.ip-phone-forum.de/showthread.php?t=161714">http://www.ip-phone-forum.de/showthread.php?t=161714</a > </li ></ul > pesco@khjk.org (pesco) http://www.khjk.org/log/2009/nov/telnetd.html 1 Nov 2009 00:00 GMT Semiautomating Accountancy for Fun and Profit http://www.khjk.org/log/2009/oct/ledger.html <div class="float" style="float:none"><div class="floatcontent"><a href="img/2009/z23.medium.jpg"><img src="img/2009/z23.klein.jpg" alt="z23.klein.jpg" /></a ></div ><div class="floatcaption">Zuse Z23 @ Technikmuseum Berlin — definately a semiautomatic accountant! </div ></div >Is anybody out there using <a href="http://wiki.github.com/jwiegley/ledger">ledger</a > or one of its <a href="http://wiki.github.com/jwiegley/ledger/ports">siblings</a > for their personal accounting? If not, take this as a recommendation. It's a command-line tool to generate various financial reports from a plain text listing of account transactions. If you happen to have access to your bank transactions in csv format, the script I wrote yesterday may be useful to you. It reads comma-separated values from stdin and writes ledger entries to stdout. If you're German, your likely way to get csv files from your bank is via HBCI. The right tool for the job appears to be <a href="http://www.aquamaniac.de/sites/aqbanking/">aqbanking</a >. Hooking it up to the bank is a bit of fiddling, so I'll reproduce the quick how-to here. This is assuming authentication via PIN/TAN: <pre><code>$ aqhbci-tool4 adduser -t pintan --context=1 --hbciversion=300 \ -b BLZ -u NUTZERKENNUNG -c KUNDENKENNUNG \ -s SERVERURL \ -N "Real Name" $ aqhbci-tool4 getsysid -c KUNDENKENNUNG $ aqhbci-tool4 getaccounts -c KUNDENKENNUNG </code ></pre >To fetch transactions from all accounts and print them in csv format: <pre><code>$ aqbanking-cli request -c /tmp/foo.ctx --transactions $ aqbanking-cli listtrans -c /tmp/foo.ctx </code ></pre >The csv2ledger script is tailored to the default output format of the above. I also have made a small shell script to drive these two commands and pipe the result through the converter. It accepts an optional date range to which to restrict the output. Appendix: <ul><li><a href="log/2009/oct/csv2ledger.hs">csv2ledger.hs</a > converts comma-separated values to ledger entries. There are a few configuration settings at the top of the script to tell it about account names and the input format. </li ><li><a href="log/2009/oct/buchungen.sh">buchungen.sh</a > fetches transactions with aqbanking and shows them via csv2ledger. Usage: <pre><code>buchungen.sh [startdate [enddate]] # date format: YYYYMMDD </code ></pre ></li ></ul > pesco@khjk.org (pesco) http://www.khjk.org/log/2009/oct/ledger.html 5 Oct 2009 21:40 GMT The <q>Skein</q > hash function in 256 lines of C http://www.khjk.org/log/2009/sep/skein512.html Note: I'm going to switch this thing over to English now, because I expect to ask some non-Germans for feedback in the future. I might also translate some old posts. <div class="float" style="float:none"><div class="floatcontent"><a href="img/2009/teichufer.medium.jpg"><img src="img/2009/teichufer.klein.jpg" alt="teichufer.klein.jpg" /></a ></div ><div class="floatcaption">A view across the lake at HAR 2009 towards the CCC's geodesic party tent. Me and friends camped just about outside the right edge of the picture. </div ></div >So here's the latest installment of my exploits into the forbidden realm of implementing cryptographic primitives. After building my little crypto chat experiment last month, one thing sorely missing was message authentication (from p2p.c): <pre><code>printf("receiving packets on port %d\n", LOCALPORT); printf("CAUTION: Message senders can be spoofed.\n"); </code ></pre >The obvious solution to this problem are message authentication codes, particularly because the diffie-hellman setup already yields shared secrets between any two parties. A typical way to construct MACs is to take a cryptographic hash function and compute its value over a combination of the message and the secret (the standard construction of this kind is called HMAC). So I set out to find a nice little hash function which could be easily and elegantly implemented. Unfortunately, the obvious candidates didn't quite satisfy me. I kept looking and eventually ended up with the promising description of <a href="http://www.skein-hash.info/">Skein</a >: <blockquote>Skein is a new family of cryptographic hash functions. Its design combines speed, security, simplicity, and a great deal of flexibility in a modular package that is easy to analyze. </blockquote >Without much regret, I commited the next step up on the ladder of serious crimes in the construction of crypto systems: I set out to use an unproven algorithm. Hooray! >:) Incidentally, one very nice feature of Skein is that it already offers a mechanism to turn it into a keyed hash function. If my understanding is correct, this is essentially due to the fact that Skein is actually derived from a block cipher (actually called <q>Threefish</q > ;)). I have yet to implement this MAC mode, but it's basically a detail once the rest is set up. As of yesterday, the code finally produces the correct output on the official one-byte test vector. Feel free to try it on the longer ones. What took me so long? First, there was <a href="http://www.har2009.nl">HAR</a >. I had it pretty much complete at that point, except for one of those nasty segfault bugs. When I took a good hard look at things again this week, it turned out to be an overlong memset() corrupting my stack. God, I love those! ;) There are some limitations to the code at this point: <ul><li>Only the 512-bit variant of Skein is implemented. However, the threefish function is already generalized to any block size, so the others should be easy to add. </li ><li>The supplied main routine simply hashes the test vector, prints the result and exits. Anything more useful basically needs to wait for the next point: </li ><li>No support for directly slurping input from FILE handles. I've already prepared the code for it, just need to write the actual routine. </li ><li>As stated above, no MAC mode, yet. None of the other fancy stuff (personalization, tree hashing, …) either. Just plain and simple hashing. </li ></ul >Appendix: <a href="log/2009/sep/skein.c">skein.c</a > pesco@khjk.org (pesco) http://www.khjk.org/log/2009/sep/skein512.html 26 Sep 2009 15:40 GMT Peer-to-Peer Kryptochat http://www.khjk.org/log/2009/aug/p2p.html Meine Kryptoexperimente schreiten weiter fort. Am vergangenen Wochenende habe ich mir meinen <a href="log/2009/jul/elgamal.html">ElGamal</a >-Code nochmal vorgenommen und daraus ein kleines anonymes Peer-to-Peer-Netz gebaut. Man erwarte jetzt bitte nichts weltbewegendes, es handelt sich nach wie vor um kaum mehr als eine Fallstudie zu Montgomery-Multiplikation. Allerdings eine, ueber die man verschluesselte Kurznachrichten austauschen kann. ;) <div class="float" style="float:none"><div class="floatcontent"><a href="img/2009/serviette.medium.jpg"><img src="img/2009/serviette.klein.jpg" alt="serviette.klein.jpg" /></a ></div ><div class="floatcaption">That's the way we do it. Meine Serviette aus der Hotelbar in Berlin vor ein paar Wochen. </div ></div >Das ganze ist ein einfaches Broadcast-Netz: Jeder Knoten sendet eingehende Nachrichten, die nicht an ihn selbst addressiert sind, weiter an jeden seiner Nachbarn. Wenn er eine Nachricht schonmal gesehen hat, wird sie verworfen. Erwaehnenswert: <ul><li>Jeder Knoten im Netzwerk besitzt ein Schluesselpaar. Der public key ist seine Netzwerkadresse. </li ><li>Pakete bestehen komplett aus Chiffretext, keine weiteren Header. Der Empfaenger erkennt seine Nachrichten daran, dass er sie erfolgreich entschluesseln kann. </li ><li>Eigentlich klar, aber trotzdem toll: Das Netz ist unabhaengig vom Internet. Ich habe es fuer UDP implementiert, aber das ist willkuerlich. Amateurfunk, rohes Ethernet oder carrier pidgeons gingen auch. Insbesondere ist mir NAT egal, sobald ich das Ding fuer meinen OpenWRT-Router kompiliert habe. </li ><li>Es gibt vorerst keine Nachrichtenauthentisierung. D.h. keine Sicherheit, dass Nachrichten wirklich vom angebenen Absender stammen. </li ></ul > Anlage: <a href="log/2009/aug/p2p.tgz">Source</a > (jetzt auch Linux-kompatibel), <a href="log/2009/aug/README.p2p">README</a > PS. Neues Cleartext-Feature: Stichpunktlisten. pesco@khjk.org (pesco) http://www.khjk.org/log/2009/aug/p2p.html 7 Aug 2009 21:19 GMT