Archive for the ‘clojure’ Category

making roguelikes with Clojure

Just a quick post to remind me and anyone else what writing RLs in Clojure looks like today (of course the Java landscape here is vast, so I’ll just cherry pick a couple of Java libraries).

libs

(for Clojurescript)

http://tapiov.net/unicodetiles.js/

http://ondras.github.com/rot.js/hp/

(for Clojure)

https://code.google.com/p/blacken/

https://github.com/SquidPony/SquidLib

https://github.com/sjl/clojure-lanterna/

https://github.com/mikera/orculje

games

https://github.com/thomcc/dunjeon

http://stevelosh.com/blog/2012/07/caves-of-clojure-01/

https://github.com/mikera/alchemy (see the README for a link to the blog and a great series of blog posts in media 7DRL)

server in Clojure, fourth attempt

Getting closer! Continue reading

server in Clojure, third attempt, solely for posterity’s sake

This is not what you want to do.

(ns exvarcis.core
  (:import [java.net InetAddress InetSocketAddress Socket]
           [java.nio ByteBuffer]
           [java.nio.charset Charset]
           [java.nio.channels
            ServerSocketChannel SocketChannel Selector SelectionKey]))

(defn buf-seq
  ([buf] (buf-seq buf 0))
  ([buf i]
   (lazy-seq  
      (when (< i (.limit buf))
        (cons (.get buf i) (buf-seq buf (inc i)))))))

(let [c (atom 0)]
  (defn new-connection []
    {:id (swap! c inc)
     :writebuf (ByteBuffer/allocate 8096)
     :readbuf (ByteBuffer/allocate 8096)}))

(defmulti select-op
  (fn [k] (.readyOps k)))

(defmethod select-op :default [k]
  (println "Default op."))

(defmethod select-op (SelectionKey/OP_ACCEPT) [k]
  (-> 
    (.accept (.channel k))
    (.configureBlocking false)
    (.register (.selector k) (SelectionKey/OP_READ))
    (.attach (new-connection)))
  (println "Client registered"))

(defn close-connection [k]
  (println "closing connection" (.channel k))
  (.cancel k)
  (.close (.channel k)))

(defmethod select-op (SelectionKey/OP_READ) [k]
  (let
    [readbuf (-> (.attachment k) (:readbuf))]
    (.read (.channel k) readbuf)
    (.flip readbuf)
    (when (= (.limit readbuf) 0)
      (close-connection k))))

(defn select! [selector]
  (.select selector)
  (let [ks (.selectedKeys selector)]
    (doseq [k ks]
      (select-op k))
    (.clear ks)))

(defn get-lines [buf]
  (let
    [original-limit (.limit buf)
    last-newline (.lastIndexOf (vec (buf-seq buf)) 10)]
    (when-not (= last-newline -1)
      (.limit buf (inc last-newline))
      (let
        [lines (->>
                  buf
                  (.decode (Charset/defaultCharset))
                  .toString
                  clojure.string/split-lines)]
        (.limit buf original-limit)
        (.compact buf)
        lines))))

(defn write-lines [lines buf k]
  (let
      [encoded-lines (->> lines (.encode (Charset/defaultCharset)))]
    (when (> (.limit encoded-lines) 0)
      (.put buf encoded-lines)
      (.flip buf)
      (.write (.channel k) buf)
      (.compact buf)
      (println buf))))

(defn write-all [ks]
  (let
    [bufs (map #(-> (.attachment %) (:readbuf)) ks)
     lines (mapcat get-lines bufs)
     lines-with-crlf (str (clojure.string/join "\r\n" lines) "\r\n")
     bufs (map #(-> (.attachment %) (:writebuf)) ks)]
    (doseq [[buf k] (map list bufs ks)]
      (write-lines lines-with-crlf buf k))))

(defn run [selector]
 (while true
   (select! selector)
   (write-all (filter #(.attachment %) (.keys selector)))
   (Thread/sleep 3000)))

(defn start [handler]
  (let [selector (Selector/open)
        acceptor (->
                  (ServerSocketChannel/open)
                  (.configureBlocking false)
                  (.bind (InetSocketAddress. "127.0.0.1" 0)))]
    (.register acceptor selector (SelectionKey/OP_ACCEPT))
    (println "Starting Ex v Arcis on" (.getLocalAddress acceptor))
    (handler selector)))

(defn -main []
  (start run)
)

server in Clojure, second attempt

(ns exvarcis.core
  (:import [java.net InetAddress InetSocketAddress Socket]
           [java.nio ByteBuffer]
           [java.nio.channels ServerSocketChannel SocketChannel Selector SelectionKey]))

(defmulti select-op
  (fn [chkey server] (.readyOps chkey)))

(defmethod select-op (SelectionKey/OP_ACCEPT) [chkey server]
  (do (-> 
        (.accept (:acceptor server))
        (.configureBlocking false)
        (.register (:selector server) (SelectionKey/OP_READ)))
      (println "Client registered")))

(defmethod select-op :default [chkey server]
  (println "Default op."))

(defn start []
  (let [acc (-> (ServerSocketChannel/open) (.configureBlocking false) (.bind (InetSocketAddress. "127.0.0.1" 0)))
        sel (Selector/open)]
    (.register acc sel (SelectionKey/OP_ACCEPT))
    {:selector sel :acceptor acc}))

(defn -main []
  (let [server (start)
        sel (:selector server)]
    (println "Starting Ex v Arcis on" (.getLocalAddress (:acceptor server)))
    (while true
      (.select sel)
      (let [sel-keys (.selectedKeys sel)]
        (doseq [chkey sel-keys]
          (select-op chkey server))
        (.clear sel-keys)
        (Thread/sleep 3000)))))

Getting better — shortened -main by moving logic into multimethods. I like how those work; you don’t need a conditional branch to test which operation to do, which makes the doseq nice. In the case of a select call it’s not that big a deal since there are only a few operations, but what the hey. I now realize that dispatching functions using a map/dictionary rather than conditionals (something I gravitated to early in Python) serves kind of the same purpose, but multimethods are a more complete expression of the idea. That Thread/sleep is just there so I have time to watch stuff in the repl by the way — I’ll have to put a stop condition in the while there eventually too, right now I just interrupt it at the repl.

The while is just a macro for a loop/when/recur. I keep thinking that there’s some way to do that with an infinite lazy sequence, but maybe not.

Continue reading

A single-threaded multiplexing server in Clojure, first attempt

(ns exvarcis.core
  (:import [java.net InetAddress InetSocketAddress Socket]
           [java.nio ByteBuffer]
           [java.nio.channels ServerSocketChannel SocketChannel Selector SelectionKey]))

(defn start []
  (let [acc (-> (ServerSocketChannel/open) (.configureBlocking false) (.bind (InetSocketAddress. "127.0.0.1" 0)))
        sel (Selector/open)]
    (.register acc sel (SelectionKey/OP_ACCEPT))
    {:selector sel :acceptor acc}))

(defn -main []
  (let [server (start)]
    (println "Starting Ex v Arcis on" (.getLocalAddress (:acceptor server)))
    (while true
      (.select (:selector server))
      (let [sel-keys (.selectedKeys (:selector server))]
        (doseq [ch-key sel-keys]
          (cond
           (.isAcceptable ch-key)
           (do (->
                (.accept (:acceptor server))
                (.configureBlocking false)
                (.register (:selector server) (SelectionKey/OP_READ)))
               (println "Client registered"))))
        (.clear (.selectedKeys (:selector server))))
      (Thread/sleep 3000))))

No doubt due to the influence of the recent MudBytes thread about “keeping it simple” I’m currently having a go with writing Ex v Arcis from the ground up, more or less…hmm, I may have misinterpreted that thread. But anyway, here’s the first attempt. It’s very bad, but it gets the job done. Well, sort of — all it does at the moment is accept new connections.

What’s awful about it? I could be wrong as I’m just learning Clojure and barely know Java, but I don’t think I have a good handle on the Java interop yet. It’s a bit hard to figure out as at the moment it’s mostly all Java interop. Dealing with mutable Java collections (the selectedKeys for example of the Selector) with Clojure collection functions (like doseq) is still a little beyond my grasp. But it’s a start.

There are dozens (hundreds?) of example mud codebases out there if you want to learn how to write a server, and plenty of tutorials on basic single-threaded nonblocking TCP servers. I found a few tutorials that were especially helpful.

http://onjava.com/pub/a/onjava/2002/09/04/nio.html?page=1

http://rox-xmlrpc.sourceforge.net/niotut/index.html

http://www.owlmountain.com/tutorials/NonBlockingIo.htm#_Toc524339526

For Java interop with Clojure the standard Java docs are of course indispensable:

http://docs.oracle.com/javase/7/docs/api/java/nio/channels/ServerSocketChannel.html

Finally, if you had to look at one mud as an example, I think I would choose Miniboa:

http://code.google.com/p/miniboa/

The file async.py (http://code.google.com/p/miniboa/source/browse/trunk/miniboa/async.py ) is what you want, though the Telnet implementation is a great reference too.

I would be remiss if I did not mention Mire, https://github.com/technomancy/mire/blob/master/src/mire/server.clj , a multithreaded Clojure mud. Multithreaded muds (that is, a thread per client connection) in Clojure are rather nice to write compared with other languages because of Clojure’s concurrency story. Perhaps in the end that’ll be the way to go. But I had a hankering to write a single-threaded non-blocking server, so for now that’s the path I’ll take.

edit:

Somehow I missed this:

http://pepijndevos.nl/2011/06/18/nio-in-clojure.html

feeds and rivers

I’ve started to poke at something that’s been on the back burner for too long.

I put out IFURLs (a weekly curation and summation of interactive fiction links) for about a year, and while I enjoyed the process I felt like I reached a natural limit doing everything by hand after a while, so I put it on hold. The new idea (well, it’s certainly not new at this point) was to build a tool that would make the process easier and invite collaboration from like minded folks. I thought that while I was at it I could add feeds from outside the strictly-defined parser IF community, since I did that anyway with the list by hand. Choice-based games, muds, electronic writing of every kind — what I wanted to make was a huge firehose of all that stuff and then develop the tool to make it easy to filter and sort that information for the reader, the ultimate goal always being to present a weekly select list of the most interesting links. So it’s not a planet site, and not an add-a-link Reddit or Hacker News clone; I’ve seen these kinds of sites described as rivers and that’s not a bad description.

I’ve taken the first step of putting about 30 feeds (some are planets as well) on a page.

I’m using Clojure, noir-async, and clj-rome to put it all together.

Clj-rome wraps the rome-fetcher RSS library, and that seems to present the first issue. You can see in the screenshot that some of the items include dates and some don’t. It appears that rome-fetcher isn’t parsing some of the RSS pubdates correctly. The Mudbytes feed items for example includes pubdates, but I don’t get them in the rome-fetcher parsed feed.

I remember 5 or so years ago when I swore I never would touch Java code. I seriously had a natural aversion to it from the first time I had half an idea of what it was all about. I guess Time is the great Trickster, isn’t it?

I’m not sure how to proceed with the pubdate issue. There was a somewhat related issue filed at the rome-fetcher Jira page, and I commented on that. In the worst case I can add my own date based on when I pull down the feed item, since it’s not that important that my date syncs precisely with the actual post date of whatever I’m grabbing.

A nice thing about Java is its layers of abstraction. It feels industrial and solid. Of course that’s also a downside; getting at a problem seems to require a lot more peeling back of layers, though I think (hope) some of this complexity is just a lack of familiarity.