Live-Debugging Remote Clojure Apps with Drawbridge
Last updated December 16, 2019
Clojure developers are used to a very interactive development
environment in which the editor is closely integrated with the running
process via a read/eval/print loop (REPL). It’s possible to reproduce
this to a degree with actual Heroku dynos using heroku run lein repl
,
but this creates a new process rather than connecting to existing
processes which are serving web requests. This makes it unsuitable for
certain debugging tasks. Using
nREPL and its
drawbridge transport, it’s
possible to connect a REPL session over HTTP, giving you direct access
to a web dyno.
The code for the app below is available on GitHub.
Setup
This article assumes the use of Leiningen 2. Download the latest release of Leiningen if you haven’t already.
Drawbridge is a transport for nREPL (Clojure’s networked REPL) that is implemented as a Ring handler, making it easily integrated into existing Ring applications. Let’s write a whimsical web application for calculating the magnitude of a given laugh.
$ lein new heroku-ring chortles
$ cd chortles
$ heroku apps:create
Creating hollow-stone-7912... done, stack is heroku-18
http://hollow-stone-7912.herokuapp.com/ | git@heroku.com:hollow-stone-7912.git
Git remote heroku added
Add com.cemerick/drawbridge
to the :dependencies
list in project.clj
:
(defproject chortles "1.0.0-SNAPSHOT"
:description "Just for laughs"
:url "http://chortles.herokuapp.com"
:dependencies [[org.clojure/clojure "1.4.0"]
[ring/ring-jetty-adapter "1.1.0"]
[com.cemerick/drawbridge "0.0.6"]])
Basic application
The benefits of the live REPL are clearer if the application has some
state to inspect; otherwise you might as well use the naive heroku
run lein repl
approach. Here is the implementation of our app in the
file src/chortles/web.clj
. It has no routing and simply returns a
JSON response.
(ns chortles.web
(:require [ring.adapter.jetty :as jetty]))
(defonce scores (atom []))
(defn calculate-chortle-magnitude [chortle]
(let [sub-chortles (re-seq #"(?im)ha+" chortle)
caps (apply + (for [sub sub-chortles c sub
:when (Character/isUpperCase c)] 1))]
(+ (count sub-chortles) caps)))
(defn percentile [magnitude]
(* 100.0 (/ (count (filter (partial >= magnitude) @scores))
(count @scores))))
(defn app [req]
(let [chortle (slurp (:body req))
magnitude (calculate-chortle-magnitude chortle)]
(swap! scores conj magnitude)
{:status 200
:headers {"Content-Type" "application/json"}
:body (format "{\"%s\": %s, \"percentile\": %s}"
chortle magnitude (percentile magnitude))}))
(defn -main [& [port]]
(let [port (Integer. (or port (System/getenv "PORT")))]
(jetty/run-jetty #'app {:port port})))
Along with returning the magnitude of a given chortle, it also
keeps a history of chortles in the scores
atom and returns a
percentile score for the given chortle compared to everything it has
calculated so far. Let’s get this deployed.
$ git init
$ git commit -a -m "Initial commit"
$ git push heroku master
It should be all set and ready to serve responses to curl:
$ curl http://hollow-stone-7912.herokuapp.com -d hahahaha
{"hahahaha": 4, "percentile": 100.0}
$ curl http://hollow-stone-7912.herokuapp.com -d HAHAHAhahaHA
{"HAHAHAhahaHA": 14, "percentile": 100.0}
$ curl http://hollow-stone-7912.herokuapp.com -d HAH
{"HAH": 3, "percentile": 33.33333333333333}
Drawbridge handler
Now you want to be able to access the application’s internal state.
This can be done by wrapping the handler with drawbridge middleware
back in our web.clj
file. The cemerick.drawbridge/ring-handler
function needs to be wrapped in a few other standard ring middleware
functions to pull the request apart; in a nontrivial app you would
probably already have declared these as part of your own code.
(def drawbridge-handler
(-> (cemerick.drawbridge/ring-handler)
(keyword-params/wrap-keyword-params)
(nested-params/wrap-nested-params)
(params/wrap-params)
(session/wrap-session)))
(defn wrap-drawbridge [handler]
(fn [req]
(if (= "/repl" (:uri req))
(drawbridge-handler req)
(handler req))))
(defn -main [& [port]]
(let [port (Integer. (or port (System/getenv "PORT")))]
(jetty/run-jetty (wrap-drawbridge app)
{:port port :join? false})))
This exposes an nREPL HTTP endpoint via ring middleware under the /repl
path. Leiningen’s repl task can now be used to connect once it’s deployed.
$ git commit -a -m "Add nREPL middleware."
$ git push heroku master
Ensure that it’s working fine:
$ curl http://hollow-stone-7912.herokuapp.com -d HAHAHAhahaHA
{"HAHAHAhahaHA": 14, "percentile": 100.0}
Now connect via Leiningen:
$ lein repl :connect http://hollow-stone-7912.herokuapp.com:80/repl
Welcome to REPL-y!
[...]
user> (in-ns 'chortles.web)
#<Namespace chortles.web>
chortles.web> @scores
[14]
All the code entered at the REPL runs in the deployed process, making it ideal for debugging hard-to-reproduce issues.
Requests to apps with multiple dynos will be evenly distributed among the dynos. This makes for potentially-confusing scenarios; it is best to attempt to reproduce issues on a single-dyno app.
Adding authentication
The problem is that at this point anyone could just go in and enter arbitrary Clojure code:
$ lein repl :connect http://hollow-stone-7912.herokuapp.com:80/repl
chortles.web> (defn app [req] {:status 410, :headers {}, :body "DELETED"})
This is certainly not acceptable; clearly some security is necessary.
The ring-basic-authentication
dependency provides Ring middleware
that makes it easy to protect certain parts of an app. Add this it to
your project.clj :dependencies
with
[ring-basic-authentication "1.0.1"]
and define an authenticated?
function to read credentials from environment variables:
(defn authenticated? [name pass]
(= [name pass] [(System/getenv "AUTH_USER") (System/getenv "AUTH_PASS")]))
With this function the drawbridge-handler
can be protected:
(defn wrap-drawbridge [handler]
(fn [req]
(let [handler (if (= "/repl" (:uri req))
(basic/wrap-basic-authentication
drawbridge-handler authenticated?)
handler)]
(handler req))))
Now add the credentials to the app’s config:
$ heroku config:set AUTH_USER=flynn AUTH_PASS=reindeerflotilla
Push the changes out:
$ git commit -a -m "now with security" && git push
Now the REPL is inaccessible to unauthenticated users:
$ lein repl :connect http://hollow-stone-7912.herokuapp.com:80/repl
ExceptionInfo clj-http: status 401
[...]
Bye for now!
But of course if credentials are provided (here in the URL scheme) it’s a different story:
$ lein repl :connect http://flynn:reindeerflotilla@hollow-stone-7912.herokuapp.com:80/repl
Welcome to REPL-y!
Clojure 1.4.0
[...]
user=> @chortles.web/scores
@chortles.web/scores
[13 22 1 42]
Inspect HTTP requests
It can be helpful for debugging purposes to intercept incoming
requests both to inspect and to re-submit them at your leisure. Since
Ring represents requests as simple Clojure maps, it’s easy to store
these off temporarily. In the REPL, enter this code in the chortles.web
namespace:
(defonce requests (atom []))
(defn app [req]
(swap! requests conj req)
(let [chortle (slurp (:body req))
magnitude (calculate-chortle-magnitude chortle)]
(swap! scores conj magnitude)
{:status 200
:headers {"Content-Type" "application/json"}
:body (format "{\"%s\": %s, \"percentile\": %.2f}"
chortle magnitude (percentile magnitude))}))
This is the exact same definition of the app
function except that it swaps the req
argument into an atom for you to play with later. The changes will take effect immediately with no deploy needed, so make a few requests over curl again:
$ curl http://hollow-stone-7912.herokuapp.com -d hahahaha
{"hahahaha": 4, "percentile": 100.0}
$ curl http://hollow-stone-7912.herokuapp.com -d HAHAHAhahaHA
{"HAHAHAhahaHA": 14, "percentile": 100.0}
$ curl http://hollow-stone-7912.herokuapp.com -d HAH
{"HAH": 3, "percentile": 33.33333333333333}
Then back to the REPL:
user=> @chortles.web/requests
[{:remote-addr "10.102.5.151", :scheme :http, :request-method :post [...]}]
There should be three large request maps. To re-submit one of these
requests, simply call the chortles.web/app
function on one of them
and it will be as if a user had submitted it from a web browser:
user=> (chortles.web/app (first @chortles.web/requests))
{:status 200, :headers {"Content-Type" "application/json"}, :body "{\"\": 0, \"percentile\": 25.00}"}
Note that the requests
atom will fill up and could be a potential
memory leak, so once you’re done debugging don’t forget to redefine
the app
function with the old version which doesn’t swap each
request onto the requests
atom. You can also flush the stored
requests list by calling (swap! chortles.web/requests empty)
.
Other tools that offer Leiningen integration can be connected to this
REPL endpoint. For instance, in Emacs invoking C-u M-x run-lisp
will
prompt for a command. Enter the lein repl [...]
invocation above and
Emacs will connect so that the standard code evaluation bindings like
C-c C-r
will evaluate the given code remotely. As nREPL integration
becomes more widespread other environments will implement direct
connection to remote nREPL servers without delegating that
functionality to Leiningen.