Building a Database-Backed Clojure Web Application

Last Updated: 21 March 2014

cedar clojure

Table of Contents

This article will explore creating a database-backed Clojure web application.

The app is called Shouter, a small Twitter clone that lets users enter in “shouts” which are stored in a PostgreSQL database and displayed on the front page of the app. You can see an example of the finished Shouter deployed to Heroku or view the finished source.

Let’s get started!

Prerequisites

Connecting to PostgreSQL with clojure.java.jdbc

Database persistence is important for many web applications, including this example app. Clojure’s officially supported libraries conveniently include the clojure.java.jdbc for database persistence through the JDBC standard.

To start, create a new project called shouter with Leiningen:

$ lein new shouter

In the shouter project, add the Clojure JDBC and PostgreSQL driver dependencies. Note that the version numbers here are current at the time of this writing. Newer version numbers may be available later, but compatibility is not guaranteed.

project.clj

(defproject shouter "0.0.1"
  :description "Shouter app"
  :url "http://github.com/abedra/shouter"
  :min-lein-version "2.0.0"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.5.1"]
                 [org.clojure/java.jdbc "0.3.2"]
                 [postgresql "9.1-901.jdbc4"]])

Rather than running PostgreSQL as a system-level background daemon as some package managers do by default, it’s recommended for development work that you launch postgres yourself to avoid permissions issues and improve visibility:

$ initdb pg
$ postgres -D pg &

If these executables aren’t found, try adding /usr/lib/postgresql/*/bin to your $PATH.

Then create a local PostgreSQL database for development work:

$ createdb shouter

Boot up a REPL to start experimenting, either from the command-line or in your editor.

$ lein repl

The first thing we’ll try is creating a table in the database. Bring in the JDBC functions:

user=> (require '[clojure.java.jdbc :as sql])
nil

This resolves the clojure.java.jdbc namespace and aliases it for use as sql. To create the table:

user=> (sql/db-do-commands "postgresql://localhost:5432/shouter"
                           (sql/create-table-ddl :testing [:data :text]))
(0)

This creates a table named testing inside the database with a text field named data. Let’s put some data into the data field:

user=> (sql/insert! "postgresql://localhost:5432/shouter"
                    :testing {:data "Hello World"})
({:data "Hello World"})

Now that you have created some data, ask the database to give it back:

user=> (sql/query "postgresql://localhost:5432/shouter"
                  ["select * from testing"])
({:data "Hello World"})

Excellent, the data is easily retrievable. Drop the table so it’s not hanging around for later:

user=> (sql/db-do-commands "postgresql://localhost:5432/shouter"
                           "drop table testing")
(0)

That’s all the database basics needed to get started. For a walkthrough of some other common questions, see this guide from clojuredocs.org.

Web bindings with Compojure

Clojure is a relatively young language. That being said, a clear path in web development has emerged with Compojure. This library is at the core of Shouter.

Compojure is built on top of Ring, a general-purpose web application library similar to Ruby’s Rack. Ring will implement much of the app’s low-level glue, while Compojure will provide a concise syntax with which to define application logic.

Add Compojure to the project.clj file along with a Jetty HTTP server adapter:

project.clj

If you're using an older version of Leiningen you may need to manually run lein deps to fetch the updated dependencies here.

(defproject shouter "0.0.1"
  :description "Shouter app"
  :url "http://github.com/abedra/shouter"
  :min-lein-version "2.0.0"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.5.1"]
                 [org.clojure/java.jdbc "0.3.2"]
                 [postgresql "9.1-901.jdbc4"]
                 [ring/ring-jetty-adapter "1.2.1"]
                 [compojure "1.1.6"]])

Now add some initial code in src/shouter/web.clj:

src/shouter/web.clj

(ns shouter.web
  (:require [compojure.core :refer [defroutes GET]]
            [ring.adapter.jetty :as ring]))

(defroutes routes
  (GET "/" [] "<h2>Hello World</h2>"))

(defn -main []
  (ring/run-jetty #'routes {:port 8080 :join? false}))

Your old REPL process will be out of date since you’ve changed your dependencies, so exit it and start a new one.

$ lein repl
user> (require 'shouter.web)
nil
user> (shouter.web/-main)
2013-01-06 15:44:44.393:INFO:oejs.Server:jetty-7.6.1.v20120215
#<Server org.eclipse.jetty.server.Server@dc67248>
2013-01-06 15:44:44.446:INFO:oejs.AbstractConnector:Started
SelectChannelConnector@0.0.0.0:8080

You can now point your browser to http://localhost:8080 to see the fruits of your labor. The next step is HTML templates.

HTML templating with Hiccup

Clojure has a wide variety of HTML templating libraries, but the simplest is Hiccup. Hiccup templates are just Clojure functions that emit HTML when called. Let’s add some simple HTML templating to the app with Hiccup.

First, add Hiccup to the project.clj file:

project.clj

(defproject shouter "0.0.1"
  :description "Shouter app"
  :url "http://github.com/abedra/shouter"
  :min-lein-version "2.0.0"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.5.1"]
                 [org.clojure/java.jdbc "0.3.2"]
                 [postgresql "9.1-901.jdbc4"]
                 [ring/ring-jetty-adapter "1.2.1"]
                 [compojure "1.1.6"]
                 [hiccup "1.0.4"]])

Now start a REPL and take a quick look at how Hiccup works:

user=> (require '[hiccup.core :as h])
nil
user=> (h/html [:h1 "Hello Word"])
"<h1>Hello Word</h1>"

You can use Hiccup inside of the current Compojure application to produce all the HTML needed. In src/shouter/web.clj, add hiccup.page to the ns declaration:

(:require [hiccup.page :as page])

Now add a simple index function:

(defn index []
  (page/html5
    [:head
      [:title "Hello World"]]
    [:body
      [:div {:id "content"} "Hello World"]]))

Finally, add the index function to the routes. Your routes should end up like this:

(defroutes routes
  (GET "/" [] (index)))

Restart your application. Rather than calling -main from the repl, you can use lein run -m shouter.web to start the server if you prefer. You should see "Hello World" in smaller text this time at http://localhost:8080. If you inspect the source you should see a proper HTML document as well.

Just like Compojure, Hiccup goes deeper. This brief introduction is enough to get you moving to your real objective.

Putting it all together

Now that the basics have been covered, you can build the full Shouter application.

You can find the complete code for the Shouter application here, including a resources directory with a set of styles. Feel free to copy this directory into your project to get the same look and feel.

For the app application itself, start with src/shouter/web.clj:

src/shouter/web.clj

(ns shouter.web
  (:require [compojure.core :refer [defroutes]]
            [ring.adapter.jetty :as ring]
            [compojure.route :as route]
            [compojure.handler :as handler]
            [shouter.controllers.shouts :as shouts]
            [shouter.views.layout :as layout]
            [shouter.models.migration :as schema])
  (:gen-class))

(defroutes routes
  shouts/routes
  (route/resources "/")
  (route/not-found (layout/four-oh-four)))

(def application (handler/site routes))

(defn start [port]
  (ring/run-jetty application {:port port
                               :join? false}))

(defn -main []
  (schema/migrate)
  (let [port (Integer. (or (System/getenv "PORT") "8080"))]
    (start port)))

You’ll notice that the index function is now removed and some additional routes are in place. The new -main function is how you will start the application from the command line, and the start function is how you will work from within a REPL. Notice the addition of the shouts controller namespace. Head there and fill in the code:

src/shouter/controllers/shouts.clj

(ns shouter.controllers.shouts
  (:require [compojure.core :refer [defroutes GET POST]]
            [clojure.string :as str]
            [ring.util.response :as ring]
            [shouter.views.shouts :as view]
            [shouter.models.shout :as model]))

(defn index []
  (view/index (model/all)))

(defn create
  [shout]
  (when-not (str/blank? shout)
    (model/create shout))
  (ring/redirect "/"))

(defroutes routes
  (GET  "/" [] (index))
  (POST "/" [shout] (create shout)))

The code above is the infrastructure for handling user actions. Working from bottom to top you find the piece of code linked in web.clj, the routes. This describes how routes should be handled in this controller. Using this technique, routes can be kept in each controller for reference, and then referred to in the global routes contained in web.clj. The index and create functions are just the simple responders. Next is the view layer:

src/shouter/views/layout.clj

(ns shouter.views.layout
  (:require [hiccup.page :as h]))

(defn common [title & body]
  (h/html5
   [:head
    [:meta {:charset "utf-8"}]
    [:meta {:http-equiv "X-UA-Compatible" :content "IE=edge,chrome=1"}]
    [:meta {:name "viewport" :content
            "width=device-width, initial-scale=1, maximum-scale=1"}]
    [:title title]
    (h/include-css "/stylesheets/base.css"
                 "/stylesheets/skeleton.css"
                 "/stylesheets/screen.css")
    (h/include-css "http://fonts.googleapis.com/css?family=Sigmar+One&v1")]
   [:body
    [:div {:id "header"}
     [:h1 {:class "container"} "SHOUTER"]]
    [:div {:id "content" :class "container"} body]]))

(defn four-oh-four []
  (common "Page Not Found"
          [:div {:id "four-oh-four"}
           "The page you requested could not be found"]))

This is the base for rendering views. It sets up a template that you can call in the display functions to reduce the amount of duplication. The four-oh-four function is just a small extension that is setup in the global routes. Now on to the real views:

src/shouter/views/shouts.clj

(ns shouter.views.shouts
  (:require [shouter.views.layout :as layout]
            [hiccup.core :refer [h]]
            [hiccup.form :as form]))

(defn shout-form []
  [:div {:id "shout-form" :class "sixteen columns alpha omega"}
   (form/form-to [:post "/"]
                 (form/label "shout" "What do you want to SHOUT?")
                 (form/text-area "shout")
                 (form/submit-button "SHOUT!"))])

(defn display-shouts [shouts]
  [:div {:class "shouts sixteen columns alpha omega"}
   (map
    (fn [shout] [:h2 {:class "shout"} (h (:body shout))])
    shouts)])

(defn index [shouts]
  (layout/common "SHOUTER"
                 (shout-form)
                 [:div {:class "clear"}]
                 (display-shouts shouts)))

This is the meat of the display logic. Notice how the addition of the code in layout.clj made things a little easier. Now that the front end is in place, move on to the data layer:

src/shouter/models/migration.clj

(ns shouter.models.migration
  (:require [clojure.java.jdbc :as sql]
            [shouter.models.shout :as shout]))

(defn migrated? []
  (-> (sql/query shout/spec
                 [(str "select count(*) from information_schema.tables "
                       "where table_name='shouts'")])
      first :count pos?))

(defn migrate []
  (when (not (migrated?))
    (print "Creating database structure...") (flush)
    (sql/db-do-commands shout/spec
                        (sql/create-table-ddl
                         :shouts
                         [:id :serial "PRIMARY KEY"]
                         [:body :varchar "NOT NULL"]
                         [:created_at :timestamp
                          "NOT NULL" "DEFAULT CURRENT_TIMESTAMP"]))
    (println " done")))

Here is the code to create the shouts table in the database and define the necessary fields. Running this will allow you to store shouts:

src/shouter/models/shout.clj

(ns shouter.models.shout
  (:require [clojure.java.jdbc :as sql]))

(def spec (or (System/getenv "DATABASE_URL")
              "postgresql://localhost:5432/shouter"))

(defn all []
  (into [] (sql/query spec ["select * from shouts order by id desc"])))

(defn create [shout]
  (sql/insert! spec :shouts [:body] [shout]))

The actual data model is quite simple. All you care about is input and collecting all of the shouts.

Entry point

An “uberjar” will be generated for deployment, which is simply a jar archive of your app’s code alongside all its dependencies. You’ll have to give Leiningen the entry point namespace under :main in project.clj for your app so that the uberjar knows where go:

(defproject shouter "0.0.1"
  :description "Shouter app"
  :url "http://github.com/abedra/shouter"
  :min-lein-version "2.0.0"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.5.1"]
                 [org.clojure/java.jdbc "0.3.2"]
                 [postgresql "9.1-901.jdbc4"]
                 [ring/ring-jetty-adapter "1.2.1"]
                 [compojure "1.1.6"]
                 [hiccup "1.0.4"]]
  :main ^:skip-aot shouter.core
  :uberjar-name "shouter-standalone.jar"
  :profiles {:uberjar {:aot :all}})

There’s also an :uberjar profile used to trigger AOT (ahead-of-time) compilation during the build process only. You can put :aot :all in the top level of your project.clj, but this can introduce problems during development, so it’s best to only perform this compilation during deployment. Finally Heroku needs the entry point to be declared in a platform-agnostic way in the Procfile:

web: java $JVM_OPTS -jar target/shouter-standalone.jar

With this, you have a complete application.

Testing locally

Generate an uberjar locally to ensure it works:

$ lein uberjar
Compiling shouter.web
Compiling shouter.views.shouts
Compiling shouter.views.layout
Compiling shouter.models.shout
Compiling shouter.models.migration
Compiling shouter.controllers.shouts
Created /home/phil/src/shouter/target/shouter-0.0.1.jar
Created /home/phil/src/shouter/target/shouter-standalone.jar

Finally, start your application according to what you’ve put in the Procfile:

$ java $JVM_OPTS -jar target/shouter-standalone.jar
2014-01-20 16:29:23.309:INFO:oejs.Server:jetty-7.x.y-SNAPSHOT
2014-01-20 16:29:23.372:INFO:oejs.AbstractConnector:Started SelectChannelConnector@0.0.0.0:8080

A fully functional Shouter app should now be running at http://localhost:8080.

Deploy

Now that you’ve confirmed everything works as expected locally, it’s time to deploy the app to Heroku’s Cedar stack.

Start by committing the app to Git:

$ git init
$ git add .
$ git commit -m "init"

Next, create the the app on Heroku:

$ heroku create
Creating stormy-fog-408... done, stack is cedar
http://stormy-fog-408.herokuapp.com/ | git@heroku.com:stormy-fog-408.git
Git remote heroku added

Now you need to provision a database for the app. Earlier you used the createdb command to provision a local PostgreSQL database. On Heroku we can use the Heroku PostgreSQL database add-on to provision a remote database:

$ heroku addons:add heroku-postgresql:dev
-----> Adding heroku-postgresql:dev to stormy-fog-408... done, v2 (free)

This will add a DATABASE_URL to the app’s environment, which the app will use at run-time to connect to its remote database resource. The Heroku Postgres Hobby Tier add-on is free.

Now that your database is provisioned you can deploy the code and scale the web dynos up:

$ git push heroku master
Fetching repository, done.
[...]
To git@heroku.com:stormy-fog-408.git
50084da..4b015a9  master -> master
$ heroku ps:scale web=1
Scaling web processes... done, now running 1

Your application should now be running. Confirm that the web process is up with:

$ heroku ps
Process       State               Command

web.1         up for 15s          java $JVM_OPTS -jar target/shouter-standalone.jar

Finally, check out the finished, live application in your browser.