Testing with Datomic dev-local

A suggested approach to using Datomic dev-local for tests.

Installation

First, download dev-local by visiting the download site. The Datomic dev-local is not redistributable. We’d still like a way to make it easily available to all team members and CI. For this, we will upload the jar to a private S3 Maven repository.

Given a JAR file, there isn’t a great tool to simply put the necessary pieces into S3. I wrote a small library s3-mvn-upload to help with this. You can also do a simple S3 PutObject, but this will result in tools-deps complaining with this warning Download corrupted: Checksum validation failed, no checksums available.

To get the dev-local JAR file into S3, run the below command. Ensure you replace <version> with the correct Datomic dev-local version and <bucket> with the name of your S3 bucket.

clojure -Sdeps '{:deps {s3-mvn-upload {:mvn/version "0.2.0"}}}' -m s3-mvn-upload.core com.datomic/dev-local <version> dev-local-<version>.jar s3://<bucket>/releases

Ensure you have set up your AWS creds as documented in Maven S3 Repos. We can now depend on dev-local in a deps.edn file.

{:deps      {com.datomic/dev-local    {:mvn/version "0.9.195"}
             com.datomic/client-cloud {:mvn/version "0.8.102"}}
 :mvn/repos {"s3mvn" {:url "s3://<bucket>/releases"}}}

Usage

A common approach to testing with Datomic involves creating a fresh environment, transacting schema, finally adding test data. All we really need is the first piece — creating a fresh environment. The rest is use-case specific. To help with this, I released a library dev-local-tu. The library provides a way to create and delete Datomic "test environments."

I have found starting and stopping test components via with-open to be far better than fixtures. You get fine grain control over what starts and stops with each test. If you must, you can use fixtures instead (example)

Here’s an example usage of dev-local-tu with clojure.test.

(ns example1
  (:require
    [clojure.test :refer :all]
    [datomic.client.api :as d]
    [dev-local-tu.core :as dev-local-tu]))

(deftest test1
  (with-open [db-env (dev-local-tu/test-env)]
    (let [_ (d/create-database (:client db-env) {:db-name "test"})
          conn (d/connect (:client db-env) {:db-name "test"})
          _ (d/transact conn {:tx-data [{:db/ident       ::name
                                         :db/valueType   :db.type/string
                                         :db/cardinality :db.cardinality/one}]})
          {:keys [tempids]} (d/transact conn {:tx-data [{:db/id "a"
                                                         ::name "hi"}]})]
      (is (= {::name "hi"}
             (d/pull (d/db conn)
                     [::name]
                     (get tempids "a")))))))

We at Compute Software have been slowly moving tests that used datomic-client-memdb to use this approach instead. So far it has been working great. Happy to hear any feedback on this approach or approaches others have been taking.

Written on 2020-07-28