(s/fdef example-function :args (s/cat :x int))
(st/instrument `example-function)Clojure Spec instrumentation is immensely useful. When enabled, it provides a detailed error message when a function is invoked with invalid arguments. Time and time again, this has proven to be a big time saver. Using instrumentation as is has some challenges for my workflow.
While developing something new, my typical workflow goes something like this:
Open up a .clj file and use a keyboard shortcut to load it in the REPL.
Create a comment form below the function I am going to work on.
Write a sample call to the function I’m working on in the comment and send it to the REPL using a keyboard shortcut.
Make a change to the function.
Use the keyboard shortcut to load the file in the REPL.
Send the sample call to the REPL again.
Repeat from step 4.
To enable instrumentation for the function I’m working on, I need to require clojure.spec.test.alpha (either by adding a require in my comment block or adding it to my top level ns :require) and call the instrument function.
That’s a bit annoying but not terrible.
What becomes frustrating is anytime I make a change to my function, I need to re-enable instrumentation by calling instrument.
Again, that’s not that hard, but it’s another thing I need to remember to do.
Because of these two necessities, I end up never enabling instrumentation while developing.
My brain justifies it by writing it off as a short-term development efficiency.
Whether the result is actually efficient, I have no idea.
Either way, we can do better.
A couple years ago I wrote a library called defn-spec.
It includes a replacement macro for fdef which automatically enables instrumentation as a top-level effect.
A simplified macro expansion of the new fdef is shown below.
(s/fdef example-function :args (s/cat :x int))
(st/instrument `example-function)I never really used the library until a couple of months ago.
While working on a new, isolated feature in a real, production app, I experimented with using the defn-spec fdef macro.
I quickly remembered how incredibly useful it is to have instrumentation enabled while developing.
From catching simple typos in a keyword to telling me a value should be within a specific supported range, instrumentation-enabled development is a big win.
The performance hit is negligible with one big caveat — functions that take a fspec spec’ed function as an argument.
When a function takes a fspec spec’d function as an argument, Spec will use test.check to generatively test the passed function at runtime.
This can have a significant impact on performance.
If you see your program slowing down after working with instrumentation enabled, this could be the reason.
Moving forward I will start using this new fdef more often.
I also quite like the defn-spec macro included in the defn-spec library.
Spec2 may include a built-in way to integrate specs into defn.
I look forward to seeing what that looks like!
For those worried about the effect this will have on your production code, read on.
The above approach enables instrumentation by default.
Since you’re probably worried about performance impact in production, set the Java property clojure.spec.compile-asserts to false when launching your production Java process.
$ java -Dclojure.spec.compile-asserts="false" ...All fdef calls from the defn-spec library will exclude the st/instrument call.
All defn-spec calls will expand to a regular defn.
The Spec guide recommends not using instrumentation in production.
It is not recommended to use instrumentation in production due to the overhead involved with checking args specs.
Against the recommendation of the guide, we left instrumentation on for functions that are not performance-critical and take and return data. Although no in-depth analysis has been done, we have seen negligible impact to performance.
Instrumentation in production gives us two value-adds: better error messages & better test coverage.
Inevitably a call in production will not pass the spec a function defines. Instrumentation gives us consistent and detailed error messages to debug the problem.
Sometimes production data does not perfectly match the specs your functions define. When this happens, one of two things has occurred: the data is actually invalid and handled correctly or your spec is wrong. If it’s the latter, fixing your spec to handle all expectations increases your generative test coverage.
Enable instrumentation by default for increased development productivity. Consider enabling instrumentation in production for places where performance is not critical and throwing an exception on invalid arguments makes sense. If the defn-spec library does not fit your use case, I encourage you to submit a PR, fork it, or write your own version that better fits your use case.
I’m also quite interested in how folks are using instrumentation today. Are you only using it when running your tests? Do you enable it during development some other way? Or do you advocate for no/custom instrumentation?