January 2021 Monthly Update

By Alyssa Parado

Here’s the latest updates from the projects in January. This was the first month for ClojisR.

Clj-kondo/babashka/sci

January 1-15

Here is an overview of the work I did per project.

Babashka

Sci

Clj-kondo

Misc

January 16-31

Babashka

Sci

Clj-kondo

O’Doyle Rules

January 1-15

In the past few weeks I tried finding the low-hanging fruit to improve performance. I did this with the help of the excellent clj-async-profiler. I also tried a few ideas that ultimately didn’t pan out:

  1. Running :when conditions earlier in the network. In theory, this could reduce unnecessary work by discarding invalid facts sooner. In practice, it didn’t affect the benchmarks at all, and the code was really convoluted. So much for that.
  2. Storing matches as records instead of hash maps. In theory, this could boost performance because records have faster field access than hash maps. In practice, the benchmarks actually got a bit slower! I still haven’t figured out exactly why, but it at least indicates that field access isn’t a bottleneck.

With the micro-optimizations out of the way, the next few weeks will be all about improving perf with the RETE algorithm itself. GOTTA GO FAST.

January 16-31

I shed a lot of blood, sweat, and parenthesis to implement a common optimization called “node sharing”. Rules that pulled in similar data now could share nodes internally, avoiding duplicated effort. Then, just as I was about to cut a release, I had an epiphany: “derived facts” give us the same benefits as node sharing!

I wrote a section in the README about how to use derived facts in this way. When I applied this technique to the “dungeon” benchmark, it went from ~1700 ms to ~1100 ms – an even better improvement than from node sharing. The dungeon crawler game improved by 5 FPS with the same technique.

I ended up deleting a few hundred lines of fresh code – no need to complicate the codebase if we can just leverage an existing feature instead. I’m now going to focus on debugging and inspecting sessions. Can we do better than the almighty println? Probably not, but no harm in trying…

Calva

January 1-15

I’ve done some work in both Calva and Clojure-lsp.

Calva Work

Calva now uses clj-kondo via clojure-lsp, and no longer bundles the clj-kondo extension. This reduces Calva’s indirect memory footprint as opposed to the previous setup of using clojure-lsp and the clj-kondo extension, since the clj-kondo extension runs its own LSP server, which is no longer necessary.

My co-maintainer Peter also did quite a bit of work recently. Below is a list of Calva changes in the last couple weeks.

[2.0.151] - 2021-01-15

[2.0.150] - 2021-01-13

[2.0.149] - 2021-01-12

[2.0.148] - 2021-01-07

[2.0.147] - 2021-01-07

[2.0.146] - 2021-01-04

[2.0.145] - 2021-01-03

[2.0.144] - 2021-01-01

Clojure-lsp Work

I added a change to Clojure-lsp to prevent the process from being orphaned. If VS Code was killed in a certain time window while Clojure-lsp was initializing, the process would be left running. Now Clojure-lsp periodically checks if the parent process is alive and will exit if not. This is a suggested feature of servers by the language server protocol.

I also fixed an issue with the default classpath lookup for Windows.

January 16-31

For decorating functions instrumented for debugging, I’ve replaced the usage of clj-kondo with clojure-lsp, as well as restructured the code to make the feature work a bit better. This allowed us to remove clj-kondo as an injected jack-in dependency.

I’ve also changed the way clojure-lsp initialization progress is shown. Instead of a popup progress indicator, progress is now shown in the bottom status bar, and disappears when initialization is complete.

I’ve also helped with finding and debugging issues and testing fixes in Clojure-lsp after some significant changes were released. Much thanks to Eric Dallo for being very active, responding to and fixing issues that arise.

Calva Changes

[2.0.156] - 2021-01-28

[2.0.155] - 2021-01-27

[2.0.154] - 2021-01-27

[2.0.153] - 2021-01-19

[2.0.152] - 2021-01-19

ClojisR

January 1-15

Project Goals:

Completed tasks:

Upcoming tasks:

January 16-31

Currently, our main focus is to help different Clojure data science libraries to bring R like functionality by filling in the missing gaps.

Completed tasks:

Upcoming tasks: