Getting Clojure LSP working on a monorepo that's not really a monorepo

The problem:

• I want to use Neovim with Conjure at work for Clojure development.
• I have zero experience and understanding of how LSP servers and clients work (I didn't before this rabbit hole, anyway.)
• We have an atypical project structure at work that I couldn't find many similar examples on the internet of.
• clojure_lsp does not like odd project structures. It always wants to spin up an LSP server at the project root. A typical development workflow has me calling functions from within one project to another, typically working with at least 3 or 4 projects at once. This spins up several LSP servers, demolishing my memory and grinding every text change to a halt.

What I'd been doing up until this point was straight up turning my LSP off in Neovim or ^*shudder* hanging up my hat and going back to VSCode.

I had become intensely familiar with this spinning indicator on my editor. I had come to loathe its existence.

I read a small article the other day about how "a bad workman always blames his tools". I wanted to think I'm not a bad workman, but also found myself often thinking, "is this even worth it? VSCode works-ish. What am I trying to prove?"

At the end of the day, I love the customizability afforded by Neovim. Frankly, I simply think it's cool. I want to get better at it, and I believed I could. This belief took me down a several-week-long on-and-off discovery into trying to understand how LSP's work, and how I could make them work for me.

The solution

For some background, our architecture at work had been under a single monolith for the longest time. We'd pile new functions onto the monolith, and rinse and repeat. It became somewhat of a behemoth, with few at the company understanding how to properly navigate the codebase, and substantially increased new developer onboarding time. As part of a company-wide engineering restructure, one of the decisions we made was to try to split certain parts of the code out piecemeal, resulting in a microservice-based architecture where the folder structure looked like this -

root/
├── monolith/
│   └── project.clj
└── microservices-parent/
    ├── microservice-1/
    │   └── deps.edn
    └── microservice-2/
        └── deps.edn

I started off by trying to understand how Clojure LSP works under the hood. It's extremely performant and works great out of the box for a single simple project. As you expand outwards and add dependencies that are not under project root, it's as simple as adding new source-paths to your clojure_lsp settings file. This works perfectly if your root is the "main project" and you're pulling in other microservices. But we weren't doing that. Our project was more just microservices talking to each other, so there's no proper single "entry point" into the code.

That was the first problem. Searching "clojure_lsp monorepo" on Google, the first thing you'll see on Google is "Doesn't work in monorepos" - this open Github issue,

where someone some time ago had a slightly similar issue as I did. The next one down is "Multi root support?" - another similar issue.

Lovely.

There's other results on how to get proper monorepo support for other (mostly Emacs-based) editors, but I am not going to be switching anytime soon. VSCode (Calva) also abstracts away all of that and gives you a fairly decent out-of-box experience, but again - I am hung up on Neovim.

So okay, no simple solution. Let's backtrack. I wanted to understand why Neovim was spinning up multiple clojure_lsp's in a single session. Reading through the very helpful, but obscure (obscure to me - I'm spoiled coming from Neovim's in-editor docs 😜) Clojure LSP docs, I figured out that the LSP will try to detect a project root, and create a server off of it. The so-called "root marker"s were Clojure's equivalent of package.json - either a project.clj or a deps.edn in my case. That was problematic, since each of my microservices has its own deps.edn, hence the multiple servers.

Okay, I had found out why it was happening. So how do I go about fixing this?

clojure_lsp has the concept of a "root directory" you can set. This allows you to have a single LSP server sitting at the root of your directory, but it will still try to spin up more LSP servers if you don't spoon-feed the classpaths for it to resolve if it doesn't have a proper root marker. Again, the docs came to my rescue - setting the classpaths is actually very simple. All you have to do is update the :project-specs value in your config.edn file, like so -

[{:project-path "src/monorepo/project.clj"
   :classpath-cmd ["lein" "classpath"]}
  {:project-path "src/microservice-1/deps.edn"
   :classpath-cmd ["clojure" "-Spath"]}
  ...elided]

Simple!

Except... it's not.

lein, which is required for managing dependencies in our project.clj monorepo folder, needs to be run within the folder whose classpath you care about, so in my case, you have to run it in monolith. This is a no-no. I need my project root to be root/

So I went googling, and I found this incredibly helpful answer on Stackoverflow. I had learnt a little bit about shell scripting (although that's a story for another time) and understood the basics of creating a custom script, so I wrote into /usr/local/bin/leinthere the following:

#!/bin/sh

if getopts f: option; then
    # user supplied -f
    if [ -d "$OPTARG" ]; then
        # user also supplied name of a real dir, now in $OPTARG
        cd "$OPTARG"
        shift 2 # get rid of -f and the dir name
    else
        # user supplied -f, but not a real dir name
        echo "usage: $0 [-f project-dir] [lein args]"
        exit 1
    fi
fi

# now just run lein in the normal way:
lein "$@"

Now my config.edn file looks like this -

[{:project-path "src/monorepo/project.clj"
   :classpath-cmd ["leinthere" "-f" "src/monorepo" "classpath"]}
  {:project-path "src/microservice-1/deps.edn"
   :classpath-cmd ["clojure" "-Spath"]}
  ...elided]

And my clojure_lsp had the root directory set properly -

      ...
      clojure_lsp = {
        root_dir = '<other-folders>/actual-project-root/',
      },
      ...

... Surely that had to be it right?

Not quite. I had to muck one last thing up for myself.

I hate when software on my computer is out of date. I have a launchd job that updates all my brew and npm packages that runs every month. One of those months, the script, unbeknownst to me, had updated Neovim to v0.11. Wouldn't you know it, Neovim updated their LSP configuration functions such that my neovim config stopped working.

Long story short - I no longer have a launchd job that auto updates everything every month. The actual fix was fairly simple - I just had to switch over to the native vim.lsp functions instead of using nvim-lspconfig.

Now my editor is blazingly fast and only spins up one correct LSP server. Hooray!

TLDR/what did I learn;

• RTFM - no like actually, read it.
• clojure-lsp does not like monorepos that aren't actually monorepos.
• ChatGPT is shit for niche questions and WILL throw up red herrings.
• Not everything needs to be updated to the latest version.
• Leaky abstractions will be the death of me.
• Sometimes you will be the edge case, and you need to make a blog so that you can help someone else avoid VSCode.

Today, I gaze upon my config and weep. I have the optimal setup. I have paredit-ing down to a science. I have my editor on the left and my REPL on the right. I'm blazing through files, arriving with technical help for support tickets within the minute they come up.

I... I have duplicate error messages.

Dang it. Here we go again.

P.S. If you got to this point, thank you for reading. I have my dotfiles up on GitHub if you were interested. I want to get better at technical storytelling, and I assumed this was as good a place to start as any. I also haven't had a lot of time to work on my "developer brag list" - something many career counselors advise you to do. I think it's time to start... but then again I might disappear after this blog for another few years before surfacing my head again.