Every good work of software starts by scratching a developer’s personal itch.

Eric Raymond, The Cathedral and the Bazaar

Like most developers, I’m a creature of habit. I like git for source control, so I use git to track changes in pretty much everything, even if they’re non-source (e.g., configuration files, my resume, and even my hobbyist writing). It’s mostly a matter of sticking with patterns I know and embracing muscle memory; with git not only get version tracking, but I can easily back my work up to tons of places with one command.

$ git remote -v
all github.com:dev-pipeline/dev-pipeline.git (fetch)
all github.com:dev-pipeline/dev-pipeline.git (push)
all gitlab.com:dev-pipeline/dev-pipeline.git (push)
all bitbucket.org:dev-pipeline/dev-pipeline.git (push)

It’s also why I use CMake for all my builds. It’s flexible, it’s powerful, and I know it. I even use CMake for non-source builds (e.g., my writing) since it’s easy to use custom_command and custom_target, then all my CMake muscle memory stays consistent.

The habits come up a lot. When I need tests for a project, I use GTest. GTest is lightweight, it’s easy to use, and it brings some nice features along. The fact that it’s in my systems package manager even meant I didn’t have to deal with making it a sub-module, since sub-modules are the devil.

The other part of being a developer means I like chasing shiny things, so when i saw clang (and especially its sanitizers), I was sold. I started doing extra builds for all my projects (I didn’t usually maintain them, but I got to feel good about it). When I realized clang evens hips with its own C++ standard library, I decided to add a build for that too – after all, what if I had an implicit dependency on libstdc++’s implementation details?

My code compiled the first time. It also failed to link.

In what’s obvious in hindsight, the system-wide GTest was built with gcc and libstdc++, so the pre-compiled GTest had a different idea of things like std::string. In order to build my projects, including tests, with libc++, I had to build GTest with libc++ as well.

The problem is that sub-modules are still awful. I don’t want to tie my projects at the hip to a specific GTest URL, and the problem only gets worse as dependencies are added. The solution: build GTest using clang/libc++, install it in a proper location, and pass the appropriate arguments when configuring those builds.

And it worked. Kinda. Everything built, but it was a pain in the neck to configure a project. If I forgot to pass the right arguments to find GTest (or typo’d them), CMake would happily find the system-wide version and I’d get build failures. I could write a script to automate the process (and I did, in the short term), but these things are limited. If I added another dependency (I like to pretend I’m good at writing general-purpose libraries and tools), then I’d have to update my script for every dependency, and that’s not going to be fun.

I could use Conan, but Conan has way too many steps when libraries are under development. My workflow might involve playing with several libraries at once, especially while I’m trying to finagle APIs.

Which brings us back to the quote at the top of this post. My itch was being able to easily work on multiple libraries at the same time, using them as dependencies for other projects, without needing to jump through tons of hoops. In a perfect world, I should have no changes to my daily process (I’m okay with a bootstrap process where work folders are prepared).

dev-pipeilne is what solved my problem. The first version was designed just to fill my needs: git and CMake (I wasn’t just rambling earlier). Once the protype worked, there were new itches to scratch (e.g., not needing to specify profiles during every invocation). Despite only focusing on my personal needs, dev-pipeline got new features and improvements.

Will dev-pipeilne work for everybody? Of course not, because their itches aren’t the same as mine. Trying to solve everybody’s problems means I have too many variables, too many moving parts, and too many edge cases to get anything done. Even with my limited use cases, solving them gets me something, and with something working there’s a chance to expand it. Eventually, with enough itches scratched, a project can serve a huge number of people.