Getting rid of all AI code
Making a note taking app is, surprisingly, a lot of work. When I started work on Disorganized I thought an LLM could help. As of this weekend, pretty much every line of code written by an LLM has been deleted. Here’s why.
Why use an LLM in the first place?
I am not an AI hater, the value proposition is huge. As a solo dev on my own project, being able to speed up development would be very welcomed. My criticism in this blog post is based on results - I used an LLM to help me and have found, with time, that it’s not a good idea.
In Disorganized I have used LLM’s, Claude to be more specific, to:
Write boilerplate code like db requests that I’m too lazy to write myself.
Write advanced code that I’m not good enough to write myself
Discussing more generally how to solve a problem.
Let’s discuss them one at a time.
LLMs for boilerplate
… why is the code so weird? A lot of the boilerplate in Disorganized is written in Dart, which it should have a decent amount of training data for. But it’s doing the weirdest things, casting back and forth between types, doing checks that are completely unnecessary, catching errors just to throw a different (less descriptive) error.
The code is 3-5x longer than it would have been if I wrote it. And of course I checked it, but it would’ve been faster to just write it myself.
LLMs for advanced code generation
There have been a few instances where I’ve wanted some function to work in a specific way, but it’s been too advanced for me to pull off. So I asked the AI. In some cases it seemed like the AI did the right thing, based on my testing. Still, I’ve always ended up getting rid of it. Not because it was clearly wrong, but because most very advanced solutions aren’t worth doing.
I’ve learned that if I really have to fight to implement something, it’s probably not a good idea to implement. Good features are simple features, if something is so advanced that I can’t implement it myself, it’s a strong indicator that it shouldn’t be done.
And LLMs for discussion
Here’s a daily occurrence:
I want to implement something but I’m not sure how
Do a quick Google which doesn’t turn up an immediate answer
Ask an AI.
After asking the AI, one of three things happen:
I get a good response which leads to a good solution
I get a bad response which I detect as a bad response and I go back to google
I get a bad response which I fail to detect as a bad response
Out of these, 3 is the damaging one. At best it leads to a loss of time, at worst it leads to a broken solution making it into the app (soon to be removed, hopefully).
The AI becomes less and less useful the better I get, but all programmers sometimes have moments where we skillfully dodge the obvious solution and do something silly instead. The AI can definitely catch some of those moments.
LLMs for avoiding learning
Maybe the biggest issue of all is that LLMs are a get-out-of-jail free card for learning.
Just handing you the “correct” answer makes you miss deep dives into the documentation, or testing things you found confusing. It’s a short-term productivity increase that slows down your rate of improvement. In the long term, you lose.
Summary
I’m trying really hard to produce an internal compass to help guide my usage of LLMs.
I would love to be able to say “LLMs are never useful, never ever use them” but I just can’t stand behind such a statement. It sometimes IS useful, which just makes it all harder. I’ve never been so conflicted about a piece of technology in my life.
Maybe in five years I’ll have clearer guidelines defined for myself…