On Using LLMs

19 Mar, 2025

It's here, there, everywhere. AI Everything.

In the beginning, I classified LLMs as a novelty that can do cute tricks. Maybe it would boost productivity in general office work. But oh boy did they went for it. You have hit peak hype cycle when the majority of LinkedIn influencers can't stop talking about it.

The tooling and integrations have matured. You can give the context of your task easily with your questions -- or orders -- to get, sometimes, useful output. You get a great at summarizer, a decent proxy for web searches, filtering out the SEO crap you get between the information you need¹. Still, I remain doubtful LLMs application fits as many use cases as claimed.

Every time we let AI solve a problem we could’ve solved ourselves, we’re trading long-term understanding for short-term productivity. We’re optimizing for today’s commit at the cost of tomorrow’s ability. ²

Actually Using AI

Dismissing LLMs will not going to keep me employed long, according to fervent AI optimists. So here I go, trying them for myself.

I don't want to have to copy and paste text between different boundaries to get the value out of LLMs. For general purpose usage, open web-ui connected to a local instance of ollama suffice³.

For coding purposes, the tolerance for low quality output is narrower. As such, I have opted to pay for API credits at one of the big vendor. When my small balance dries up, I try a different provider to get a better sense of the space. I use aider as the tool to leverage the big models. It lives, at home, in the terminal.

Coding

I remain skeptical of widespread usage of LLMs for my coding tasks. I generally don't see writing code as the hard part of software engineering.

From online discourse, I see some broad areas cited as most beneficial when using LLMs powered coding assistants:

• generating boilerplate / tests
• "super" tab completion
• prototyping/ideation
• rubber duck debugging

I already solve the first point with snippets and deterministic generators. Writing a new snippet require me to understand the level of abstraction I should aim for as well as the problem space. I trigger them with as few as two or three keystroke as I don't have to externalize the context to another tool. I tried "super" tab style completion in the past. Not a fan. More often than not, the suggestions had so little context that I had to stop writing code to massage the tool into doing what I want. I reflexively stopped looking at them and always skipped to LSP suggestions⁴.

Prototyping comes in two flavors: minimal viable products to find market fit and experimenting with a technical direction or tool before committing to using it. I let founders and the people that pay me do the market fit analysis. I do technical prototyping a lot more. I gain from the pain I experience trying to integrate a new vendor API, often influencing my final decision on using them. Having the code generated for me mask that pain, possibly going past the point of no return in investing into that particular thing.

General Usage

You would think a word generator beneficial for all the bureaucracy a person has to do while trying to do their actual job. LLMs, if used for writing email, slack messages, and documentation introduce another layer of possible miscommunication. Adding more humans in a communication chain almost never result in clearer communication. I can't imagine adding an approximation of a language using human a step in the right direction. Although one can use LLMs for cheating and lying, I prefer to find ways to use them to enhance written my communication.

As a non-native English speaker, I rely on deeply understanding the language and idiom to communicate effectively⁵. In the past, I would try to generalize the question I had about my specific sequence of words to find an answer that might apply to my situation. Now I throw my text into a LLM chat window along with my question to get specific advice. If I still don't get it, I can keep probing until it shines new light into my situation or a reference to a concept I can deep dive on later.

I never generate complete phrases, let alone paragraphs. I want to articulate my points, nuanced with my voice. I ask questions, learn, and modulate my already written work. I chase the knowledge anchoring that comes with the ability to write about it. The is not shortcut to that.

Search Without the Advertising Economics

According to Kagi marketing spiel⁶, I am a heavy search user, up to an order of magnitude more. Google, the search engine, died a couple years back for me as it ramped up on advertising and lost the race to SEO optimization techniques. I go back to it when I need to buy something, because my need aligns with their incentive for that use case.

For general search DuckDuckGo felt sub par but got the job done in until I gave in and started paying for the ability to customize my experience with another provider.

While traditional search engines fills for quick queries, LLMs really shine in the space of gathering information from many sources. I use the free version of Phind, a programming specific AI search service. It filters out the SEO farms, gives you links for further exploration alongside code examples. It provides generally better information extraction the internet than pre-LLMs tools. Again, I use it as a pathway to get information and supplement my understanding.

Closing Note

I dread the day where every tech job will have a bullet point for knowing how to use a specific AI tool as a required skill. A time where it's expected that you do deliver today's value as fast as the computer generates its for you.

I believe you can learn effectively while using AI tools, but you have to work for it. And you know the average tech company won't allow the time for the extra necessary steps to enable it.

For code, I see LLMs as a shortcut to a mean, just like I see fast food as a shortcut to feeding yourself. I will use them when necessary, while keeping in mind the cost of abusing them.

But you can pry my search robot from my dead cold hands.

#long