The folly of using ChatGPT

Reflections on large language models for writing, coding, and knowledge acquisition.

When I first started this Substack, going head first into the world of writing and long-form content, a friend of mine asked me: “Will you be using ChatGPT?”

I didn’t answer right away.

And I’d be lying if i didn’t say the idea was tantalizing.

After all, large language models like ChatGPT and Google’s Bard are just tools, right? Tools to give you ideas, tools to proofread your writing, and tools to speed up your productivity.

Ultimately, to me, they sort of felt like magic spell-books to instantly make me a better writer.

I’ve always struggled with reading and writing. In the 90s, when I was in elementary school, I struggled to the point of needing one on one tutoring. My parents worried I would need to be held back a grade so I could catch up. At the same time, my oldest sister Joanna was a star student, graduating high school at the top of her class. And her favorite thing in the world was reading. She’d read dozens of books a year, knew all about historical fiction, and loved having a good, hearty intellectual conversation about Mr. Darcy.

I never really “got it”.

I didn’t necessarily feel inferior, but at the same time, for years, I was deeply uninterested in picking up a book. I loathed the idea of taking school or my studies more seriously. I mostly just wanted to play video-games and play music, having discovered my talent for jazz and guitar in my early teens. And while I had a propensity for math and science, it all seemed so drab. So I didn’t really try. None of it was actually interesting to me partly because I didn’t feel like I could be good at it. So the cycle went around: not interested, didn’t try, failed to excel which made me less interested, etc. etc.

But, one day when I was 14, Joanna took me to a Barnes & Nobel to pick out a book for my birthday (probably not the birthday gift I would have picked out for myself, but hey, that’s life). She had countless recommendations, but eventually, I settled on a 900+ page fantasy novel by Robert Jordan: The Wheel of Time. I’d never read a fantasy book like this before. Most of what we had read in school up to that point were plays, historically significant works, or non-fiction books. A.k.a, the really boring stuff. The Wheel of Time changed so much for me. It was engrossing. It transported me to a whole new world of magic, adventure, and mystique. And it showed me that books could indeed be really interesting.

That book opened the door for many other books: The Lord of The Rings, Harry Potter, and Eragon. Eventually, my interests expanded to more challenging books like Memoirs of a Geisha and Great Expectations.

These were the first sparks of my intellectual interest and curiosity.

And that curiosity continued to grow, fueling a new desire to learn things and understand them deeply, now knowing that given enough interested and grit, I could grasp the concepts eventually.

By my senior year in high school, that momentum continued and I had completed college level courses in math and science, graduating with honors. To make a long story short, this road of curiosity eventually lead me to where I am today: loving what I do as a software engineer working on huge complex systems, constantly learning and staying curious.

But throughout all that, one thing remained: my insecurity around writing and doing it well. Despite my new found love for reading and excelling at school, fueled by my curiosity, writing remained something that didn’t click for me. To this day, spelling especially remains a challenge (and don’t even get me started on grammar).

Over the last year, working as a software engineer at AWS, this gap in my writing abilities has become abundantly clear. It’s something I need to keep working on. Being a programmer, you sometimes get a free pass. After all, code doesn’t really care if you spell something right (as long as it’s spelled the same way as the other thing you’re referencing). But, at AWS, where there’s a deeply ingrained writing culture, you’re expected to write design docs, promotion docs, status docs, and more.

I’ll never forget some of the feedback I got on my first few big docs at AWS. My writing lacked cohesion and structure. My ideas were not being clearly communicated. And some of those spelling errors slipped through.

This led to a massive personal resolution: my goal in 2023 would be to become a better writer.

For awhile, I did experimented with using ChatGPT to write out a first draft of an idea for this blog. Even if it was just a little something that could get me going and get me started with researching and writing my own words. It felt like a genie in a bottle I could use to do all the hard stuff for me (a.k.a., actually do all the writing so I wouldn’t have to deal with it).

But very early on, it was clear there was a problem: the written content created by large language models lacked any sort of real voice, substance, or character. It all felt drearily hollow. And it only got in the way of the goals I was try to accomplish. I tried spending time to get the prompts just right so a valid first draft could be generated. No luck. I tried taking the dull first drafts and sprucing them up, making them my own with my own voice and my own words. Still, it wasn’t what I wanted.

Ultimately, I decided, if I do indeed want to become a better writer, I needed to start writing, not prompt engineer my way to something that’s half baked, completely hollow, and unsatisfying.

And I believe the same could be said of code generated by ChatGPT: while much of the code may be correct, I’ve personally found it to be unimaginative, brittle, riddled with bugs, difficult to mold in new ways and, in the end, not very useful beyond a certain scale. But, more metaphysically, code written by ChatGPT lacks the intangible art and joy of writing code. There’s something really wonderful about a well thought out and beautifully designed piece of code. And it’s extremely satisfying when you get to see that perfectly crafted software working in the real world. And much like ChatGPT’s writing, generated code lacks that depth, joy, character, and substance that makes for a really incredible piece of software.

This is the folly of ChatGPT: using it in a creative venture only yields the hollow average of all of its parts.

A very, very basic understanding of how large language models work is essential to understand this idea and why output from these models can be so hollow (albeit impressive): these large language models are essentially really complex and sophisticated auto complete engines. They use the inputs of billions and billions of lines of text to come up with the most likely “next” word and/or phrase. It does not have an inherent imagination. It cannot be innately creative. It is, behind it’s god like smoke and mirrors, regurgitating things from its inputs and, at times, making guesses as to the most likely next thing to generate based upon those inputs.

And, when its inputs have wildly different quality and applicability, you’ll end up with a sort of “average” output that is the result of all of its parts but will appear in some senses “correct”. These outputs will impressively fulfill the prompt, but expecting imaginative, emergent, and nuanced output from a broad use AI will likely lead to disappointment.

This year, my goal is not to become an essay prompt engineer who gets large language models to spit out a good enough sounding written piece. My goal this year is to myself become a better writer. And again, to do that I need to write.

I’m reminded of a quote from a book I read about writing from one of the greats, Stephen King:

“Writing isn't about making money, getting famous, getting dates, getting laid, or making friends. In the end, it's about enriching the lives of those who will read your work, and enriching your own life, as well. It's about getting up, getting well, and getting over. Getting happy, okay? Getting happy.”

Stephen King, On Writing: A Memoir of the Craft

Sometimes, especially in tech, we are blinded by the ways our tools maximize productivity, forgetting to look past them to see our real goals. We adopt technologies so quickly that we forget to ask if we should adopt those technologies. I for one want to continue to sit in the joy of a well written piece of code. I want to develop these skills myself through hard work and grit. There are just some things I don’t want my tools to take away from me.

Upon contemplating all this, I eventually told my friend:

“No, I won’t be using ChatGPT for my writing”.