• 25 Posts
  • 88 Comments
Joined 1 year ago
cake
Cake day: June 13th, 2023

help-circle


  • In 2005 or so, I got a tip about an application called LaunchBar, which would later be copied by Apple to replace the Sherlock search tool, and later by Microsoft in its PowerToys suite. The machine learning LaunchBar used to tailor its responses based on my previous behavior was life-changing. Instead of configuring an application, I just had to use it to change how it behaved.

    This is how language models and AI are going to improve your products. Subtly. Behind the scenes. Slightly improving a thousand different use cases, only a fraction of which your regular usage patterns are going to intersect with.








  • I’ve just spent a few weeks continually enhancing a script in a language I’m not all that familiar with, exclusively using ChatGPT 4. The experience leaves a LOT to be desired.

    The first few prompts are nothing short of amazing. You go from blank page to something that mostly works in a few seconds. Inevitably, though, something needs to change. That’s where things start to go awry.

    You’ll get a few changes in, and things will be going well. Then you’ll ask for another change, and the resulting code will eliminate one of your earlier changes. For example, I asked ChatGPT to write a quick python script that does fuzzy matching. I wanted to feed it a list of filenames from a file and have it find the closest match on my hard drive. I asked for a progress bar, which it added. By the time I was done having it generate code, the progress bar had been removed a couple of times, and changed out for a different progress bar at least three times. (On the bright side, I now know of multiple progress bar solutions in Python!)

    If you continue on long enough, the “memory” of ChatGPT isn’t sufficient to remember everything you’ve been doing. You get to a point where you need to feed it your script very frequently to give it the context it needs to answer a question or implement a change.

    And on top of all that, it doesn’t often implement the best change. In one instance, I wanted it to write a function that would parse a CSV, count up duplicate values in a particular field, and add that value to each row of the CSV. I could tell right away that the first solution was not an efficient way to accomplish the task. I had to question ChatGPT in another prompt about whether it was efficient. (I was soundly impressed that it recognized the problem after I brought it up and gave me something that ended up being quite fast and efficient.)

    Moral of the story: you can’t do this effectively without an understanding of computer science.










  • Regardless of whether or not any of the titles do or do not contain said content, ChatGPT’s varying responses highlight troubling deficiencies of accuracy, analysis, and consistency. A repeat inquiry regarding The Kite Runner, for example, gives contradictory answers. In one response, ChatGPT deems Khaled Hosseini’s novel to contain “little to no explicit sexual content.” Upon a separate follow-up, the LLM affirms the book “does contain a description of a sexual assault.”

    On the one hand, the possibility that ChatGPT will hallucinate that an appropriate book is inappropriate is a big problem. But on the other hand, making high-profile mistakes like this keeps the practice in the news and keeps showing how bad it is to ban books, so maybe it has a silver lining.