LLMs are still moving fast. I’d seen some speculation that a plateau is approaching, but so far it has not arrived. I’ve been out of the loop less than three weeks, and while I was gone: Google’s Gemini had a PR disaster, Anthropic launched its Claude 3 model family, and Groq launched its public web app (not to be confused with Grok).
The Groq launch made enough waves that despite being unplugged, I heard about it during our honeymoon and tried it out. The speed is impressive. Using Groq was a good reminder that on any dimension, one product’s “good enough” can be another’s differentiator. Six months ago, a one-, five-, or even ten-second wait for an answer from ChatGPT was absolutely worth it. The technology was so new and the conversations so valuable that the wait was both understandable, and obviously worth it. If you’d told me then that some new model (or in this case, some new architecture+hardware) was faster, I would not have been so excited. What I still want most is the better (more intelligent) model, not the faster one. I would have said speed is nice, but it’s clearly a secondary concern. That’s still how I feel, but my experience with Groq has made this a much closer call than I would have expected or predicted.
Groq’s primary product appears to be the chip and the architecture, not the chatbot, so I don’t expect to see my usage migrate from ChatGPT to Groq. Maybe OpenAI will acquire them. But it did get me reflecting on what would get me to change products, or adopt additional products for specific tasks. Over the past few months, 95% of my LLM usage has been via ChatGPT. I will try a new product when it launches—I tried out Gemini Pro 1.5 for a few days before we left on our trip, before its issues surfaced. I will also try different products when I want different opinions—for a recent research project I was asking the same questions simultaneously to ChatGPT, Gemini, and Claude. But so far, nothing has come close to displacing ChatGPT as my go-to LLM. The first-mover advantage is real, and OpenAI’s pace of innovation has been fast enough to retain their early lead, at least for me. But Groq reminded me that the race is far from won.
sabbatiblog 