The Limits of OpenAI

Constructing a custom content brief generator is an exciting endeavor. If done right, you can end up with a Clearscope-style tool that analyzes and summarizes the main topics from top-ranking search results. But testing your tool can also test your patience.

So when it came to choosing an LLM for this tool, I instinctively chose OpenAI. GPT 4 was a known quantity; it had the ecosystem and a simple API.

After some testing, however, I realized OpenAI was coming up short, especially when it came to flexibility, speed, and trial experience. That’s when I moved to Gemini 1.5.

Why I switched: short version

OpenAI's developer API is impressive on paper. It gives you access to powerful LLM models like GPT 4 and GPT 3.5. But using these models at even small scale with their free tier left me coming up short. And I was only focused on testing.

Rate limits kicked in almost immediately. For a tool intended to analyze several URLs one after the other, having to wait or retry because of usage caps really bogged the entire process down. Even after switching from GPT 4 to GPT 3.5, I still ran into constraints.

Before considering their upgrade, I wanted to see what my alternativew were.

A quick look at alternatives

In my search, Anthropic's Claude was tempting, especially when it comes to tasks needing context. But its pricing was confusing and its access was limited.

Grok was fast and open, but not quite ready for production without extra work. I needed a solution that offered managed APIs rather than one that caused additional issues.

Gemini 1.5: Faster and Cheaper

Google's Gemini API was not an obvious first pick. It was hard to discern what product would be best for me: Studio, Pro, Flash, AI Platform. I actually had to perform AI analysis to help me decide.

But once I'd figured out how to integrate Gemini 1.5 Flash into my application, I noticed an immediate improvement in speed.

What really clinched the deal was Google Cloud's $300 free trial. It's generous enough to allow for extensive testing. The trial is linked to a billing account, so users have to be mindful not to exceed the free tier. But once the trial is activated, the user has full access to the Gemini API.

I have being a big fan of corporate behemoths, but Google deserves accolades here.

The Gemini 1.5 flash model, which I'm employing for text summarization tasks, is efficient and accurate. It's also fine tuned to handle repetitive tasks, which is perfect for generating briefs in a Clearscope-style manner.

Not entirely free, but worth the cost

It's true - neither OpenAI nor Gemini really offer a usable free tier for production. But Gemini's paid tiers are reasonably priced, especially if you are able to use the 1.5 flash model instead of the more expensive 1.5 pro version.

Last words

Looking back, starting with OpenAI was a good move. For a content analysis tool that must retrieve and analyze several URLs, Gemini 1.5 Flash came through where OpenAI couldn’t: with speed, reliability, and cost effectiveness.

I'll still use OpenAI for other purposes. In this instance, Gemini came out head. If you’re creating something comparable and feel constrained by OpenAI’s capabilities, give Gemini a shot. Its $300 trial goes a long way.