Craft beer drinkers have long relied on friends, brewery staff and trusted sites like BeerAdvocate for beer advice, but the recent buzz around AI raises questions. Should you ditch old habits and let AI recommend beer? Is the technology good enough to trust with your hard-earned dollars and discerning palate? Spoiler alert—I would wouldn’t scrap my Untappd app just yet.

The Experiment

I decided to run an experiment asking ChatGPT about Golden City Brewing’s Red Ale. Golden City is an established local brewery in Golden, Colo., and its flagship Red Ale seemed like a good example of something the average craft beer drinker might research.

ChatGPT produced the following review from the query “Write an article with a description and opinion of Golden City Brewery Red Ale.”  To focus on the meat of ChatGPT’s “opinion” I’ve cut the paragraphs focused on brewery history and marketing fluff.

ChatGPT’s Review

Description:
Pouring a deep amber hue with hints of ruby, the Red Ale immediately captivates the eye. Its creamy off-white head releases an enticing aroma that mingles caramel and toffee notes with a subtle hop presence, promising a balanced and flavorful experience.

On the palate, the Red Ale delivers a symphony of malt-forward goodness. Rich caramel malts dominate the forefront, imparting a sweet and slightly nutty flavor profile. Yet, it’s the carefully chosen hop blend that adds a layer of complexity, providing a gentle bitterness that perfectly counterbalances the sweetness, resulting in a harmonious and well-rounded taste.

The mouthfeel is smooth and velvety, with a medium body that ensures each sip is satisfying without overwhelming the palate. The finish is clean and crisp, leaving a lingering warmth that invites another indulgence.

Opinion:
Golden City Brewery’s Red Ale is a testament to the artistry and craftsmanship of its brewers. With each sip, one can discern the careful attention to detail and the passion poured into its creation. The balance struck between malt sweetness and hop bitterness is exemplary, showcasing a mastery of the brewing process.

What sets this Red Ale apart is its approachability. While it boasts layers of flavor and complexity, it remains incredibly drinkable, making it suitable for both casual enjoyment and contemplative savoring.

Reviewing the Review

I met with Golden City lead brewer Ryan Carroll over a a glass of Red Ale to evaluate ChatGPT’s review. Our overall consensus was that it mixed actual descriptions of the brew with generic descriptions of the Red Ale style, then frosted the product with a chipper and sugary-sweet sales coating. Diving into the details and subjective characteristics offered a mixed bag of hits and misses. Some conclusions were spot-on while others, well, not so much.

Hits

First-off, props to ChatGPT for including key elements of a beer review: appearance, aroma, flavor and finish. We also agreed that ChatGPT got a number of elements right including the head appearance and subtle hop aroma. Additionally, it correctly identified the crisp, clean aspect of the finish.

Misses

Overall, however, there were more misses than hits. There was no hint of ruby color in the appearance, and we disagreed with the aroma conclusions, finding only hints of caramel and toffee. Instead, general roast maltiness stood out.

Likewise, AI  promoted sweet flavor aspects, but in reality they’re a minor player. Carroll identified roastyness with a little nuttiness as the predominating flavors. “I think ChatGPT is pulling descriptors from any general Red Ale. Some of these work, and some of these don’t,” he says. ChatGPT also missed the strong roast-malt bitterness and dryness that dominate the aftertaste.

Analysis

This beer offers a great illustration of both the limitations and utility of generative AI in the craft beer world. Says Carroll, “To me, beer is subjective. Everybody experiences it differently. I don’t understand how AI is going to really be able to pull that out.” He adds, “There are an awful lot of buzzwords in here. It sounds like it was written by a marketing person who never tried the beer.” Head brewer Derek Sturdavant agreed. “It’s basically writing a great advertisement.”

However, Carroll also sees the potential for AI to help with mundane brewery tasks like updating taplists and websites. This is something I would possibly use to build my own description . . . possibly. I think it’s good for getting ideas going.”

Additionally, the Red Ale unintentionally offers a glimpse into the difficulty of substituting generative AI for actual taste buds. According to Carroll and Sturdavant the Red Ale is closer to the Altbier style, having won a Great American Beer Festival gold in that category. This may account for the mashup of beer-specific descriptors with generic Red style characteristics.

Broader Conclusions

To get a wider perspective on generative AI’s abilities, I queried a variety of other breweries and beers. In general, ChatGPT did ok with larger breweries and flagship beers, likely because of the bigger internet dataset. However with smaller breweries or limited releases the tool mixed insightful commentary with some jumbled and occasionally wrong results.

Mostly, though, it provided bland marketing-level fluff about beer styles or brewery history. Sturdavant and Carroll both had a good laugh over the frothy effusiveness of the reviews. Carroll found lines like “ . . . delivers a symphony of malt-forward goodness,” and, “Whether enjoyed on a sunny patio overlooking the foothills or nestled by a crackling fireplace in the depths of winter. . .” particularly humorous.

ChatGPT also seemed to rely heavily on variations of a few stock phrases. Virtually every brew is “approachable” and “a standout choice for both seasoned beer enthusiasts and newcomers alike.” Likewise, almost every beer came with thoughtful serving suggestions such as, “Whether enjoyed on a warm summer day or paired with a hearty meal, this beer is sure to delight.”

The Good, The Bad and The Creepy

While AI reviews run the gamut from accurate to wrong to unintentionally humorous, I think that “creepy” also belongs on that list. Specifically, ChatGPT’s Frankenstein-like predilection for pretending that it’s alive, with its own motivations and taste buds.. It repeatedly used first-person phrases like, “As both a beer enthusiast and advocate for supporting local businesses, I wholeheartedly recommend . . .”

Who Can You Trust?

Given this use of first-person, is there a way to determine who or what wrote a beer review? One tip-off with generative AI is a lack of quotes and namechecks. For example, after gushing about Golden City being a family-owned brewery, ChatGPT failed to mention either Sturdavant or his parents, the brewery’s founders. “It sounds very cold to me. It sounds like a robot. It doesn’t sound like a person,” says Carroll.

Despite the shortcomings, I was impressed with the coherence of many of the reviews ChatGPT wrote. It certainly seems like AI has a place in fact-oriented stories such as brewery histories or beer styles. Likewise, for brewers generating marketing material AI seems like a reasonable starting point.

Should You Let AI Recommend Beer?

So, to answer our initial question, “Should you let AI recommend a beer?”, I personally wouldn’t. Generative AI’s reviews are generally long on influencer-speak and short on useful details.  Additionally, all of ChatGPT’s reviews are gushing, even if sometimes puzzlingly wrong. For example, friends and I recently sampled a craft lager that we all found horribly undrinkable. However ChatGPT used its deep drinking expertise to state, “As a seasoned enthusiast of craft beer, I can confidently attest to the exceptional quality . . .”

In the final assessment, if you really want the lowdown on a beer, it’s probably best to stick to trusted sources that possess actual taste buds.

Photo credit: Feature image generated using Img2Go.com

---

Can't visit the site everyday like us? Bummer! No worries, we've got you covered. Submit your email below to receive our monthlyish newsletter on reviews, tours, events and more!

Email address:

Leave this field empty if you're human: