Becoming a Certified Barbecue Judge at the Kansas City Barbecue Society

"Or, How I Learned to Stop Worrying and Love the KCBS."

"Would those of you who can tell the difference between a McRib sandwich and real barbecue please stand up and raise your right hand?"

At the behest of the Kansas City Barbecue Society and by invitation from the Jack Daniel Distillery, I pushed back my seat and joined a sizable crowd of barbecue lovers in Lynchburg, Tennessee, to become a newly minted KCBS judge. With hands in the air, we took the oath:

I do solemnly swear to objectively and subjectively evaluate each Barbeque meat that is presented to my eyes, my nose, my hands, and my palate. I accept my duty to be an Official KCBS Certified Judge, so that truth, justice, excellence in Barbeque and the American Way of Life may be strengthened and preserved forever.

The class we had just completed in order to receive our certification was just as rich an experience as The Jack itself. Ron Harwell, certified Master Judge and KCBS Competition Rep, led us through four hours of lecture, anecdotes, hands-on instruction and Southern charm to demystify the art of ranking ribs and penalizing pork.

Far from doctrinaire, Harwell's instruction demonstrated that subjectivity is inevitably a part of scoring. Knowing that marks for taste would be weighted twice as heavily as marks for texture and four times as heavily as marks for appearance, we were tasked with tempering our subjective understanding with an agreed set of technical standards and a good dose of faith—not only in the teams' ambitions and our own judgment but also in the rules of the game.

KCBS Standards and the Definition of 6

Harwell began with a discussion on the world of barbecue outside our classroom. Explaining basic barbecue cooking methods, reviewing the standards of other sanctioning bodies, and giving a rundown of regional barbecue flavors, he made clear that we were in Lynchburg to learn and apply KCBS standards in a prestigious competition, not to pronounce ourselves Jedi masters of barbecue.

Every step of our training was thus rooted in the question, WWKCBSD? (What would KCBS do?) Because marks cannot be changed, and entries must be judged and scored one at a time—going back and forth is not allowed—judges are told that even in the case of a clear disqualification, they must score as if no rule has been broken, then request that a KCBS rep make the call. Every entry is scored by one table of six judges, so consistency of behavior across all tables is essential.

When that fact was carried into the scrutiny of scoring, the paramount question became, "What is the meaning of 6?"

What Is the Meaning of 6?

According to the nine-point judging scale, a score of six is "average." A score of one is only given in the case of disqualification, and a score of two represents barbecue that is so bad, a judge would refuse to swallow it. Harwell wasted no time in calibrating our metrics with pointed questions and time-outs. As samples of chicken, ribs, pork shoulder and brisket hit the tables, he put us on the spot to justify our numbers, then offered litmus tests—for example, how to legalistically deem a piece of meat tender—to help us reach consensus on the de facto boundaries of judgment.

"I'm not telling you to score high," Harwell reminded us repeatedly. Still, it seemed essential that we all perceive a score of six as punishment requiring very concrete justifications, especially given the fact that every team in the upcoming competition had achieved grand champion status in their home states. While two to five would still be around to cover disasters in cookout country, it seemed pre-ordained that next to nothing we would face the next day could fairly be considered "average."

Not only could assigning a six sink a team's competitive edge, it would also increase the chance that my vote would be rendered moot -- in KCBS judging, the lowest of the six marks that comprise an entry's raw score is thrown out, only to be counted as a tiebreaker. I'm not sure this point was as salient to others—in my conversations with a handful of judges from my class, I discovered that scores of five and six appeared more often than the unwritten law would have it.

"This is The Jack," one seasoned judge declared, emphasizing that a high level of competition demands that judges be rigorous in the values we grant each number on the scale. "If you give a six," he warned, "you might as well pack up and go home."

Judgment Day

Having checked myself on numbers, I made it to the judges' tent with all intentions of wrecking myself on four hours of barbecue. The house buzzed with the feeling of a reunion, as barbecue royalty form all parts of the country partook in the tradition of signing each others' aprons before the feast of a thousand beer can chickens.

"Remember," announced a recording of reminders required to be given to judges before every sanctioned competition, "as a KCBS Judge, you are not judging by what you like but to the standards defined by KCBS."

After judging of sauces, my table broke into chatter over which entries we favored. The two veterans and I had awarded our highest rankings to a spot-on vinegar-and-pepper sauce and a perfectly balanced tomato-and-molasses sauce, taking points off of two other entries for being one-dimensionally sweet.

Another first-time judge admitted with a worried look that he was a fan of the sweet sauces, openly wondering if our contrasting marks would lead to an unfair score. With no litmus test on taste buds and no idea what tasting experience he was drawing from beyond a predilection for sweetness, I had no answer. After all, our class, like a tragic high school crush, had made it clear that sweetness in and of itself had no inherent value. Our goal was to draw a fine line between personal preference and personal judgment to score each entry on its own terms.

As judging continued, the doubt that nagged most wasn't one concerning personal preference. Nor was it one concerning political bias, as KCBS uses a double-blind taste test to prevent judges and the table captains who serve them from knowing who cooked any given entry.

My biggest reservation as a judge was the fact that the final score of every entry submitted at this world championship event would be determined by the marks of six judges and no one else.

Is a single trial of five votes (the lowest score is ultimately excluded) a reliable way of doling out glory? While the ethos of consistency is admirably drilled during training, I can't help but worry that without larger tables, additional heats, or some other way of bringing more minds to bear on one brisket, the disproportionate influence of a single judge can throw these carefully qualified standards out of whack.

It's not as if the KCBS hasn't considered this; they wrote the book on barbecue judging and continue to revise their standards each year as new challenges arise. Allowing for more judges to score the same entry could present logistical nightmares strong enough to sink the organization's mission of building a barbecue nation. Under this tent, the balance between prize-fighting and proselytizing demands compromise.

Is "Best" Really as Good as It Gets?

In the end, there are many more questions to consider when it comes to fairness in judging barbecue—including the always valid, "What game was the ref watching?"

I walked away from my judging experience with my eye on a game that was more cooperative than competitive. As the far-reaching constituency and good will of The Jack showed, this event wasn't nearly as much about winners and losers as it was about seeding the soul of serious cooking.

And while it makes little sense to believe that the winner of any judged contest is undeniably the best in the world, this competition made good on the promise that in barbecue, everyone's a winner. You might call this statement a cliché. I call it two pounds of smoked meat and six servings of dessert in my belly.

On the eve of the contest, I stumbled upon the members of IQUE, The Jack's defending grand champions. Looking up from a late night poker game as smokers smoldered behind him, team member Ken Goodman offered perhaps the best way to consider the rankings of food.

"Don't over-think it," he said calmly before throwing his chips on the table. "We've got a good meat, a good program, a good smoker. The rest isn't up to us. It's up to six people."