You'd expect the King County Bar Association (KCBA), the professional organization that supports local lawyers, to have one of the best systems for telling us who to pick in all those judicial races down at the bottom of the ballot.

But as it turns out, the KCBA—rather alarmingly—conducted a ratings process this year that went so awry that it has been publicly questioned by two appellate court judges, two local superior court judges, and even former state supreme court justice Faith Ireland, who told the Associated Press last week, "Something is seriously broken in the KCBA judicial evaluation process."

KCBA, we have a big problem.

Your ratings are supposed to be the gold standard, and their influence is always magnified by the degree to which opinion leaders and endorsement boards lean on them. Even the Stranger Election Control Board was using your ratings to help us figure out who these judicial candidates are, which ones we should ignore, and which ones we should back in our paper. Or, we were using them until we started scratching our head at this year's weird KCBA results.

At the center of it all, multiple "not qualified" ratings that were given to women and minority candidates for King County Superior Court who have decades of legal experience and strong support from well-respected judges.

The case of Hong Tran appears to be the most egregious.

A candidate for King County Superior Court Position 29, Tran has been a lawyer and activist for social justice for roughly 20 years, but she was rated "not qualified" by the KCBA (a rating the group has now suspended and is reconsidering after protests). "I was stunned," Tran says. She feels the ratings process "is suspect now."

Two appellate court judges, Michael S. Spearman and Ann Schindler, wrote letters to the bar association in June urging a reconsideration of Tran's poor rating, with both describing her as "exceptionally well qualified." Ronald Kessler, the court's chief criminal judge, says "the rating of Ms. Tran as unqualified raises credibility questions as to the ratings process."

Something similar happened to Elizabeth Berns, another respected lawyer with two decades of experience. She was rated "not qualified" and describes her experience with the KCBA as "awful," involving an interview that she says was "unnecessarily and unprofessionally hostile" and a background check that exhibited "several errors," among them "misidentifying me for another superior court candidate." Meanwhile, another candidate was told she didn't provide enough information for a rating, but the KCBA refused to explain what information was missing.

"It would be appropriate for the bar to take a look at its process," says superior court judge Jim Doerty.

One of Judge Doerty's suggestions for fixing the process: Have multiple people conduct the kind of KCBA reference checks that went so wrong in the Berns case, so as to limit the potential for individual bias and error.

Here's another couple of suggestions, courtesy of me and the SECB: If you have insufficient information to rate a candidate, tell her!

Also: Explain your ratings, which right now are typically just a few words long ("qualified," "not qualified," "exceptionally well qualified," etc).

Maybe the KCBA doesn't think it needs to show its work, but this year's mistakes show that, in fact, the group does. Otherwise, we can't—and shouldn't—trust it to judge our judges.

This article has been updated since its original publication.