Category Archives: Accuracy

I’m teaching a fact-checking workshop in Seattle on November 4th

I’ll be teaching a fact-checking workshop in Seattle on Saturday, November 4th for the Northwest Independent Editors Guild. This will be a practical how-to workshop for anyone interested in accuracy–editors, writers, or readers–and registration is open to all. I hope you can join me! Here are the details:

Don’t Assume Anything: Practical Fact-Checking

Fact-checking is an important and useful skill for editors, writers, and
readers. But how can you tell whether a piece of writing or a source is
accurate, fair, and credible?

Join researcher Lisa Gold for a how-to workshop guiding you through the
steps of fact-checking—reading skeptically, asking questions, deciding what
to check, assessing the accuracy of different types of facts, finding and
evaluating sources, working with authors, and making corrections. Lisa will
answer questions and share tips, examples, and resources. This practical,
hands-on workshop will take us beyond the introduction to fact-checking
that Lisa gave at the November 2016 Northwest Independent Editors Guild
meeting [watch it here: https://lisagoldresearch.wordpress.com/2016/11/15/my-talk-about-fact-checking/].

Join us Saturday, November 4, from 10:00 am to 1:00 pm at the Phinney
Neighborhood Center Community Hall (Brick Building, Lower Level) for this workshop. The cost will be $45 for Editors Guild members, $50 for nonmembers.

Register for the workshop online here: https://edsguild.org/dont-assume-anything-practical-fact-checking/

Speaker Bio: Lisa Gold is a freelance researcher, fact-checker, and writer. She has fact-checked many kinds of writing, including magazine articles, reported features, essays, book reviews, historical novels, and nonfiction books. She does creative research for authors of fiction and nonfiction, and she teaches research workshops. You’ll find Lisa online at www.lisagold.com and on Twitter at @bylisagold.

 

 

Advertisements

My talk about fact-checking

The Editors Guild recorded my talk about fact-checking last night and posted it to their YouTube channel, so you can watch it here:

Here’s a link to the PDF handout I prepared and refer to in my talk, with links to selected resources and information about fact-checking:
lisa-gold-fact-checking-eds-guild-handout

As I noted in the meeting description, fact-checking is about ensuring that a piece of writing and its sources are accurate, fair, and credible, and protecting writers and publications from errors, criticism, fraud, and lawsuits. I talked about the skills it requires (an obsession with accuracy, skepticism, critical thinking, the ability to do research and find and evaluate sources, and a willingness to ask questions), who does it, why it’s so rare these days and what types of publications/media generally do or don’t do it. I described the fact-checking process for a major magazine feature, what kinds of things you check and particular trouble spots, discussed some cautionary tales, gave fact-checking tips, and answered questions from the audience.

Let me know if you have any comments or questions. I’d also like to know if there’s any interest in me writing about or teaching classes on fact-checking, research, information literacy, or other topics.

For further reading on these and other subjects, browse my website/blog and my Twitter feed.

The CRAP test for evaluating sources

I frequently blog about evaluating sources— it was the subject of my very first post–so it should come as no surprise that I liked “Crap Detection, A 21st Century Literacy” from the Libraries and Transliteracy blog, which I found through the Librarian in Black.

I wanted to point out two great items featured in the post: Howard Rheingold’s “Crap Detection 101,” and the librarian-created CRAP test for evaluating sources based on “Currency, Reliability, Authority and Purpose/Point of View”:

Currency

  • How recent is the information?
  • How recently has the website been updated?
  • Is it current enough for your topic?

Reliability

  • What kind of information is included in the resource?
  • Is content of the resource primarily opinion? Is it balanced?
  • Does the creator provide references or sources for data or quotations?

Authority

  • Who is the creator or author?
  • What are the credentials?
  • Who is the publisher or sponsor?
  • Are they reputable?
  • What is the publisher’s interest (if any) in this information?
  • Are there advertisements on the website?

Purpose/Point of View

  • Is this fact or opinion?
  • the creator/author trying to sell you something?
  • Is it biased?

Though the questions are familiar (I put together a similar list for my research classes), I love the acronym CRAP, as it is descriptive and memorable. I know I’ll be using it, and I hope it helps you keep in mind some of the criteria to consider when evaluating sources.

Did Tony Blair borrow dialogue from the movie The Queen?

Today’s Telegraph has an article by Tim Walker in which Peter Morgan, Oscar-winning screenwriter of The Queen, says he suspects that Tony Blair incorporated lines from the movie into his autobiography:

In A Journey, Blair claims that the Queen said to him: “You are my 10th prime minister. The first was Winston. That was before you were born.” In Morgan’s script to the 2006 film The Queen, Mirren, in the title role, tells Michael Sheen’s Blair: “You are my 10th prime minister, Mr Blair. My first was Winston Churchill.” Morgan tells me: “I wish I could pretend that I had inside knowledge, but I made up those lines. No minutes are taken of meetings between prime ministers and monarchs and the convention is that no one ever speaks about them, so I didn’t even attempt to find out what had been said.

“There are three possibilities. The first is I guessed absolutely perfectly, which is highly unlikely; the second is Blair decided to endorse what I imagined as the official line; and the third is that he had one gin and tonic too many and confused the scene in the film with what had actually happened, and this I find amusing because he always insisted he had never even seen it.”

As this is impossible to fact-check without the cooperation of Elizabeth II, we may have to give Tony Blair the benefit of the doubt and just marvel at Peter Morgan’s ability to get inside the heads of his characters. (If you haven’t seen it, it’s an excellent film– the screenplay and the performances are exceptional.)

“As an educator it’s my duty to empower you to think…”

In this entertaining and informative video clip, astrophysicist Neil deGrasse Tyson explains and debunks the 2012 “end of the world” hype for a group of students at the 2010 World Science Festival. He also simply and eloquently explains why it’s important to think critically, evaluate information, and be scientifically literate, and in the process shows just what a great teacher he is. This, for me, is the money quote, which I transcribed from the video:

This is what we’re confronted with in this age of rapid dissemination of information, whether or not it is correct. So, I could just tell you it’s all bunk, but then you wouldn’t be empowered to understand why, other than to quote, “Oh, Dr. Tyson said,” and I never want you to quote me citing my authority as a scientist for your knowing something. If that’s what you have to resort to, I have failed as an educator. As an educator it’s my duty to empower you to think, so that you can go forth and think accurate thoughts about how the world is put together, inoculating you against the charlatans out there who will exploit your ignorance…. Science literacy is not just how much science you’ve memorized. No, it’s how is your brain wired for inquiry, what is the next question you ask when someone wants to sell you something….

Watch the whole thing:

Thanks to the Friendly Atheist for pointing out the video in his blog.

Too little, too late: Wikipedia decides accuracy is good and vandalism is bad

An article in today’s New York Times revealed that Wikipedia, after years of embarrassing incidents,  “will begin imposing a layer of editorial review on articles about living people”:

Wikipedia, one of the 10 most popular sites on the Web, was founded about eight years ago as a long-shot experiment to create a free encyclopedia from the contributions of volunteers, all with the power to edit, and presumably improve, the content.

Now, as the English-language version of Wikipedia has just surpassed three million articles, that freewheeling ethos is about to be curbed.

Officials at the Wikimedia Foundation, the nonprofit in San Francisco that governs Wikipedia, say that within weeks, the English-language Wikipedia will begin imposing a layer of editorial review on articles about living people.

The new feature, called “flagged revisions,” will require that an experienced volunteer editor for Wikipedia sign off on any change made by the public before it can go live. Until the change is approved — or in Wikispeak, flagged — it will sit invisibly on Wikipedia’s servers, and visitors will be directed to the earlier version.

The change is part of a growing realization on the part of Wikipedia’s leaders that as the site grows more influential, they must transform its embrace-the-chaos culture into something more mature and dependable.

Roughly 60 million Americans visit Wikipedia every month. It is the first reference point for many Web inquiries — not least because its pages often lead the search results on Google, Yahoo and Bing. Since Michael Jackson died on June 25, for example, the Wikipedia article about him has been viewed more than 30 million times, with 6 million of those in the first 24 hours.

“We are no longer at the point that it is acceptable to throw things at the wall and see what sticks,” said Michael Snow, a lawyer in Seattle who is the chairman of the Wikimedia board. “There was a time probably when the community was more forgiving of things that were inaccurate or fudged in some fashion — whether simply misunderstood or an author had some ax to grind. There is less tolerance for that sort of problem now.”

…Although Wikipedia has prevented anonymous users from creating new articles for several years now, the new flagging system crosses a psychological Rubicon. It will divide Wikipedia’s contributors into two classes — experienced, trusted editors, and everyone else — altering Wikipedia’s implicit notion that everyone has an equal right to edit entries.

That right was never absolute, and the policy changes are an extension of earlier struggles between control and openness.

For example, certain popular or controversial pages, like the ones for the singer Britney Spears and for President Obama, are frequently “protected” or “semi-protected,” limiting who, if anyone, can edit the articles…

The new system comes as some recent studies have found Wikipedia is no longer as attractive to first-time or infrequent contributors as it once was.

Ed H. Chi of the Palo Alto Research Center in California, which specializes in research for commercial endeavors, recently completed a study of the millions of changes made to Wikipedia in a month. He concluded that the site’s growth (whether in new articles, new edits or new contributors) hit a plateau in 2007-8.

For some active Wikipedia editors, this was an expected development — after so many articles, naturally there are fewer topics to uncover, and those new topics are not necessarily of general interest.

But Mr. Chi also found that the changes made by more experienced editors were more likely to stay up on the site, whereas one-time editors had a much higher chance of having their edits reversed. He concluded that there was “growing resistance from the Wikipedia community to new content.”

To other observers, the new flagging system reflects Wikipedia’s necessary acceptance of the responsibility that comes with its vast influence.

“Wikipedia now has the ability to alter the world that it attempts to document,” said Joseph Reagle, an adjunct professor of communications at New York University whose Ph.D. thesis was about the history of Wikipedia.

Under the current system, it is not difficult to insert false information into a Wikipedia entry, at least for a short time. In March, for example, a 22-year-old Irish student planted a false quotation attributed to the French composer Maurice Jarre shortly after Mr. Jarre’s death. It was promptly included in obituaries about Mr. Jarre in several newspapers, including The Guardian and The Independent in Britain. And on Jan. 20, vandals changed the entries for two ailing senators, Edward M. Kennedy and Robert C. Byrd, to report falsely that they had died.

Flagged revisions, advocates say, could offer one more chance to catch such hoaxes and improve the overall accuracy of Wikipedia’s entries.

Foundation officials intend to put the system into effect first with articles about living people because those pieces are ripe for vandalism and because malicious information within them can be devastating to those individuals.

Exactly who will have flagging privileges has not yet been determined, but the editors will number in the thousands, Wikipedia officials say. With German Wikipedia, nearly 7,500 people have the right to approve a change. The English version, which has more than three times as many articles, would presumably need even more editors to ensure that changes do not languish before approval.

“It is a test,” said Jimmy Wales, a founder of Wikipedia. “We will be interested to see all the questions raised. How long will it take for something to be approved? Will it take a couple of minutes, days, weeks?”

Mr. Wales began pushing for the policy after the Kennedy and Byrd hoaxes, but discussions about a review system date back to one of the darkest episodes in Wikipedia’s history, known as the Seigenthaler incident.

In 2005, the prominent author and journalist John Seigenthaler Sr. discovered that Wikipedia’s biographical article connected him to the assassinations of John F. Kennedy and Robert F. Kennedy, a particularly scurrilous thing to report because he was personally close to the Kennedy family.

Since then, Wikipedians have been fanatical about providing sources for facts, with teams of editors adding the label “citation needed” to any sentence without a footnote.

“We have really become part of the infrastructure of how people get information,” Mr. Wales said. “There is a serious responsibility we have.”

I’m not impressed. Though this may prevent some of the more outrageous vandalism, it doesn’t go far enough. Why does the new policy only apply to articles on living people? What about the rest of the articles? What about the bad information that already exists throughout the site? Are the Wikipedia editors going to systematically review existing articles or only new changes to those articles? Who are these “editors” and what are their qualifications?

I’m glad there is finally some acknowledgment among the powers that be at Wikipedia that accuracy is important. But that’s not enough. If accuracy is important, you have to make it a priority and do things on many different levels to try to achieve it. You have to apply your policies to the entire site, not just some articles. You have to bring in people with knowledge, experience, and qualifications to do real editing and fact-checking. (With all of the unemployed editors, fact-checkers, and journalists out there, why not hire a few and let them work their magic.) This new policy is not really about making Wikipedia more accurate, it’s just about trying to stop the embarrassing vandalism stories that hit the news with disturbing regularity.

“Never assume anything!”: Tips for greater accuracy

In light of my recent blog posts about errors and fact-checking, I thought I’d link to some resources to help writers improve their accuracy. Though some of these sources were written for journalists, much of the advice applies equally well to anyone who researches, writes, or edits. It’s important to remember that writers are ultimately responsible for their own work, and they can no longer just assume that their mistakes will be caught and corrected by copy editors or fact-checkers.

This list of “44 Tips for Greater Accuracy” is by Frank E. Fee Jr., the Knight Professor of Editing at the Scripps School of Journalism at Ohio University. There are two versions of Fee’s tips on the web: the first is a concise list, and the second has additional explanatory comments by Fee. You should read the whole thing, but here are some of his more important and universal tips:

  • Always do the math. Don’t rely on another person’s figures…
  • Never disregard a question that has been raised by another reader [or] by that small, sometimes indistinct voice in the back of your head…
  • Never assume anything!
  • We have to see the forest and the trees, so always read (at least) once for content and effect [and] read (at least) once for the mechanical errors (grammar, punctuation, keyboarding).
  • Always use all of the tools available to you (dictionary, stylebook, spell-checker, reference books, etc.). Don’t be too busy or too proud to check a fact.
  • Never trust anything in the [newspaper] clips. How do you know the first story was correct? Do you know for sure corrections caught up with the library clip or archive copy? Has something changed since that story was written?
  • Always get another pair of eyes to look at copy…
  • Always analyze any correction you see — yours or another’s. Ask: How did the error occur? How could it have been avoided? What would I do next time?
  • Always give any sensitive, unusual or tricky material one last look.
  • Always go back and read the full sentence if you’ve changed a word or two in copy. Watch for subject-verb agreement, missing info, duplication, etc.
  • In doubt? Always call the reporter, wire service, or even the source. We’re after the truth, not just a plausible narrative…
  • Always be careful how you ask questions when checking a fact. Leading questions may lead you into trouble. Ask open questions that ensure complete, open answers.
  • Never commit to print anything that you don’t understand. If you don’t know, what are the chances readers will? In pinning down your own understanding, you may: learn something; find a better way to say it; find a more accurate way to say it.
  • Never correct an error until you’re sure you made one. Retrace your steps. Don’t take someone else’s word that copy is wrong; check it out. This will help you discover why the error was made.
  • Always remember: Errors can come in clusters. Finding one may not find them all. There may be others.
  • “Fee’s Theorem”: “The most severe error in any one passage of a story will divert attention from the less severe errors in the same passage. The bigger the error, the more likely it will be the only one caught at that reading. Subsequent readings will tend to continue to eliminate only successive next-most-glaring errors.”

Accuracy First (for reporters)” is a handout that was developed as part of the American Press Institute’s seminar, “Our Readers Are Watching.” Here are a few highlights:

Ensuring accuracy involves several steps:

  • Asking effective questions.
  • Taking accurate notes.
  • Gathering source documents.
  • Questioning information.
  • Verifying information.
  • Fact-checking your story.

Get the names right

Screw up a name and readers who know how that person spells the name will not trust anything else you write. And the source will certainly question your ability or commitment to getting anything else right…

How do you know that?

Judith Miller of the New York Times blamed her inaccurate reporting on weapons of mass destruction on her sources. “If your sources are wrong, you are wrong,” she wrote. Don’t ever buy or use that excuse. The story has your name on it. You are responsible for the information in your story, however you attribute it. Do all you can to evaluate the source and verify the information.

Get to the source. When a character gives you a fact in an interview, get used to asking, “How do you know that?” This gets you to the source of the information. The person you’re talking to may be mistaken or lying or not remember the complete story. Asking “How do you know that?” helps you find the best source for the information. If you’re hearing a story second- or third-hand, trace it back to its origin. If someone is citing statistics to you, get the report that is the source of those statistics. Then you can verify, add context and find more stats.

Evaluate the source. Ask questions of your source (and other sources) that will help you determine how knowledgeable and reliable this person is: Does the source hold a position that would give her official access to this information? Is the source well enough connected to learn this information unofficially? Has this person given you reliable (or unreliable) information before? Has this person given you inaccurate information before? What is the source’s motivation for talking to you? Is the source willing to go on the record and stand behind her story publicly? Who else knows this? Who else knows more about this?

Evaluate the information. Ask questions of your source (and other sources) that will help you determine how knowledgeable and reliable this information is: Does your source know whether this is theory, speculation, rumor or fact? If the information is factual, is it current? Is it complete? What is the context?…

Verify using other sources

Who else knows? Seek other people who are knowledgeable about this situation. They can confirm or refute what you’ve been told. They can fill in gaps. Seek to resolve differences. Again, ask them how they know. Beware the echo chamber: You aren’t receiving confirmation if your second source only knows the information because the first source told her.

Seek documentation. Find official data, records and reports that can confirm, refute or expand upon what you have been told. If you are writing about a court hearing you didn’t attend, get the official transcript. Photographs might help you verify some details…

Go online. Seek verification (or original information) at the official web site of the organization you’re writing about and web sites of agencies that regulate the organization and interest groups that monitor the organization. Be as wary of information you find on the internet as you would of any other source of information. Especially be wary of information from sites that don’t verify their information, such as Wikipedia…

Chip Scanlan’s article on the Poynter Online website, “Getting it Right: A Passion for Accuracy,” contains advice and links to other sources.

Sarah Harrison Smith’s 2004 book The Fact Checker’s Bible: A Guide to Getting It Right has information on reading for accuracy, what to check, researching facts, and assessing the credibility of reference sources.

If you know of other useful sources you’d like to recommend, please do so in the comments to this post.