Category Archives: Evaluating sources

My talk about fact-checking

The Editors Guild recorded my talk about fact-checking last night and posted it to their YouTube channel, so you can watch it here:

Here’s a link to the PDF handout I prepared and refer to in my talk, with links to selected resources and information about fact-checking:
lisa-gold-fact-checking-eds-guild-handout

As I noted in the meeting description, fact-checking is about ensuring that a piece of writing and its sources are accurate, fair, and credible, and protecting writers and publications from errors, criticism, fraud, and lawsuits. I talked about the skills it requires (an obsession with accuracy, skepticism, critical thinking, the ability to do research and find and evaluate sources, and a willingness to ask questions), who does it, why it’s so rare these days and what types of publications/media generally do or don’t do it. I described the fact-checking process for a major magazine feature, what kinds of things you check and particular trouble spots, discussed some cautionary tales, gave fact-checking tips, and answered questions from the audience.

Let me know if you have any comments or questions. I’d also like to know if there’s any interest in me writing about or teaching classes on fact-checking, research, information literacy, or other topics.

For further reading on these and other subjects, browse my website/blog and my Twitter feed.

Let’s talk about search

So my last post, “Yet another study shows that ‘digital natives’ suck at searching,” seems to have struck a nerve– it’s received over 5000 hits (thanks to links from BoingBoing and Fark, as well as Twitter and Facebook), and I’ve been reading  the wide range of comments that have sprung up in various places (including my blog, BoingBoing, and the original article at Inside Higher Ed).

I think what many people (especially students) don’t understand is that search is both a tool and a process, requiring different skills, knowledge, and experience. You can learn just enough to get by or really master it with a little curiosity, persistence, time, and practice. There are many ways to do this, and you don’t need a formal class– you can teach yourself (as I did).  There are lots of online resources to help you, including tutorials and how-to guides on university and library websites and specific search engine help pages. (And don’t forget about librarians, a seriously underused resource.) There are links to some resources in my posts and my blogroll and I’ll add more soon.

I do believe it’s important for students to be taught (and regularly practice)– at school, in libraries, and at home– the essentials of digital and information literacy and critical thinking, starting at a young age  and continuing throughout their education. These are important life skills which are being sadly neglected.

Yes, it’s a truth universally acknowledged that most students are lazy and want to get quick and “good enough” results. But the problem is that they don’t know what they don’t know. (As the ERIAL researchers noted, “students were just as unaware of the extent of their own information illiteracy as everyone else.”) They have no idea that there’s a world of information out there that you can’t find through a Google search. Most of it has never been digitized and probably never will be (for lack of funding and copyright concerns, among other reasons). Some has been digitized but is locked in proprietary databases and the “invisible web.” Most books and articles published in the US after 1922 are still under copyright, so even if they’ve been digitized chances are they aren’t free (unless you borrow them from a library). Even if information has been indexed in Google, you may never find it if you don’t know how to properly search for it.

Google could certainly improve the situation, but it is a company of engineers trying to make search as easy and simple as possible for the vast majority of users, giving them a single “magic box” into which they can type anything and get results, even if they’ve spelled the keywords wrong or don’t really know what they are looking for. Some of the “improvements” they’ve made over time have made it frustrating for advanced users like me, such as ignoring the terms I’ve actually typed and substituting what they assume I’m looking for, or filtering my results based on my past search history. And if you want more advanced search options, Google doesn’t make it easy to find or learn about them, and their help articles often aren’t helpful at all. Search is not just an engineering problem to be solved– it is both an art and a science, and there is no one-size-fits-all solution. But no matter how good or flawed a tool like Google search is, anyone can learn how to use it well and get far better results.

A number of people have asked for some advice and tips on search, so here you go.

General advice:

  • When using any search engine, database, or website with search functions, take a few moments to read the instructions or help pages to figure out how to use the site to its full advantage.
  • Search engines are constantly evolving, so you should periodically review the instructions to see if you need to make changes in the way you search.
  • Every search engine is different, so what works for one won’t necessarily work for all, and each may produce different results using the same search terms.
  • If there is an “advanced search” option, you should always use it, as it will give you much more control over your searching and the results.
  • Refine your search. Experiment with different keywords and combinations of keywords. Look for clues to other possible keywords, such as related terms, alternate names, and subject-specific terminology. If you don’t get the results you are looking for, keep trying different things.
  • Remember to look beyond just the first few search results.

Some Google-specific tips:

  • Google’s search tips and help articles can be hard to find, but they do have useful information. Here are direct links:
  • Use Google’s advanced search function, which allows you to limit your search  in many different ways and combinations (all these words, this exact wording or phrase, one or more of these words, don’t show pages that have any of these unwanted words, language, file type, search within a site or domain, etc.). There is no longer a link to it on the main search page– it’s now hidden behind the gear icon (search settings) in the upper right corner. Here’s the direct link:  http://www.google.com/advanced_search?hl=en
  • If you’d rather use Google’s main search box instead of the advanced search, the help articles I linked to above have command shortcuts you can use, such as placing quotation marks around exact phrases. Note that Google no longer uses all Boolean operators. (You don’t need AND as it is the default in all searches. You can still use OR. Don’t use NOT, instead place a minus sign (-) directly before any words or terms you want to exclude.)
  • Google has many specialized search functions for images, news, blogs, scholarly papers, books, patents, etc.  Look in the upper left-hand corner, click “more,” then click “even more” for a full list. Here’s the direct link to the list:  http://www.google.com/intl/en/about/products/index.html
  • All the words you put in the query will be used and the order you put them in matters.
  • Search is case insensitive, punctuation is usually ignored, and common words (the, a) are usually ignored.
  • Google automatically searches for common variations of a keyword.

A final note: Improving your search skills is important, but it’s even more important that you think critically and evaluate your search results and sources.  (See some of my previous posts for more about this.)

Yet another study shows that “digital natives” suck at searching

I’ve blogged before about studies showing that so-called “digital natives” lack basic information literacy skills and have great difficulty doing academic research and finding and evaluating sources.  (My two posts on Project Information Literacy studies are here and here.)

This Inside Higher Ed article reported today on the results of new studies by the ERIAL (Ethnographic Research in Illinois Academic Libraries) Project. Here’s an excerpt, but you should read the whole thing:

“The majority of students — of all levels — exhibited significant difficulties that ranged across nearly every aspect of the search process,” according to researchers there. They tended to overuse Google and misuse scholarly databases. They preferred simple database searches to other methods of discovery, but generally exhibited “a lack of understanding of search logic” that often foiled their attempts to find good sources….

The most alarming finding in the ERIAL studies was perhaps the most predictable: when it comes to finding and evaluating sources in the Internet age, students are downright lousy….

The prevalence of Google in student research is well-documented, but the Illinois researchers found something they did not expect: students were not very good at using Google. They were basically clueless about the logic underlying how the search engine organizes and displays its results. Consequently, the students did not know how to build a search that would return good sources. (For instance, limiting a search to news articles, or querying specific databases such as Google Book Search or Google Scholar.)

Duke and Asher said they were surprised by “the extent to which students appeared to lack even some of the most basic information literacy skills that we assumed they would have mastered in high school.” Even students who were high achievers in high school suffered from these deficiencies…

In other words: Today’s college students might have grown up with the language of the information age, but they do not necessarily know the grammar.

“I think it really exploded this myth of the ‘digital native,’ ” Asher said. “Just because you’ve grown up searching things in Google doesn’t mean you know how to use Google as a good research tool.”

Even when students turned to more scholarly resources, that did not necessarily solve the problem. Many seemed confused about where in the constellation of library databases they should turn to locate sources for their particular research topic: Half wound up using databases a librarian “would most likely never recommend for their topic.”…

Years of conditioning on Google had not endowed the Illinois Wesleyan students with any searching savvy to speak of, but rather had instilled them with a stunted understanding of how to finely tune a search in order to home in on usable sources, concluded the ERIAL researchers.

Regardless of the advanced-search capabilities of the database they were querying, “Students generally treated all search boxes as the equivalent of a Google search box, and searched ‘Google-style,’ using the ‘any word anywhere’ keyword as a default,” they wrote. Out of the 30 students Duke and Asher observed doing research, 27 failed to narrow their search criteria at all when doing so would have turned up more helpful returns.

Unsurprisingly, students using this method got either too many search results or too few. Frequently, students would be so discouraged they would change their research topic to something more amenable to a simple search….

Duke and Asher noted: “Students showed an almost complete lack of interest in seeking assistance from librarians during the search process.” Of all the students they observed — many of whom struggled to find good sources, to the point of despair — not one asked a librarian for help.

In a separate study of students…, other ERIAL researchers deduced several possible reasons for this. The most basic was that students were just as unaware of the extent of their own information illiteracy as everyone else.

How are students supposed to acquire these important digital and information literacy skills if they aren’t being taught in schools, many parents and teachers lack these skills themselves, and the librarians who have the skills are ignored or fired as libraries close in record numbers?

No, Oxford hasn’t abandoned the Oxford comma

As long-time readers of this blog know, I love the Oxford comma (also known as the serial or series comma). I wrote in a post over two years ago:

If only we could convince the Times (and other newspapers) to use the serial comma (also known as the series comma or the Oxford comma). I’m a big fan of the serial comma, and the Chicago Manual of Style now “strongly recommends this widely practiced usage, blessed by Fowler and other authorities…, since it prevents ambiguity.”  Here’s an example from the Times that shows what can happen without the serial comma: “By train, plane and sedan chair, Peter Ustinov retraces a journey made by Mark Twain a century ago. The highlights of his global tour include encounters with Nelson Mandela, an 800-year-old demigod and a dildo collector.” Perhaps the most famous example of why the serial comma should be used is this apocryphal book dedication: “To my parents, Ayn Rand and God.” (For the origins of these two examples, see this 2006 Language Log post and this 2003 Language Hat post.)

This morning a GalleyCat post reported that the use of the Oxford comma is now discouraged by the University of Oxford (in their online Writing and Style Guide). This has, of course, ignited the Twittersphere with strong reactions, pro and con.

However, it turns out that the story isn’t really true.  Galleycat has just added an update to their blog post:

Reader Michael Williams adds this clarification: “That’s the University of Oxford PR department style guide. Oxford University Press is a commercially and editorially autonomous organization.”

So, much ado about nothing. (Yet another reminder not to believe everything you read on the Internet.) But even if it were true, that would certainly not convince me to give up the Oxford comma.

Update, 6/30/11: 

Oxford Dictionaries tweeted this morning:

The Oxford comma is alive and well at Oxford University Press: http://oxford.ly/mnx8XK

This AP article explains:

But have no fear, comma-philes: the Oxford comma lives.

Oxford University Press, birthplace of the Oxford comma, said Thursday that there has been no change in its century-old style, and jumped into the Twittersphere to confirm that it still follows the standard set out in “New Hart’s Rules.”

The only explicit permission to dispense with the Oxford comma — apparently the cause of the alarm — was in a guide for university staff on writing press releases and internal communications. “It’s not new, it’s been online for several years already,” said Maria Coyle in the university press office.

Yet the report caused a Twitterstorm….

The CRAP test for evaluating sources

I frequently blog about evaluating sources— it was the subject of my very first post–so it should come as no surprise that I liked “Crap Detection, A 21st Century Literacy” from the Libraries and Transliteracy blog, which I found through the Librarian in Black.

I wanted to point out two great items featured in the post: Howard Rheingold’s “Crap Detection 101,” and the librarian-created CRAP test for evaluating sources based on “Currency, Reliability, Authority and Purpose/Point of View”:

Currency

  • How recent is the information?
  • How recently has the website been updated?
  • Is it current enough for your topic?

Reliability

  • What kind of information is included in the resource?
  • Is content of the resource primarily opinion? Is it balanced?
  • Does the creator provide references or sources for data or quotations?

Authority

  • Who is the creator or author?
  • What are the credentials?
  • Who is the publisher or sponsor?
  • Are they reputable?
  • What is the publisher’s interest (if any) in this information?
  • Are there advertisements on the website?

Purpose/Point of View

  • Is this fact or opinion?
  • the creator/author trying to sell you something?
  • Is it biased?

Though the questions are familiar (I put together a similar list for my research classes), I love the acronym CRAP, as it is descriptive and memorable. I know I’ll be using it, and I hope it helps you keep in mind some of the criteria to consider when evaluating sources.

Did Tony Blair borrow dialogue from the movie The Queen?

Today’s Telegraph has an article by Tim Walker in which Peter Morgan, Oscar-winning screenwriter of The Queen, says he suspects that Tony Blair incorporated lines from the movie into his autobiography:

In A Journey, Blair claims that the Queen said to him: “You are my 10th prime minister. The first was Winston. That was before you were born.” In Morgan’s script to the 2006 film The Queen, Mirren, in the title role, tells Michael Sheen’s Blair: “You are my 10th prime minister, Mr Blair. My first was Winston Churchill.” Morgan tells me: “I wish I could pretend that I had inside knowledge, but I made up those lines. No minutes are taken of meetings between prime ministers and monarchs and the convention is that no one ever speaks about them, so I didn’t even attempt to find out what had been said.

“There are three possibilities. The first is I guessed absolutely perfectly, which is highly unlikely; the second is Blair decided to endorse what I imagined as the official line; and the third is that he had one gin and tonic too many and confused the scene in the film with what had actually happened, and this I find amusing because he always insisted he had never even seen it.”

As this is impossible to fact-check without the cooperation of Elizabeth II, we may have to give Tony Blair the benefit of the doubt and just marvel at Peter Morgan’s ability to get inside the heads of his characters. (If you haven’t seen it, it’s an excellent film– the screenplay and the performances are exceptional.)

“As an educator it’s my duty to empower you to think…”

In this entertaining and informative video clip, astrophysicist Neil deGrasse Tyson explains and debunks the 2012 “end of the world” hype for a group of students at the 2010 World Science Festival. He also simply and eloquently explains why it’s important to think critically, evaluate information, and be scientifically literate, and in the process shows just what a great teacher he is. This, for me, is the money quote, which I transcribed from the video:

This is what we’re confronted with in this age of rapid dissemination of information, whether or not it is correct. So, I could just tell you it’s all bunk, but then you wouldn’t be empowered to understand why, other than to quote, “Oh, Dr. Tyson said,” and I never want you to quote me citing my authority as a scientist for your knowing something. If that’s what you have to resort to, I have failed as an educator. As an educator it’s my duty to empower you to think, so that you can go forth and think accurate thoughts about how the world is put together, inoculating you against the charlatans out there who will exploit your ignorance…. Science literacy is not just how much science you’ve memorized. No, it’s how is your brain wired for inquiry, what is the next question you ask when someone wants to sell you something….

Watch the whole thing:

Thanks to the Friendly Atheist for pointing out the video in his blog.