In their 2005 article in First Monday, Daniel J. Cohen and Roy Rosenzweig recount the story of a remarkably prescient colleague, Peter Stearns, who “proposed the idea of a history analog to the math calculator, a handheld device that would provide students with names and dates to use on exams–a Cliolator, he called it, a play on the muse of history and the calculator.” [1] Cohen and Rosenzweig took Stearns’s idea and ran with it. They set out to build the Cliolator in the form of a software algorithm called “H-Bot” which served as a history fact finder, scouring the web for information to answer questions about the past. Even with all its limitations and the limits to what was available online in 2004-05, H-Bot was remarkably accurate. It was especially adept at identifying dates and simple definitions. Where it fell short was in more complex questions, including “hows” and “whys”.
Since 2005, the web has grown well beyond the scale of information available to H-Bot, providing a much larger reservoir of data to crawl. And artificial intelligence and machine learning software have brought us much closer to the so-called Cliolator. Google, Amazon, Microsoft, and Apple have all developed quasi-artificial intelligence voice assistants that can parse natural language queries and supply answers drawn from the web. Google Home and Amazon Echo are the most extraordinary examples of these voice assistants, stand-alone devices with the ability to listen to and answer questions. Are these the ultimate versions of the Cliolator?
I decided to put one to the test to see how well this form of artificial intelligence could perform in my introductory Canadian history course. One of the sections of my final exam asks students to identify a key point from the course and explain its historical significance. In answering the questions from this section of the exam, students must demonstrate some simple factual knowledge by identifying each key point and exercise some analytical ability by explaining why it was significant. Could a Google Home pass this section of my exam? I tested it on nine key points:
Wendat Confederacy
Seigneurial System
Expulsion of the Acadians
Battle of the Plains of Abraham
Louis Riel and the Red River Resistance
Numbered Treaties
1942 Conscription Plebiscite
Marsh Report, 1942
Social Gospel
Google Home’s ability to explain the historical significance of these terms was considerably more limited than its ability to simply identify the terms. Identification was dependent upon the quality of information available on the web. Google Home typically drew from Wikipedia, Canadian Encyclopedia, the History Channel, and a handful of other websites to find information about Canadian history. Overall, the quality of that information was pretty good. When trying to answer an analytical question like why something is historically significant, it uses the same methodology. Google Home crawls the web to determine the best answer based on what is already available online. It does not generate its own insights; it is incapable of combining pieces of information to yield new understandings. If someone hadn’t already written something on the web to explain why one of these terms was significant, Google Home was unable to answer the question.
This was the greatest limitation of this device and why voice assistants are forms of quasi-artificial intelligence. Google Home can recognize and parse natural language questions. It can search the web to answer those questions and make decisions about relevance to determine the best (and most likely correct) answer. It cannot generate new ideas based on the information it finds. It can tell me what the seigneurial system was but it cannot tell me why the Canadian government abolished it in 1854. It can tell me who won the Battle of the Plains of Abraham, but it can only offer hints of its significance based on previously published material online. It knows what happened in the past, but it cannot interpret the past.
My little experiment with Google Home reveals some important insights for historians to consider. First, we need more high quality historical research on the web. Voice assistants like Google Home and the Amazon Echo rely entirely upon what they can find on the open web. As a result, they pull from Wikipedia and Canadian Encyclopedia, popular online sources that are open to search engines. If historians want to ensure that broad public audiences have access to the best possible information, they should contribute to these sources and make more of their own research open and available online. Second, artificial intelligence is already influencing historical practice. The Google Home is not going to replace historians, but it is an example of how advanced algorithms and search engines can influence research practice and history education. As digital humanists continue to argue, understanding how these algorithms work is crucial to understanding digital research methodologies. Finally, voice assistants are not likely to be the most important artificial intelligence tools for historical research. At best, having a Google Home or an Amazon Echo on your desk will be as helpful to historical research as a calculator is for mathematical research. AI software that assists in the analysis of historical documents and other data will have far more pervasive influence on historical research in the future. But I’ll save that topic for another post.
What do you think of my highly unscientific experiment? How well did Google Home do on my Canadian history exam? Leave your thoughts in the comments below.
Sean Kheraj is an associate professor of Canadian and environmental history in the Department of History at York University. He blogs at http://seankheraj.com.
[1]Â Cohen, Daniel J. and Roy Rosenzweig. “Web of Lies? Historical Knowledge on the Internet” First Monday 10.12 (2005).
