Stop Search . . . Start Finding! Use Artificial Intelligence to Gain Insight

By Bruce G. Buchanan, Ph.D., November 19, 2018

Good information matters: Well-informed employees are more productive. Well-informed consumers are better customers. And as employees and consumers ourselves, we need good information to make decisions. Everyone is a consumer of information, and Artificial Intelligence can help. Who can I ask to point me in the right direction, so I waste less time searching?

IDC data show that “the knowledge worker spends about 2.5 hours per day, or roughly 30% of the workday, searching for information. . . . 60% [of company executives] felt that time constraints and lack of understanding of how to find information were preventing their employees from finding the information they needed.” For example,

  • Problems encountered today may already have been solved in the past, if only the right information can be found. This improves efficiency.
  • It may not be obvious, but reliable health and wellness information is an important factor in improving wellness and therefore productivity. govdata show that U.S. workers miss one-half to one-and-a-half work days a year due to illness.

Companies also have a stake in making good information available to the general public beyond the scope of their own employees. Providing good information for consumers’ well-informed decisions gives a company the ability to

  • Increase brand visibility and loyalty
  • Earn the reputation as a trusted source
  • Increase revenues through product confidence.

Consumer education and consumer awareness sites focus on important public service issues such as avoiding fraud, making smart financial decisions, avoiding unsafe products, and staying healthy. Consumer Reports, for example, provides a wealth of information to help readers make informed buying choices for a very wide range of products and services. And for every product and service, search engines can find thousands of articles on the web that are relevant in some way. For the most part, these sites are maintained by teams of people who scour each week’s or each day’s magazines, newspapers, and journals, find relevant articles, sort them into groups that satisfy the requests of thousands of individuals, and send personalized emails. But, as with many repetitive tasks, it is more efficient to use AI to do much of the work.

Information overload—How do I find anything?

The web is a modern resource containing information on every conceivable topic. But the volume of available information is intimidating, some of it is misleading, and not all of it is relevant. We’re all familiar with information overload. The cost is difficult to quantify, but the divergence between “not-enough” and “too-much” information has become only greater with the rise of the web in the 1990s.

Dean Rieck summarizes the problem nicely. “When people are feeling overwhelmed, they react in the only way they can:

  • They skip over or set aside difficult information for another time; or
  • They filter out difficult messages and look for ones that are easier; or
  • They try to deal with the information, but make mistakes that prevent them from responding in the way you want; or worst of all
  • They ignore your message completely.”

AI can help to provide an intelligent assistant for individuals seeking information in order to select gems out of the landslide. Here are some of the considerations that can guide an intelligent system—AI or human editor—in providing relevant information to the consumer community.

Can I trust what I find on the internet? What’s legitimate and what’s coming from fake sites?

Use only credible sources

Long before the web, and continuing to the present, knowledgeable friends and colleagues—known as the “invisible college” since the 17th century—have given all of us pointers to the best information on topics they know better than we do. We often try web searching first, but then turn to trusted colleagues out of frustration to sort through the landslide of pointers and avoid wasting time looking at extraneous or, even worse, misleading articles. Subject matter experts know the journals, periodicals, and blogs that provide reliable information. So do editors and reference librarians—and so can AI.

Eliminate duplication across sources

Many articles on the web and in corporate archives are duplicated. Often one news story from a wire service is repeated verbatim many times or reprinted with only small modifications. Or the same development will be described in new words but with no new information. Authors often deposit multiple versions of a document on a shared site without removing the old versions. All this leads to wasted time on the part of recipients. There are many available apps for eliminating duplicate and similar files, some free, so it suffices to say this step is helpful.

I wish my search engine could understand what I mean, not just match the words. How do I get beyond keyword searching?

Keyword searching, the tried-and-true method of finding information on the web, began as subject-matter indexing in which documents were tagged with category names. A 3rd century BCE librarian developed a broad subject-matter indexing scheme of the roughly half-million items in the classical Library of Alexandria. Melvil Dewey took indexing to a new level in 1876 with publication of the widely used Dewey Decimal System. These systems relied on manual work to tag each document with a static list of words and phrases. Another leap forward came when documents were digitized and searchable by computers and a manually created “back of the book index ” could be dynamically created. The keyword-in-context, or KWIC, system was introduced in 1957 and has been a mainstay of information retrieval ever since. We can do better, however, by tagging articles with more than keywords and static index terms. Colleagues in the invisible college are reliable, in part, because they have read the contents of the books and articles in their subject areas. They have knowledge of both the subject matter and the language in which it is addressed. With advances in natural language processing in recent years, AI can now go beyond static indexing to dynamic conceptual indexing. Reading and extracting information from the text, including figures and tables, sounds simple because people, even children, learn to do just that as they learn to read. However, books, articles, presentations, and notes are stored online in a variety of digital formats with information buried in text. For example, tables stored in spreadsheets are relatively easy to analyze; the same tables presented in PDF documents require considerable structuring before they can be analyzed. AI software can harness computer vision techniques and knowledge of how tables are built and used by humans to accomplish the task. Metadata, such as authors’ names and modification dates, are routinely saved with online documents and are available for successive filtering. AI can combine those data with the results of conceptual tagging, hierarchical indexing, and keyword searching to provide a variety of ways that individuals can find the information they need.

I wish my search engine knew what I was looking for. How can I see only the results relevant to what I need to know right now?

Tailor the results to personal interests

Knowing a person’s information needs allows AI to suggest highly relevant items to read. However, our needs and interests change frequently, often creating a niche interest for which there are no data on which to apply machine learning. It is important that we have the means to easily tell a system what we become interested in. Often what we’re interested in has not been anticipated in any fixed set of categories but is an amalgam of several. For example, a cancer patient (or physician) may be interested in results of new clinical studies about preventing nerve damage in patients undergoing chemotherapy. While no single tag, search term, or filter directly targets a small number of articles, AI can apply a combination of several to find just the relevant ones. Once we identify an interest, we should have the option to receive notification of relevant new articles when they appear in the future, without having to initiate a new search. “Push” technology is not new, but AI-assisted notifications are an important next step in making sure that the consumers of information are delivered just what they need without a lot of effort on their part.

Provide links to original material with a brief summary

The text of the original article retains important detail and nuance that is impossible to capture in short tags used for search and retrieval. From a set of likely titles, then, it is important to provide summaries that tell readers enough to judge relevance for their specific needs without reading the full article. The first few sentences of a document are not always a good summary. AI can extract enough meaning to build a better summary.

Automate the process to avoid manual effort

Here is how i2k Connect can help

Finally, the whole process can be automated by moving each of the steps into an AI-powered intelligent assistant that knows the subject matter and individuals’ personal needs. It enables people to find what they need rather than wasting their valuable time to search with obsolete strategies. If you are interested in learning more about how Artificial Intelligence can enhance your ability to provide good information to your company’s employees and customers, contact us today.


Bruce Buchanan – Co-Founder and Chief Scientist, Bruce is chiefly responsible for the AI science used in the i2k Connect Platform. He is also University Professor of Computer Science Emeritus at the University of Pittsburgh. Before joining the Pitt faculty as a Professor of Computer Science, Medicine, and Philosophy, he was Professor of Computer Science (Research) at Stanford where he worked on the Dendral, Meta-Dendral, Mycin, and Protean systems. He has supervised more than two dozen Ph.D. dissertations in AI and related fields. Bruce holds a Ph.D. in Philosophy from Michigan State University; he is a Fellow of AAAI and the American College of Medical Informatics; an elected member of the National Academy of Medicine; and has served as the secretary-treasurer and president of AAAI.

About i2k Connect – An Artificial Intelligence software and service company transforming business. Our novel Artificial Intelligence technology automatically identifies and tags documents with unique, accurate, and consistent metadata. Our deep data extraction uses computer vision, natural language processing, and statistical analysis to understand the information in documents and deliver it to the people who need it to quickly make decisions. Our AI Platform and domain expertise are focused on vertical industries including oil, gas & utility, financial services, and healthcare.