Deduplication of News Stories
Deduplicate News Search Results in LexisNexis Academic
Sometimes the same news story is repeated, with little or no difference, in multiple newspapers or the news aggregator sources. The Deduplicate feature allows you to remove these duplicates from your result set.
How It Works
When you run an Easy/News search or use one of the special News search forms available on the navigation menu, your results will include the deduplication tool, pictured below. By default, deduplication is set to "off" so that all search results will be displayed. Changing this setting will remove duplicate stories from your result set.
Working with Deduplicated Results
Here's a news results set with Deduplication set to "Off":
Here's a news results set with Deduplication set to "On: High":
Notice that in the first image, item 5 and 6 are duplicate results. However, in the second image, the list has been deduplicated. You can also see that you have the option to view the duplicate articles.
High Similarity vs. Moderate Similarity
You can choose to turn deduplication on to two different degrees: High similarity and Moderate similarity. "High Similarity" means that the system will require a high degree of similarity before it considers documents to be duplicates. "Moderate Similarity" means that the system will tolerate a greater difference between two documents and still consider them duplicates.