Course:LIBR557/2020WT2/stopping rules

Stopping Rules

A person may come up with an information query and not go through with searching for the information. However, if the person goes through with the decision to start an information search, they will similarly have to make a decision about when to stop the search. Stopping rules are rules that were created to help quantify when a person will choose to end their information search (Maxwell et al., 2015).

William S. Cooper

Cooper came up with two stopping rules which he covered in a journal article he wrote in 1973 for the Journal of the American Society for Information Science.

Frustration Point Rule

The frustration point rule stipulates that for all users independent from any search systems, a user will stop their search only when they have encountered a certain number of irrelevant search items (Cooper, 1973).

Satisfaction Stopping Rule

The satisfaction stopping rule stipulates that for all users independent from any search systems, a user will stop their search only when they have encountered a certain number of relevant search items (Cooper, 1973).

D. H. Kraft & T. Lee

Kraft and Lee proposed three stopping rules in an article published in Information processing & management in 1979.

For their rules they propose the following statements (Kraft & Lee, 1979):

N = the total number of documents in the retrieved set;

R = the total number of relevant documents in the retrieved set;

I = N -R = the total number of irrelevant documents in the retrieved set;

p = R/N = the proportion of relevant documents in the retrieved set;

4 = IIN = the proportion of irrelevant documents in the retrieved set;

r = the number of relevant documents sought by the user;

i = the number of irrelevant documents the user will tolerate, i.e. the number of irrelevant

documents in the scan beyond which the scan must terminate;

X = the number of relevant documents found in the scan:

Y = the number of irrelevant documents found in the scan; and

2 = X + Y = the number of documents examined in the scan.

Using those statements they created this equation to represents the probability function of the the number of documents examined in the scan: Pr(Z=k) +{(R/r-1)(I/k-r)/ (n/k-1)} {(R-r +1)/(N-k +1)}

Satiation Rule

The satiation rule follows the same principles as the satisfaction stopping rule and proposes that a search will end when the user has found the number of documents that satisfy them (Kraft & Lee, 1979).

Disgust Rule

The disgust rule follows the same principles as the frustration point rule and proposes that a search will end when the user has become too disgusted by irrelevant information to continue (Kraft & Lee, 1979).

Combination Rule

The combination rule combines the two previous rules and proposes that a user will stop either when they are satisfied by the results they found or are too disgusted by the results to continue (Kraft & Lee, 1979).

Precision and Recall

These stop rules can view in the context of precision and recall. If a search system has a high level of precision and recall, the user is likely to stop because of satisfaction, while if the system has a low level of either, the user is likely to stop because of frustration or disgust. Two stratagems that information systems have developed to increase the precision of the results are content-based and collaborative filtering.

Challenges

The biggest challenge facing stopping rules is that there are so many factors to consider when stopping a search. The combination rule is an example of a researcher combining factors for why an individual may choose to stop the search. However, even combining two rules can't cover all eventualities. For one, different levels of experience in a search field will determine how someone will search for items (Maxwell et al., 2015). There are outside circumstances like noise level, or the mood of the person conducting the search may change the amount of time someone is willing to search for. In 2015 an experiment was conducted, which showed that the frustration-based rule created the best approximations of real-world behavior (Maxwell et al., 2015). There will most likely never be a rule with 100% accuracy, but creating stopping rules is essentially trying to come up with the most accurate theory, and so the creators of stopping rules will always be striving for increased accuracy.

Bibliography

Cooper, W. S. (1973). On Selecting A Measure of Retrieval Effectiveness Part II. Implementation of the Philosophy. Journal of the American Society for Information Science (Pre-1986), 24(6), 413–424.

Kraft, D. H., & Lee, T. (1979). Stopping rules and their effect on expected search length. Information Processing & Management, 15(1), 47–58. https://doi.org/10.1016/0306-4573(79)90007-4

Maxwell, D., Azzopardi, L., Järvelin, K., & Keskustalo, H. (2015). Searching and Stopping: An Analysis of Stopping Rules and Strategies. Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, 313–322. https://doi.org/10.1145/2806416.2806476