Knowledge-Aware-Search-Evaluation

Scenario1

The table below show the set of user queries forumulated across the three sites (Google, Hakia and DuckDuckGo in response to the first query scenario, for the following information need:

 

Information Need	Is the document snippet sharing specific information on the use of Buprenorphine in dosages exceeding 4mg daily?
Specific Query	buprenorphine dosage excess 4mg daily
Subset of User Defined Queries
	daily buprenorphine dosage > 4mg
	using bupe more than 4mg
	buprenorphine dosage excess 4mg daily
	buprenorphine dosage more than 4mg daily
	buprenorphine dosage over 4mg daily
	(buprenorphine OR bup) dosage daily ("above 4mg" OR "over 4mg" OR "more than 4mg")

Our System

1. Upon access to the interface. Please select the third data source in the list, if not selected by default.
2. Then carefully browse the dropdownlist and select the ontology nonterminal class ENTITY
   1. Scroll through the dropdownlist and select the entity Buprenorphine
3. Next select the nonterminal PRONOUN
   1. Again carefully scroll through the dropdownlist and select the nonterminal PERSONAL_PRONOUN
4. Then select the nonterminal DOSAGE from the third dropdownlist
   1. Carefully select the greater than sign (>) from the first dropdownlist
   2. Enter the NUMERIC_AMOUNT -- 4 in the second field
   3. Finally, select the UNIT -- mg from the dropdownlist
5. Select the nonterminal INTERVAL
6. '# Scroll through the list and select the nonterminal BY_DAY
   1. Then hold down the Ctrl key and click the nonterminal BY_HOUR
7. Click Search.

Note: If you make a mistake, then reload the page, and repeat these instructions.

 

Scenario 2

	Scenario 1	Scenario 2
Query Type	Is the document snippet sharing specific information on the use of Buprenorphine in dosages exceeding 4mg daily?	Is the document snippet discuss discuss negative sentiments (or experiences) towards Buprenorphine use by an individual?
Specific Query	buprenorphine dosage excess 4mg daily	buprenorphine bad experience
User Defined Query
	daily buprenorphine dosage > 4mg	buprenorphine side effect
	using bupe more than 4mg	side effects for buprenorphine
	buprenorphine dosage excess 4mg daily	side effect for buprenophine | bup | bupe | bupes | suboxone
	buprenorphine dosage more than 4mg daily	buprenorphine bad experience
	buprenorphine dosage over 4mg daily	"buprenorphine" "horrible experience"
	(buprenorphine OR bup) dosage daily ("above 4mg" OR "over 4mg" OR "more than 4mg")	"buprenorphine" "horrible"
		"buprenorphine" ("horrible" OR "side effect")
		"buprenorphine" ("bad experience" OR "side effect")