Knowledge-Aware-Search-Evaluation

Note that for our Evaluation, all queries were run on THE SAME WEB FORUM SITE for Google, Hakia, DuckDuckGo and our System

Scenario1

Information Need		Is the document sharing specific information on the use of Buprenorphine in dosages exceeding 4mg daily?
Subset of User Defined Queries" (used with Hakia, Google & DuckDuckGo)
		site:forum.opiophile.com daily buprenorphine dosage > 4mg
		site:forum.opiophile.com using bupe more than 4mg
		site:forum.opiophile.com buprenorphine dosage excess 4mg daily
		site:forum.opiophile.com buprenorphine dosage more than 4mg daily
		site:forum.opiophile.com buprenorphine dosage over 4mg daily
		site:forum.opiophile.com (buprenorphine OR bup) dosage daily ("above 4mg" OR "over 4mg" OR "more than 4mg")
To Query Our System		<Buprenorphine><PERSONAL_PRONOUN>[> 4 mg]<BY_DAY | BY_HOUR>
		# Upon access to the interface. Please select the third data source in the list, if not selected by default. Then carefully browse the dropdownlist and select the ontology nonterminal class ENTITY Scroll through the dropdownlist and select the entity Buprenorphine Next select the nonterminal PRONOUN Again carefully scroll through the dropdownlist and select the nonterminal PERSONAL_PRONOUN Then select the nonterminal DOSAGE from the third dropdownlist Carefully select the greater than sign (>) from the first dropdownlist Enter the NUMERIC_AMOUNT -- 4 in the second field Finally, select the UNIT -- mg from the dropdownlist Select the nonterminal INTERVAL Scroll through the list and select the nonterminal BY_DAY Then hold down the Ctrl key and click the nonterminal BY_HOUR Click Search.

Note: If you make a mistake, then reload the page, and repeat these instructions.

Scenario 2

Information Need		Does the document snippet discuss negative sentiments (or experiences) towards Buprenorphine use by an individual?
Subset of User Composed Query (used with Hakia, Google & DuckDuckGo)
		site:forum.opiophile.com buprenorphine side effect
		site:forum.opiophile.com side effects for buprenorphine
		site:forum.opiophile.com side effect for buprenophine | bup | bupe | bupes | suboxone
		site:forum.opiophile.com buprenorphine bad experience
		site:forum.opiophile.com "buprenorphine" "horrible experience"
		site:forum.opiophile.com "buprenorphine" "horrible
		site:forum.opiophile.com "buprenorphine" ("horrible" OR "side effect") "buprenorphine" ("bad experience" OR "side effect")
To Query Our System	<Buprenorphine><PERSONAL_PRONOUN><NEGATIVE>
	Upon access to the interface. Please select the third data source in the list, if not selected by default. Then carefully browse the dropdownlist and select the ontology nonterminal class ENTITY Scroll through the dropdownlist and select the entity Buprenorphine Next select the nonterminal PRONOUN Again carefully scroll through the dropdownlist and select the nonterminal PERSONAL_PRONOUN Then select the nonterminal SENTIMENT from the third dropdownlist Scroll through the dropdownlist and select the nonterminal NEGATIVE Click Search.

Note: If you make a mistake, then reload the page, and repeat these instructions.

Scenario 3

Information Need	Is the document sharing specific information on the use of Buprenorphine in dosages exceeding 4mg daily?
Provided User Query	site:forum.opiophile.com buprenorphine dosage excess 4mg daily

To Query our system, repeat the steps in Scenario 1
Note: If you make a mistake, then reload the page, and repeat these instructions.

Scenario 4

Information Need	Is the document snippet discuss discuss negative sentiments (or experiences) towards Buprenorphine use by an individual?
Provided User Query	site:forum.opiophile.com buprenorphine bad experience

To Query our system, repeat the steps in Scenario 2
Note: If you make a mistake, then reload the page, and repeat these instructions.