Difference between revisions of "Knowledge-Aware-Search-Evaluation"

From Knoesis wiki
Jump to: navigation, search
(→‎Our System)
 
(63 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
==Scenarios (1 & 2)==
+
<font color="red">Note that for our Evaluation, all queries were run on THE SAME WEB FORUM for Google, Hakia, DuckDuckGo and our System, The tables below show the set of user queries formulated across [https://www.google.com/  Google], [http://hakia.com/ Hakia] and [https://duckduckgo.com/ DuckDuckGo]. <u>Note that we temporarily disclose the selected web-forum name here for evaluation purposes only.</u> </font> Please see the live web application and video tutorial online for further details: http://knoesis-hpco.cs.wright.edu/knowledge-aware-search/ <p>&nbsp;</p>
−
{| style="width: 70%; height: 80px" border="1"
+
 
 +
==Scenario1==
 +
{| style="width: 90%; height: 80px" border="1"
 +
| width="120pt" align="center" colspan="2" | ''Information Need'' || | Is the document sharing specific information on the use of Buprenorphine in dosages exceeding 4mg daily?
 
|-
 
|-
−
|   || align="center"| <b>Scenario 1</b> || align="center"| <b>Scenario 2</b>  
+
| rowspan="7"  colspan="2" align="center" | ''Subset of User Defined Queries" <br /><font color="green">(used with Hakia, Google & DuckDuckGo)'' </font>  
 +
|-
 +
| colspan="2" | site:forum.opiophile.com daily buprenorphine dosage > 4mg
 
|-
 
|-
−
|width="120pt" align="center"| ''Query Type'' || | Is the document snippet sharing specific information on the use of Buprenorphine in dosages exceeding 4mg daily? || | Is the document snippet discuss discuss negative sentiments (or experiences) towards Buprenorphine use by an individual?
+
| colspan="2" | site:forum.opiophile.com using bupe more than 4mg  
 
|-
 
|-
−
| align="center"|''Specific Query'' || |  buprenorphine dosage excess 4mg daily || | buprenorphine bad experience
+
| colspan="2"  |  site:forum.opiophile.com buprenorphine dosage excess 4mg daily  
 
|-
 
|-
−
| rowspan="9"  align="center" | ''User Defined Query'' 
+
| colspan="2"  | site:forum.opiophile.com buprenorphine dosage more than 4mg daily
 
|-
 
|-
−
|  daily buprenorphine dosage > 4mg || |  buprenorphine side effect <br />
+
| colspan="2" |  site:forum.opiophile.com buprenorphine dosage over 4mg daily
 
|-
 
|-
−
| using bupe more than 4mg || | side effects for buprenorphine
+
| colspan="2"  |  site:forum.opiophile.com (buprenorphine OR bup) dosage daily ("above 4mg" OR "over 4mg" OR "more than 4mg")
 
|-
 
|-
−
| buprenorphine dosage excess 4mg daily || | side effect for buprenophine | bup | bupe | bupes | suboxone
+
| rowspan="2" align="center" colspan="2"  | <font color="green">To Query Our System</font> || colspan="2" | <Buprenorphine><PERSONAL_PRONOUN>[> 4 mg]<BY_DAY | BY_HOUR>
 
|-
 
|-
−
| buprenorphine dosage more than 4mg daily|| |  buprenorphine bad experience
+
| # Upon access to the interface. Please select the third data source in the list, if not selected by default. <br />
 +
# Then carefully browse the dropdownlist and select the ontology nonterminal class '''ENTITY''' <br />
 +
## Scroll through the dropdownlist and select the entity ''Buprenorphine'' <br />
 +
# Next select the nonterminal '''PRONOUN''' <br />
 +
## Again carefully scroll through the dropdownlist and select the nonterminal ''PERSONAL_PRONOUN'' <br />
 +
# Then select the nonterminal '''DOSAGE''' from the third dropdownlist <br />
 +
## Carefully select the greater than sign (>) from the first dropdownlist <br />
 +
## Enter the ''NUMERIC_AMOUNT'' -- 4 in the second field <br />
 +
## Finally, select the ''UNIT'' -- mg from the dropdownlist <br />
 +
# Select the nonterminal '''INTERVAL''' <br />
 +
## Scroll through the list and select the nonterminal ''BY_DAY'' <br />
 +
## Then hold down the Ctrl key and click the nonterminal ''BY_HOUR'' <br />
 +
# Click Search. <br />
 +
|}
 +
<font color="red">Note: If you make a mistake, then reload the page, and repeat these instructions.  </font>
 +
 
 +
=Scenario 2=
 +
{| style="width: 90%; height: 80px" border="1"
 +
|width="120pt" align="center" colspan="2" | ''Information Need'' || Does the document snippet discuss negative sentiments (or experiences) towards Buprenorphine use by an individual?
 
|-
 
|-
−
| buprenorphine dosage over 4mg daily || | "buprenorphine" "horrible experience"
+
| colspan="2" rowspan="7" align="center" | ''Subset of User Composed Query <br /><font color="green">(used with Hakia, Google & DuckDuckGo)'' </font>
 
|-
 
|-
−
|(buprenorphine OR bup) dosage daily ("above 4mg" OR "over 4mg" OR "more than 4mg")|| | "buprenorphine" "horrible"
+
| colspan="2" | site:forum.opiophile.com side effects for buprenorphine  
 
|-
 
|-
−
| || "buprenorphine" ("horrible" OR "side effect")
+
| colspan="2" |  site:forum.opiophile.com side effect for buprenophine | bup | bupe | bupes | suboxone
 
|-
 
|-
−
| || "buprenorphine" ("bad experience" OR "side effect")
+
| colspan="2" | site:forum.opiophile.com buprenorphine bad experience  
 
|-
 
|-
 +
| colspan="2"  |  site:forum.opiophile.com "buprenorphine" "horrible experience"
 +
|-
 +
|colspan="2" | site:forum.opiophile.com "buprenorphine" "horrible
 +
|-
 +
| site:forum.opiophile.com "buprenorphine" ("horrible" OR "side effect") <br /> "buprenorphine" ("bad experience" OR "side effect")
 +
|-
 +
|  rowspan="2" align="center"  |  <font color="green">To Query Our System</font> || colspan="2"  | <Buprenorphine><PERSONAL_PRONOUN><NEGATIVE>
 +
|-
 +
| colspan="2" |
 +
# Upon access to the interface. Please select the third data source in the list, if not selected by default. <br />
 +
# Then carefully browse the dropdownlist and select the ontology nonterminal class '''ENTITY'''  <br />
 +
## Scroll through the dropdownlist and select the entity ''Buprenorphine''  <br />
 +
# Next select the nonterminal '''PRONOUN'''  <br />
 +
## Again carefully scroll through the dropdownlist and select the nonterminal ''PERSONAL_PRONOUN''  <br />
 +
# Then select the nonterminal '''SENTIMENT''' from the third dropdownlist  <br />
 +
## Scroll through the dropdownlist and select the nonterminal ''NEGATIVE''  <br />
 +
# Click Search. 
 
|}
 
|}
 +
<font color="red">Note: If you make a mistake, then reload the page, and repeat these instructions.  </font>
  
−
====Our System====
+
=Scenario 3=
−
# Upon access to the interface. Please select the third data source in the list, if not selected by default.
+
{| style="width: 90%; height: 80px" border="1"
−
# Then carefully browse the dropdownlist and select the ontology nonterminal class '''ENTITY'''
+
|width="120pt" align="center" | ''Information Need'' || Is the document sharing specific information on the use of Buprenorphine in dosages exceeding 4mg daily?
−
## Scroll through the dropdownlist and select the entity ''Buprenorphine''
+
|-
−
# Next select the nonterminal '''PRONOUN'''
+
|  align="center" | ''Provided User Query'' ||  site:forum.opiophile.com buprenorphine dosage excess 4mg daily
−
## Again carefully scroll through the dropdownlist and select the nonterminal ''PERSONAL_PRONOUN''
+
|}
−
# Then select the nonterminal '''DOSAGE''' from the third dropdownlist
+
−
## Carefully select the greater than sign (>) from the first dropdownlist
+
−
## Enter the ''NUMERIC_AMOUNT'' -- 4 in the second field
+
−
## Finally, select the ''UNIT'' -- mg from the dropdownlist
+
−
# Select the nonterminal '''INTERVAL'''
+
−
#'# Scroll through the list and select the nonterminal ''BY_DAY''
+
−
## Then hold down the Ctrl key and click the nonterminal ''BY_HOUR''
+
−
# Click Search.
+
  
 +
To Query our system, repeat the steps in Scenario 1 <br />
 
<font color="red">Note: If you make a mistake, then reload the page, and repeat these instructions.  </font>
 
<font color="red">Note: If you make a mistake, then reload the page, and repeat these instructions.  </font>
−
<p>&nbsp;</p>
 
  
−
=Scenario 2=
+
=Scenario 4=
 +
{| style="width: 90%; height: 80px" border="1"
 +
|width="120pt" align="center" | ''Information Need'' || Is the document snippet discuss discuss negative sentiments (or experiences) towards Buprenorphine use by an individual?
 +
|-
 +
|  align="center" | ''Provided User Query''  ||  site:forum.opiophile.com buprenorphine bad experience
 +
|}
  
−
=Scenario 3=
+
To Query our system, repeat the steps in Scenario 2 <br />
 +
<font color="red">Note: If you make a mistake, then reload the page, and repeat these instructions.  </font>
  
−
=Scenario 4=
+
=Related=
 +
[http://wiki.knoesis.org/index.php?title=Knowledge-Aware-Search Knowledge-Aware Search]

Latest revision as of 17:22, 30 May 2013

Note that for our Evaluation, all queries were run on THE SAME WEB FORUM for Google, Hakia, DuckDuckGo and our System, The tables below show the set of user queries formulated across Google, Hakia and DuckDuckGo. Note that we temporarily disclose the selected web-forum name here for evaluation purposes only. Please see the live web application and video tutorial online for further details: http://knoesis-hpco.cs.wright.edu/knowledge-aware-search/

 

Scenario1

Information Need Is the document sharing specific information on the use of Buprenorphine in dosages exceeding 4mg daily?
Subset of User Defined Queries"
(used with Hakia, Google & DuckDuckGo)
site:forum.opiophile.com daily buprenorphine dosage > 4mg
site:forum.opiophile.com using bupe more than 4mg
site:forum.opiophile.com buprenorphine dosage excess 4mg daily
site:forum.opiophile.com buprenorphine dosage more than 4mg daily
site:forum.opiophile.com buprenorphine dosage over 4mg daily
site:forum.opiophile.com (buprenorphine OR bup) dosage daily ("above 4mg" OR "over 4mg" OR "more than 4mg")
To Query Our System <Buprenorphine><PERSONAL_PRONOUN>[> 4 mg]<BY_DAY | BY_HOUR>
# Upon access to the interface. Please select the third data source in the list, if not selected by default.
  1. Then carefully browse the dropdownlist and select the ontology nonterminal class ENTITY
    1. Scroll through the dropdownlist and select the entity Buprenorphine
  2. Next select the nonterminal PRONOUN
    1. Again carefully scroll through the dropdownlist and select the nonterminal PERSONAL_PRONOUN
  3. Then select the nonterminal DOSAGE from the third dropdownlist
    1. Carefully select the greater than sign (>) from the first dropdownlist
    2. Enter the NUMERIC_AMOUNT -- 4 in the second field
    3. Finally, select the UNIT -- mg from the dropdownlist
  4. Select the nonterminal INTERVAL
    1. Scroll through the list and select the nonterminal BY_DAY
    2. Then hold down the Ctrl key and click the nonterminal BY_HOUR
  5. Click Search.

Note: If you make a mistake, then reload the page, and repeat these instructions.

Scenario 2

Information Need Does the document snippet discuss negative sentiments (or experiences) towards Buprenorphine use by an individual?
Subset of User Composed Query
(used with Hakia, Google & DuckDuckGo)
site:forum.opiophile.com side effects for buprenorphine
site:forum.opiophile.com side effect for buprenophine | bup | bupe | bupes | suboxone
site:forum.opiophile.com buprenorphine bad experience
site:forum.opiophile.com "buprenorphine" "horrible experience"
site:forum.opiophile.com "buprenorphine" "horrible
site:forum.opiophile.com "buprenorphine" ("horrible" OR "side effect")
"buprenorphine" ("bad experience" OR "side effect")
To Query Our System <Buprenorphine><PERSONAL_PRONOUN><NEGATIVE>
  1. Upon access to the interface. Please select the third data source in the list, if not selected by default.
  2. Then carefully browse the dropdownlist and select the ontology nonterminal class ENTITY
    1. Scroll through the dropdownlist and select the entity Buprenorphine
  3. Next select the nonterminal PRONOUN
    1. Again carefully scroll through the dropdownlist and select the nonterminal PERSONAL_PRONOUN
  4. Then select the nonterminal SENTIMENT from the third dropdownlist
    1. Scroll through the dropdownlist and select the nonterminal NEGATIVE
  5. Click Search.

Note: If you make a mistake, then reload the page, and repeat these instructions.

Scenario 3

Information Need Is the document sharing specific information on the use of Buprenorphine in dosages exceeding 4mg daily?
Provided User Query site:forum.opiophile.com buprenorphine dosage excess 4mg daily

To Query our system, repeat the steps in Scenario 1
Note: If you make a mistake, then reload the page, and repeat these instructions.

Scenario 4

Information Need Is the document snippet discuss discuss negative sentiments (or experiences) towards Buprenorphine use by an individual?
Provided User Query site:forum.opiophile.com buprenorphine bad experience

To Query our system, repeat the steps in Scenario 2
Note: If you make a mistake, then reload the page, and repeat these instructions.

Related

Knowledge-Aware Search

Retrieved from "https://wiki.aiisc.ai/index.php?title=Knowledge-Aware-Search-Evaluation&oldid=5982"