Entity Summary

Created on Wednesday May 14th 2014

Creating Faceted (divesified) Entity Summaries

Creating entity summaries has been of contemporary interest in the Semantic Web community in the recet past. In our approach called FACES: FACed Entity Summaries, we are interested in generating diversified and user friendly summaries.

Preliminaries

Problem Statement - An entity is usually described using a conceptually different set of facts to improve coverage. We want to select a ‘representative’ subset of this set in a good summary to uniquely identify the entity.

Definitions 1-4 defines basic notions related to entity summaries. They are as stated in [1].

Definition 1 : A data graph is a digraph G = V, A, Lbl_V , Lbl_A , where (i) V is a finite set of nodes, (ii) A is a finite set of directed edges where each a ∈ A has a source node Src(a) ∈ V, a target node Tgt(a) ∈ V, (iii) Llb_V : V → E ∪ L and (iv) Lbl_A : A → P are labeling functions that map nodes to entities or literals and edges to properties.

Defintion 2 : A feature f is a property-value pair where Prop(f ) ∈ P and Val(f ) ∈ E ∪ L denote the property and the value, respectively. An entity e has a feature f in a data graph G = V, A, Lbl_V , Lbl_A if there exists a ∈ A such that Lbl_A (a) = Prop(f ), Lbl_V (Src(a)) = e and Lbl_V (Tgt(a)) = Val(f ).

Definition 3 : Given a data graph G, the feature set of an entity e, denoted by FS(e), is the set of all features of e that can be found in G.

Definition 4 : Given FS(e) and a positive integer k < |FS(e)|, summary of entity e is Summ(e) ⊂ FS(e) such that |Summ(e)| = k.

Faceted entity summaries

An entity is described by a feature set. A feature (f ) is basically characterized by the property (P rop(f )) and value (V al(f )). In fact, a property binds a specific meaning to an entity using a value. We observe in general that different properties represent different aspects of an entity. For example, profession and spouse properties of an entity (of type person) represent two different aspects. The first defines an intangible value and the second defines a human; one talking about the entity’s professional life and the other about its social life. Based on this observation, we can formalize facets for a feature set.

Facets: Feature set F S(e) of an entity e can be partitioned as a collection of facets F (e). The notion of a facet of a feature set can be defined using partitions as follows.

Definition 5 : Set F (e), a collection of facets of e, is a partition of F S(e). That is, the following conditions hold for F (e). (1) ∅ ∈ F (e). (2)⋃_X∈F(e) X = F S(e). X∈F (e) (3) if X,Y ∈ F (e) and X = Y then X ∩ Y = ∅.

Definition 6 : Given a feature set FS(e), a collection of facets F(e), and a positive integer k < |FS(e)|, faceted entity summary of e FSumm(e), is a collection of features such that FSumm(e) ⊂ FS(e), |FSumm(e)| = k, and if k > |F(e)| then ∀X ∈ F(e) ⇒ X∩ FSumm(e) = ∅ else ∀X ∈ F(e) ⇒ |X∩ FSumm(e) | ≤ 1.

Note that if the number of facets is n and the size of the summary is k, at least one feature from each facet is included in the summary if k > n. If k < n, then at most one feature from each facet is included in the summary.

Clustering

The FACES approach generates faceted entity summaries that are both concise and comprehensive. Conciseness is about selecting a small number of facts. Comprehensiveness is about selecting facts to represent all aspects of an entity that improves coverage. Diversity is about selecting facts that are orthogonal to each other so that the selected few facts enrich coverage. Hence, diversity improves comprehensiveness when the number of features to include in a summary is limited. Conciseness may be achieved by following various ranking and filtering techniques. But creating summaries that satisfy both conciseness and comprehensiveness constraints simultaneously is not a trivial task. It needs to recognize facets of an entity that features represent so that the summary can represent as many facets (diverse and comprehensive) as possible without redundancy (leads to conciseness). Number and nature of clusters (corresponding to abstract concepts) in a feature set is not known a priori for an entity and is hard to guess without human intervention or explicit knowledge. Therefore, a supervised clustering algorithm or unsupervised clustering algorithm with prescribed number of clusters to seek cannot be used in this context. To achieve this objective, we have adapted a flexible unsupervised clustering algorithm based on Cobweb [2][3] and have designed a ranking algorithm for feature selection.

Hierarchical conceptual clustering

We use Cobweb [2] algorithm as the hierarchical conceptual clustering algorithm in our problem. " Cobweb is an incremental system for hierarchical clustering. the system carries out a hill-climbing search through a space of hierarchical classification schemes using operator that enable bidirectional travel through this space." Cobweb uses a heauristic measure called category utility to guide search. Category utility is a tradeoff between intra-class similarity and inter-class dissimilarity of objects (attribute-value pairs). Intra-class similarity is the conditional probability of the form P(A_i = V_ij| C_k), where A_i = V_ij is an attribute-value pair and C_k is a class. When this probability is larger, more memebrs from the class share more values. Inter-class similarity is the conditional probability P(C_k| A_i = V_ij). When this probability is larger, fewer objects sharing similar values are in other classes. Therefore, categori utility (CU) is defined as the product of intra-class and inter-class similarities. For a partition {C₁, C₂,...., C_n}, CU is defined as follows,

Finally, they define CU as the gaining function as below.

Cobweb algorithm is explained in Figure 1 and its four operators are explained in Figure 2 (taken from [3]).

Figure 1. Cobweb algorithm

Figure 2. Auxiliary Cobweb operators

Modifying Cobweb for entities

We modified original Cobweb algorithm Cu to suit our problem as follows.

We could not use CLASSIT[3] implementation of Cobweb as we do not have a distribution of terms as we only have a word set. We build a wordset as follows.

1. Tokenize property and value in a feature.

We take property name and values of a feature. Then tokenize property name and get typing information of the value. Then these tokenized words, original terms, and typing terms are added in a set WS.

2. Expand terms

We expand tokenized and typing terms using WordNet. For expansion we retrieve hypernyms. We add these terms into word set WS.

Ranking of features

We rank features in each facet using informativeness of the feature and popularity of the value of the feature. They are as follows.

Evaluation

For this evaluation we evaluate FACES against RELIN and SUMMARUM. We chose DBpedia as the dataset as it was used in RELIN and is a huge dataset containng multi-domain entities. We extracted 50 entities randomly from English DBpedia version 3.9. We asked 15 human judges to create length 5 and 10 entity summaries (ideal summaries) and used them as the gold standard. We also made sure that each entity gets at least 7 ideal summaries. We could not use experiment data of RELIN as authors of RELIN confirmed that the data are not available. We avoid processing properties such as owl:sameAs, rdf:type, db:wordnet_type. db:wikiPageWikiLink, db:wikiPageExternalLink, db:wikiPageUsesTemplate, db:wikiPageRevisionID, db:wikiPageID, dc:subject, and db:Template. In the sample dataset, there are at least 17 ditinct properties and 19 - 88 distinct features per entity.

Experiment 1

Results of our evaluation with the Gold standard are listed in Table 1. It shows that FACES clearly outperforms other systems. RELINM is a modification of RELIN to have the capability to pick filter duplicate properties in the summary. Table 2 shows that our change of search service to RELIN does not affect the final outcome. We tested this using randomly selected entities of size 5.

Table 1. Evaluating the quality of summaries under each setting, % quality improvement (↑) using FACES compared to others for k=5 and k=10, respectively and average time taken per entity.

System	k = 5	FACES % ↑	k = 10	FACES % ↑	time/entity in seconds
FACES	1.4314	NA	4.3350	NA	0.76 sec.
RELIN	0.4981	187 %	2.5188	72 %	10.96 sec.
RELINM	0.6008	138 %	3.0906	40 %	11.08 sec.
SUMMARUM	1.2249	17 %	3.4207	27 %	NA
Ideal summ agreement	1.9168		4.6415

Table 2. Comparison between Google and Sindice search APIs for a small random sample of entities (5 entities).

k = 5		k = 10
Google search API	Sindice seach API	Google search API	Sindice search API
3.5	3.4	0.5333	0.5428

Experiment 2

This experiment measures FACES ability to identify facets. In other words, its ability to identify conceptually similar groups in comparison to ideal summaries. For this purpose we just ealuate property name overlap between computer generated summaries and ideal summaries. The reults are shown in Table 3.

Table 3. Evaluating the quality of correct concepts picked using property overlap under each setting, % quality improvement (↑) using FACES compared to others for k=5 and k=10, respectively.

System	k = 5	FACES %↑	k = 10	FACES %↑
FACES	1.8649	NA	5.6931	NA
RELIN	0.7339	154 %	3.3993	69 %
RELINM	0.8695	114 %	4.1551	37 %
SUMMARUM	1.6484	13 %	4.4919	27 %
Ideal summ agreement	2.3194		5.6228

Experiment 3

In this experiment, we asked 69 users to vote for the best summary that helps them to identiy the entity. In the first usecase, we used FACES and RELIN side by side. Then we used all three systems side by side and results are shown in table 4. Users were not given information of the systems that produced summaries.

Table 4. Evaluating user preferences for entity summaries using 69 user participants.

Experiment	FACES %	RELINM %	SUMMARUM %
Experiment 1	84 %	16 %	NA
Experiment 2	54 %	16 %	30 %

The following graph presents statistics of the blind-folded user evaluation where user preference for randomly selected 10 entities are recorded. SUMMARUM performed better than FACES in only two entities and on average, FACES outperformed other systems.

Figure 3. User preference of system summaries for randomly selected 10 instances

Dataset

Evauation data is available for download

References

[1] Cheng, Gong, Thanh Tran, and Yuzhong Qu. "RELIN: relatedness and informativeness-based centrality for entity summarization." In The Semantic Web–ISWC 2011, pp. 114-129. Springer Berlin Heidelberg, 2011.
[2] Fisher, D.H.: Knowledge acquisition via incremental conceptual clustering. Machine learning 2(2), 139–172 (1987)
[3] Gennari, J.H., Langley, P., Fisher, D.: Models of incremental concept formation. Artificial intelligence 40(1), 11–61 (1989)

to be added in the paper
[4] Nenkova, Ani, and Kathleen McKeown. "A survey of text summarization techniques." In Mining Text Data, pp. 43-76. Springer US, 2012.
[5] Das, Dipanjan, and André FT Martins. "A survey on automatic text summarization." Literature Survey for the Language and Statistics II course at CMU 4 (2007): 192-195.
[6] Li, Xuan, Liang Du, and Yi-Dong Shen. "Update summarization via graph-based sentence ranking." Knowledge and Data Engineering, IEEE Transactions on 25, no. 5 (2013): 1162-1174.
[7] Erkan, Günes, and Dragomir R. Radev. "LexRank: Graph-based lexical centrality as salience in text summarization." J. Artif. Intell. Res.(JAIR) 22, no. 1 (2004): 457-479.