Analyzing URL Chatter on Twitter - Revision history

Pablo at 16:51, 26 May 2010

2010-05-26T16:51:31Z

Pablo at 18:22, 10 February 2010

2010-02-10T18:22:09Z

Pablo: /* Results */

2010-02-02T17:25:13Z

‎Results

Pablo: /* Results */

2010-02-02T17:24:52Z

‎Results

Pablo at 17:23, 2 February 2010

2010-02-02T17:23:18Z

Pablo: /* Data Analysed */

2010-02-01T17:44:32Z

‎Data Analysed

Pavan: /* Data Analysed */

2010-01-29T00:11:34Z

‎Data Analysed

Pablo: /* Software Archictecture */

2010-01-27T20:40:58Z

‎Software Archictecture

Pablo at 16:54, 27 January 2010

2010-01-27T16:54:11Z

Pablo at 22:15, 21 January 2010

2010-01-21T22:15:21Z

@@ Line 1: / Line 1: @@
-This is a class project by Pavan, supervised by Pablo.
+This is a class project by [[User:Pavan]], supervised by [[User:Pablo]]. It has been shown that social networks play a crucial role in the spread of information <ref name=contagion>Kristina Lerman, Rumi Ghosh. ICWSM'10. [http://arxiv.org/abs/1003.2664 Information Contagion: an Empirical Study of the Spread of News on Digg and Twitter Social Networks]</ref> In this project we explore mentions of Web documents on Twitter to analyze the thematic, spatial and temporal aspects of content recommendation.
 ==Project Description==
@@ Line 25: / Line 25: @@
 *2. Since HTTP connections take a long time, we only resolve short URIs. The first step then is to recognize URLs as long or short. There are a lot of services to shorten the url but no global service to do the converse. We kept a list of known "shorteners" and only resolved those.
-===Software Archictecture===
+===Software Architecture===
 Code is available at svn/classprojects/CitizenSensor/URLExtractorWeb/.
@@ Line 85: / Line 85: @@
 * [[Social Signals]]

← Older revision		Revision as of 18:22, 10 February 2010
Line 81:		Line 81:

	[[Category:Information Extraction]][[Category:Information Exploration]]		[[Category:Information Extraction]][[Category:Information Exploration]]
		+
		+	== Related ==
		+
		+	* [[Social Signals]]

@@ Line 78: / Line 78: @@
 * '''Timeline Perspective''': The main interface of Twitris was able to show data from a fixed time point spread over geography (GoogleMap). We added a timeline perspective to Twitris so that data can be shown spread over time as well. [http://knoesis1.wright.edu/twitris-timeline/ timeline demo]
 * '''Twitris on the LOD''': We extracted mentions to entities that are already on the LOD cloud and added links from the tweets to those entities. We therefore made Twitris part of the LOD cloud. [http://knoesis1.wright.edu/twitris-timeline/lod lod demo]
-* '''Flexible query interface''': We installed Cuebee to allow users to query the information extracted from tweets with total freedom to explore the geographical, time or thematic dimensions. [http://knoesis1.wright.edu/twitris-timeline/ flexible querying demo]
+* '''Flexible query interface''': We installed Cuebee to allow users to query the information extracted from tweets with total freedom to explore the geographical, time or thematic dimensions. [http://knoesis1.wright.edu/twitris-timeline/query.html flexible querying demo]
 [[Category:Information Extraction]][[Category:Information Exploration]]

@@ Line 1: / Line 1: @@
 This is a class project by Pavan, supervised by Pablo.
@@ Line 77: / Line 76: @@
 == Results ==
-* '''Timeline Perspective''': The main interface of Twitris was able to show data from a fixed time point spread over geography (GoogleMap). We added a timeline perspective to Twitris so that data can be shown spread over time as well.
+* '''Timeline Perspective''': The main interface of Twitris was able to show data from a fixed time point spread over geography (GoogleMap). We added a timeline perspective to Twitris so that data can be shown spread over time as well. [http://knoesis1.wright.edu/twitris-timeline/ timeline demo]
-* '''Twitris on the LOD''': We extracted mentions to entities that are already on the LOD cloud and added links from the tweets to those entities. We therefore made Twitris part of the LOD cloud.
+* '''Twitris on the LOD''': We extracted mentions to entities that are already on the LOD cloud and added links from the tweets to those entities. We therefore made Twitris part of the LOD cloud. [http://knoesis1.wright.edu/twitris-timeline/lod lod demo]
-* '''Flexible query interface''': We installed Cuebee to allow users to query the information extracted from tweets with total freedom to explore the geographical, time or thematic dimensions.
+* '''Flexible query interface''': We installed Cuebee to allow users to query the information extracted from tweets with total freedom to explore the geographical, time or thematic dimensions. [http://knoesis1.wright.edu/twitris-timeline/ flexible querying demo]
 [[Category:Information Extraction]][[Category:Information Exploration]]

@@ Line 1: / Line 1: @@
 ==Project Description==
-This project helps analyzing the URLs available in the tweets with the theme. The data(Tweets) crawled for Twitris project is being used (see [[#Data_Analysed]]).
+This project helps analyzing the URLs available in the tweets with the theme. The data (tweets) crawled for [[Twitris]] project are being used (see [[#Data_Analysed]]).
 ===Introduction(Objectives and Motivation)===

@@ Line 41: / Line 41: @@
 ==Data Analysed==
-===URls===
 *Total Tweets used : 3,188,262
 *URLs Extracted : 1,477,356

@@ Line 46: / Line 46: @@
 *Unique Short URls : 827,750
 *Unique Long(original) Urls : 649,165
 ===Entities===
@@ Line 51: / Line 54: @@
 *Entities Extracted from 3,188,262 tweets : 25,393,165
 *Unique Entities Extracted : 244,383 (The Entities should be filtered since during Extraction we do not filter URLs which contains entities such as "htt","bit","ly" which has occurred most number of times.
 == Results ==

@@ Line 29: / Line 29: @@
 The front end is done using HTML, CSS, XML and Java Script.  The [[http://simile.mit.edu/wiki/Exhibit/Timeline_View Exhibit Timeline]] javascript is used to show our analysis. The javascript takes an XML as the input and provides a graphical interactive object on the Web page.
-The backend is composed by a data layer, a processing layer and an API layer.
+The backend is composed by a data layer, a processing layer and an API layer. See also: [http://knoesis.wright.edu/internal/wiki/index.php?title=Lotter Lotter]
 * '''Data layer''': connects and queries the DB.
 * '''Processing layer''': Contains classes that extract information from the raw data so that we can then store the processed information in the DB.
@@ Line 37: / Line 37: @@
 The location attribute of the response gives the original url which is then stored in the DB.
-* '''API layer''': servlets which forms an programmatic interface to the frontend. It takes the requests from the front end, calls the Db connector for the data and responses by xml. Four servlet classes are used for different purposes.''' What are the servlets????'''
+* '''API layer''': servlets which forms an programmatic interface to the frontend. It takes the requests from the front end, calls the Db connector for the data and responses by xml. Four servlet classes are used for different purposes.''' Describe the servlets here.
 ** Tagger: this service takes in the text for a tweet and calls the appropriate sequence of classes from the processing layer to extract URLs and entities. Returns an RDFa annotated HTML snippet.

← Older revision		Revision as of 16:54, 27 January 2010
Line 57:		Line 57:
	* '''Twitris on the LOD''': We extracted mentions to entities that are already on the LOD cloud and added links from the tweets to those entities. We therefore made Twitris part of the LOD cloud.		* '''Twitris on the LOD''': We extracted mentions to entities that are already on the LOD cloud and added links from the tweets to those entities. We therefore made Twitris part of the LOD cloud.
	* '''Flexible query interface''': We installed Cuebee to allow users to query the information extracted from tweets with total freedom to explore the geographical, time or thematic dimensions.		* '''Flexible query interface''': We installed Cuebee to allow users to query the information extracted from tweets with total freedom to explore the geographical, time or thematic dimensions.
		+
		+	[[Category:Information Extraction]][[Category:Information Exploration]]

← Older revision		Revision as of 22:15, 21 January 2010
Line 57:		Line 57:
	* '''Twitris on the LOD''': We extracted mentions to entities that are already on the LOD cloud and added links from the tweets to those entities. We therefore made Twitris part of the LOD cloud.		* '''Twitris on the LOD''': We extracted mentions to entities that are already on the LOD cloud and added links from the tweets to those entities. We therefore made Twitris part of the LOD cloud.
	* '''Flexible query interface''': We installed Cuebee to allow users to query the information extracted from tweets with total freedom to explore the geographical, time or thematic dimensions.		* '''Flexible query interface''': We installed Cuebee to allow users to query the information extracted from tweets with total freedom to explore the geographical, time or thematic dimensions.
−
−	~~== Ongoing Work / Current Tasks ==~~
−
−	* The Entities should be filtered since during Extraction we do not filter URLs which contains entities such as "htt","bit","ly" which has occurred most number of times.
−	* Get average size of short URIs and long URIs
−	* Get a list of most common entities
−	* Get a list of most common entity-types
−
−	~~==Future work==~~
−	*1. Make sure the URL is not short by checking it recursively
−	*2. Themes-Entity extraction from the tweets in addition to using the present available themes
−	*3. Link tweets to the Linked Open Data cloud.
−	*3.1. Expose RDF describing tweets and their relationships with entities, themes, time, geospatial and URLs
−	*3.2. Link to URIs in the cloud. (depends on item 2)
−	*5. Compare themes/entities extracted from tweets with entities extracted from search query logs
−	*6. Font size depending on the number of occurrences
−	*7. Text Boxes rather than drop down to find the urls of the host typed
−	*8. Search Engine kind of application.