Revision as of 20:05, 22 April 2010

At any second of the day, millions of Web users are simultaneously publishing opinions, observations and suggestions, or generally "social signals" that may represent invaluable information for businesses and researchers around the world.

In this work we investigate the representation of social signals as structured data in order to enable flexibility in handling the information overload of those interested in collectively analyzing social signals for sensemaking.

This is work in progress by Pablo N. Mendes (Kno.e.sis), Alex Passant (DERI), Pavan Kapanipathi (Kno.e.sis) and Amit P. Sheth (Kno.e.sis). It builds upon Twitris and SMOB.

Quick Info

• Real Time: the load estimate for the health care topic drinking from the firehose is
  • 1 post per second
  • 35K triples per hour (tph) or 10 triples per second, steady over HTTP SPARQL Update. Feasible?
• Writeup: Get it from here.

Pitching

• Introduction: But are tweets interesting at all?
  • Every public tweet, ever, since Twitter’s inception in March 2006, will be archived digitally at the Library of Congress. That’s a LOT of tweets, by the way: Twitter processes more than 50 million tweets every day, with the total numbering in the billions. http://www.loc.gov/tweet/how-tweet-it-is.html

• Annotation: Why annotate tweets?
  • Ideas for Twitter’s new Annotations — from obvious to intriguing http://digital.venturebeat.com/2010/04/16/twitter-annotations/

• Information Overload: but how can you make sense of so much data?
  • As reported by ReadWriteWeb recently, during an emergency it’s practically impossible to get status updates on things like roads, hospitals, airports, and people using Twitter [1]
  • Twitter as a poor vehicle for marketing [2]. Many people make up hashtags as they tweet, exploding the semantic graph, creating more semantic dispersion. Some promising new tools that can help you quickly put a hashtag in context — or let people easily look up the meaning of the hashtags you launch or use [3]
  • Wouldn’t it be cool if Twitter had a topic backbone and you could snap your tweets to it as you write them? [4]

• Use cases and commercial interest
  • Many companies using are using microblogging data. Some companies call it real-time web intelligence or business intelligence social media,
  • http://www.evri.com/
  • http://tweetmeme.com/
  • http://www.sysomos.com/
  • http://www.bing.com/twitter
  • http://www.gnip.com/
  • http://www.ellerdale.com/
    • the ellerdale project makes data more relevant and valuable. ellerdale develops and licenses a web intelligence platform optimized for large, real-time data feeds, including all tweets

sent world-wide.

• • Content recommendation
    • http://getglue.com/

• Information Delivery: Push vs Pull
  • Siegel’s rule for information life span: The half-life relevance of a piece of pushed information is about the same as the frequency of the medium. [5]
    • Twitter developed a new set of frameworks @anywhere for adding this Twitter experience anywhere on the web. Imagine being able to follow a New York Times journalist directly from her byline, tweet about a video without leaving YouTube, and discover new Twitter accounts while visiting the Yahoo! home page—and that’s just the beginning. [6]
  • Persistent Search: http://billburnham.blogs.com/burnhamsbeat/2006/04/persistent_sear.html
  • Understanding the Real-Time Web for Web Developers [7]
  • Decentralized Microblogging
    • decentralized semantic microblogging.

Architecture

sparqlPuSH http://code.google.com/p/sparqlpush

Related

• Bibliography of Research on Twitter & Microblogging [8]
• Priamos a middleware architecture for real time semantic web [9]

At Kno.e.sis

• Social Signals @kno.e.sis
• A. Sheth, Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A comprehensive path towards event monitoring and situational awareness, February 17, 2009.
• A. Sheth, Citizen Sensing, Social Signals, and Enriching Human Experience- IEEE Internet Computing, July/August 2009.
• Meenakshi Nagarajan, Karthik Gomadam, Amit P. Sheth, Ajith Ranabahu, Raghava Mutharaju, Ashutosh Jadhav: Spatio-Temporal-Thematic Analysis of Citizen Sensor Data: Challenges and Experiences. WISE 2009: 539-553 [10]

At DERI

• Microblogging: A Semantic Web and Distributed Approach (Paper at SFSW2008)
• A Distributed Semantic Microblogging Platform (Challenge submission SFSW2008)

At Twitter.com

• Twitter Annotation Feature
  • Twitter is planning a feature that allows you to annotate a tweet with structured metadata. http://groups.google.com/group/twitter-api-announce/browse_thread/thread/fa5da2608865453
  • Twitter will launch Annotations next quarter. Annotations will allow developers to “add any arbitrary metadata to any tweet.” For instance, today Tweets can be tagged by the application that created it, the location it was sent from, and who it is replying to. All of this metadata allows Tweets to be sorted and filtered in interesting ways. Rather than wait for Twitter to officially launch a new type of annotation, Twitter is opening that up to developers who can come up with their own. http://techcrunch.com/2010/04/14/twitter-user-streams-annotations/#ixzz0lrTNKGvY
  • not only because it now allows for additional context to be attached without impacting the 140-character limit, but also because it can be used by end-applications to add or derive richer semantic value from otherwise terse content. http://nityan.wordpress.com/2010/04/22/f8-chirp-and-the-increasing-importance-of-the-semantic-web/

• Twitter user streams
  • "User Streams" (supporting real-time push updates to clients without rate limiting) http://techcrunch.com/2010/04/14/twitter-user-streams-annotations/

Semantic Microblogging

• http://smob.me
• http://semantictweet.com
• http://smesher.org/
• http://openmicroblogging.org/
• http://status.net/

• http://semantictwitter.appspot.com/
  • HyperTwitter is semantic hashtags on Twitter. Associate hashtags together and then performer searches. Clever. Though you might want to create a special Twitter account for doing the associations rather than sending these commands through your main Twitter account.
  • Technical Report

• http://www.semanticwave.com/blog/archives/2008/01/hashtags.jsp
• http://www.microformats.org/wiki/twitternanoformats

Maybe related

• Short and Tweet: Experiments on Recommending Content from Information Streams [11]
• Cheng. Fall'09 class project at iSchool (Berkeley). Classifying Metatweets pdf
• Klout on health care: [12]
• Topsy on health care: [13]

• Krishnamurty, Gill, Arlitt. SIGCOMM'08. A few chirps about twitter. pdf
  • classifies 100,000 users in broadcasters, acquaintances, miscreants or evangelists.

Streaming SPARQL

• Barbieri et al. C-SPARQL: SPARQL for Continuous Querying WWW'09 poster EDBT'10
• Barbieri and Della Valle, LDOW2010. A Proposal for Publishing Data Streams as Linked Data (A Position Paper) [14]
• Streaming SPARQL - Extending SPARQL to Process Data Streams ESWC'08
• A SPARQL Engine for Streaming RDF Data SITIS'07

• DSMS - Data Stream Management Systems
  • Abadi, SIGMOD'03. Aurora: A Data Stream Management System
  • Arasy, 04. STREAM: The Stanford Data Stream Management System

Scalability

• 4store Amazon Machine Image and Billion Triple Challenge Data Set http://thinklinks.wordpress.com/2009/10/27/4store-amazon-machine-image-and-billion-triple-challenge-data-set/