Twarql
From Knoesis wiki
Twarql: Twitter Feeds through SPARQL
Contents
Introduction
Our approach encompasses the following steps:
- extract content (entity mentions, hashtags and URLs) from microposts;
- encode content in a structured format (RDF) using shared vocabularies (FOAF, SIOC, MOAT, etc.);
- enable structured querying of microposts (SPARQL);
- enable subscription to a stream of microposts that match a given query (Concept Feeds);
- enable scalable real-time delivery of streaming data (SparqlPuSH).
Architecture
- See the workflow between the components of the architecture
Tweet Annotation
- extract content (entity mentions, hashtags and URLs) from microposts;
- encode content in a structured format (RDF) using shared vocabularies (FOAF, SIOC, MOAT, etc.);
Concept Feeds
- enable structured querying of microposts (SPARQL);
- enable subscription to a stream of microposts that match a given query;
SPARQLPuSH
- More about the SPARQL Push implementation: Introducing sparqlPuSH, open source implementation
Twarql API
REST Endpoints
Summary:
- URL scheme: http://<base-url>/<operation>/?<parameter>=<value>&...&output=<output format>
- Base URL: http://knoesis1.wright.edu/twarql
- Operations: search, register, stream, query
- Output formats: twitter-json, sparql-json, entities
Parameters:
- http://knoesis1.wright.edu/twarql/search?keyword=k1,...,kn&output=<output type>
- input: keywords, output type (twitter-json, entities, sparql-json)
- output: tweets, entities, triples
- http://knoesis1.wright.edu/twarql/search?keyword=k1,...,kn&output=<output type>
- D2R
- http://knoesis1.wright.edu/twarql/register?query=<sparql query>&id=<concept feed id>
- #id
- http://knoesis1.wright.edu/twarql/stream?keyword=k1,...,kn&output=<output type>
Output Formats
We also provide output according to the format presented on the twitter-api-announce message.
{ "text" : "hey @raffi tell @noradio to check out http://dev.twitter.com #hot", ... "entities" : { "user_mentions" : [ { "id" : 8285392, "screen_name" : "raffi", "indices" : [4, 9] }, { "id" : 3191321, "screen_name" : "noradio", "indices" : [16, 23] } ], "urls" : [ { "url" : "http://dev.twitter.com", "indices" : [38, 64] }, ], "hashtags" : [ { "text" : "#hot", "indices" : [66, 69] "url" : "http://search.twitter.com/search?q=%23hot" } ] } ... }
Supported Clients
- SPARQL Protocol-compliant Clients
- Cuebee is a SPARQL query formulation and results exploration engine. We provide a TweetExplorer that can be directly plugged into Cuebee.
- Regular Twitter clients
- To support plain Twitter clients, we may have to implement their style to content negotiation http://www.zienit.nl/blog/2010/01/rest/control-jax-rs-content-negotiation-with-filters
- RSS/Atom clients