IndieWebCamp is a 2-day creator camp focused on growing the independent web

Twitter

v65oai7fxn47qv9nectx_200x200.png Twitter (also known as Twitter.com) is a popular content hosting silo most well known for 140 character text notes.

Features

Twitter provides the following features:

Replacing

If you recreate these features on your own site, including POSSE support to Twitter for each, then you no longer have any need to directly use Twitter.

To fully replace your usage of Twitter you must also support (roughly ordered by importance, expected frequency of usage)

  1. import your pre-indieweb tweets to your own domain (accessible via your own site navigation, search, archives).
  2. on a tweet permalink: (e.g. that you navigated to from a link elsewhere)
    • click "Reply", "Retweet", or "Favorite" and have the action go to your site instead (which then POSSE's it back to Twitter).
  3. on a Twitter profile page (e.g. that you navigated to from a link elsewhere)
    • click "Follow" or "Unfollow" and have the action take place on your site instead (which your site could POSSE sync to Twitter as well)
  4. expanded search for all the use-cases that you currently use Twitter search for. This is likely different for different people and is thus open to some interpretation. Here are some specifics to support search to replace your usage of Twitter. Support search of:
    • all posts of from everyone who has posts on your site (just you if you're the only author)
      • only posts from those you follow in your reader (or just your posts if you lack a reader)
      • only posts with media (e.g. photo or video even if just auto-embedded)
      • only "top" posts (make up your own meaning of "top" as Twitter's is undefined)
      • only posts with location info that are "near you" (make up your own meaning as Twitter's is undefined)
    • authors (people) of posts on your site (just yourself if you only show posts written by you)


The end result is that all your interactions with Twitter and Tweets via your own site need to be at least as seamless as Twitter's UI (so you'll use your own UI instead).

Clients

Twitter has several clients that all have a fairly complete implementation of the above features:

Recommendations

CC license your tweets

Note, tweetcc's creators decided to take it down in July 2013."[5]

We're looking into having tweetcc restored.[6]

Use the site http://www.tweetcc.com/ to Creative Commons license your tweets.[7]

This allows anyone else wanting to:

  • copy
  • quote
  • syndicate

Etc. your tweets to do so, per that CC license (e.g. CC-BY, attributing you) independent of Twitter's TOS, and thus independent of any Twitter requirements, e.g. their display "guidelines".

Reverse Syndicating At-Replies

Per the POSSE model, you may want to reverse-syndicate @-replies on Twitter to the tweet-copies of your post as "comments" back into your post permalink page on your own site. By using TweetCC, you can check to see if those @-replying to you have CC-licensed their tweets and then incorporate them using the CC license, displaying them however they fit in with your site design, instead of having to shoehorn Twitter's display "guidelines" into your site / post-permalink page design.

Data Export

Download all your tweets

  1. Go to: https://twitter.com/settings/account
  2. near the bottom of the page there may be a "Your Twitter archive" section
  3. See the (Request your archive) button? Click it.
  4. Wait some amount of time for a link to your archive to show up in your email associated with the account.

The Twitter archive is made of per month JSON files. It can be used as a way to PESOS a whole Twitter history, even though POSSE is the recommended way to do it.

The most interesting things in the JSON twitter archive are:

  • both long and shortened URLs
  • "new" RTs are also written the classic Way, making easy to republish them in a PESOS way.
  • every related information needed to rebuild a conversation

After you download your archive, you can update in using a Ruby Gem called GrailbirdUpdater.

Download Direct Messages

  • You can use this method to download DMs from Twitter. It is complicated to set up and involves registering a new Twitter application. It can only archive about a year's worth of DMs.

Contents

Porting to the IndieWeb

Implementations

freedom.io (source) can copy all of your tweets to WordPress, Tumblr, Blogger, creating a new blog post for each, with the original dates and with replies and retweets (yours and others') included as comments.

It's rough and unpolished. Ryan eventually plans to polish it up and advertise it more widely.

Challenges

  • porting over content and maintaining URL redirects
    • not currently possible to redirect Twitter permalinks to your own site AFAIK. Tantek 14:33, 17 November 2012 (PST)
  • porting over @-replies
  • porting over retweets
    • Twitter has an API endpoint for getting retweets of a particular tweet.
      • The first 100 retweets of any particular tweet can be fetched via the API
  • porting over favorites
    • Twitter has *no* direct API for getting favorites of a particular tweet. You can get them as they happen from the streaming API, or you can screen scrape, but neither are ideal. Bridgy has example code for both.
  • API limited to most recent 3,200 tweets
  • equivalent user interface
    • "twitter's editor is hard to beat though, like auto-expanding twitter handles and telling you how many characters you have left (with accounting for URL-shortening)"
      • Re: "how many characters you have left (with accounting for URL-shortening)" - there's CASSIS functions for that:
        • tw_text_proxy($t) - given plain text for a note, it returns a string with character count that Twitter would give it, e.g. after t.co-wrapping URLs. Useful for length estimations, and doing truncation/ellipsing computations.
          • e.g. 140-strlen(tw_text_proxy($t)) gives you # of characters left
        • note_length_check($note, $maxlen, $username) - given a string $note (e.g. the output from tw_text_proxy() above), a $maxlen (e.g. 140), and a $username (e.g. "@indiewebcamp"), it returns result codes (roughly based on / reusing from HTTP)
          • 200 - $note exactly fits $maxlen characters when retweeted (if $username was provided)
          • 206 - $note fits $maxlen and has room to spare even when retweeted (if $username provided)
          • 207 - $note fits $maxlen with room to spare, but a retweet would get truncated (if $username provided)
          • 208 - $note exactly fits $maxlen and a retweet would be over and get truncated
          • 413 - (entity too large) $note is over $maxlen

POSSE to Twitter

In short, you have to use Twitter's snowflake API to POSSE posts to Twitter.

POSSE to Twitter in general

There are particularly good ways to POSSE notes and articles to Twitter which are documented in latter sections. Here are some general guidelines for POSSEing to Twitter.

Given Twitter's limitations:

  • 140 characters of content
  • links are t.co'd into 20 (or 21) characters for http (or https) links)
  • local user (@-name) references only
  • local tag (#hashtag) references only

When POSSEing to Twitter:

  1. provide a permalink (or perma-short-ID for entirely fitting notes) to your original at the end of the syndicated copy
  2. abbreviate notes / article names to fit
  3. include hashtags as part of that abbreviated content (even if they're beyond the ellipsis horizon) to increase syndication based discovery/searchability of your content
  4. translate indiewebsite people references into their Twitter local user @-name equivalents

Once POSSE is successful:

  • construct a URL (https) for the POSSE copy tweet and link to it from the original post using rel-syndication

The sections below contain details on how to do some of this for notes and articles in particular.

POSSE Notes to Twitter

For notes, you can POSSE out up to 140* characters of your note to Twitter.

POSSE entire note to twitter

For notes that fit entirely within a tweet (*including permashortcitation), you should include a post permashortcitation at the end of your tweet. Examples:

has the permashortcitation of a permashortid in a classical parenthetical citation:

  • (ttk.me t4Ec3)

which is easily expanded (see related: original-post-discovery) by unparenthesizing and joining the two pieces with a forward slash "/":

  • ttk.me/t4Ec3

which resolves (redirects) to the original post:

Why permashortcitation instead of a link

You should NOT include a post permashortlink at the end of tweet copies of notes that include the entire note.

There's an unspoken convention on Twitter that a link in a tweet (especially at the end of a tweet) should provide more information.

Having an active permashortlink at the end of your tweet when all you provide on your own site is the *exact* same content is bad UX for your friends that follow you on Twitter.

Note the complaints ([8][9](response), [10](response), [11][12]) and questions[13] over time that have been raised in response to apparent linking to "duplicate" or "the same" content.

The whole point of POSSE is that you still care about your friends on Twitter (or other silos) reading you, so you should care about their UX also.

Only provide permashortlinks as part of your POSSE tweets when they link to an the original post with more content than in the POSSE tweet copy. Otherwise use permashortcitations so that your original posts are still automatically discoverable using the original-post-discovery algorithm.

POSSE abbr note to twitter

If your note (with permashortid) exceeds 140 characters, you should abbreviate your note to 117 (116) characters (e.g. using the CASSIS ellipsize_to_word function), and then put a post perma(short)link URL at the end. Shorten to 117 (116) characters (or shorter) because it takes 1 character for the space before the URL, and then Twitter t.co's all URLs (including yours) into 22 characters (23 characters for new https URLs).

When shortening to 117 (116) characters, you must count other http (https) URLs within that as 22 (23) characters (excepting schemeless pathless ccTLDs). You can use the CASSIS tw_text_proxy function to automatically replace http (https) URLs in a string with 22 (23) character proxy URLs. The return result can be used for counting how many characters the string will use as a "tweet". This is useful for e.g. evaluating ellipsize_to_word on the result, to get a length of characters that you can POSSE, and then using that to help determine how much of the original (non-proxy) string can be POSSEd.

You may want to put an ellipsis character at the end of your POSSEd note to Twitter before your perma(short)link. Be sure to leave room for the ellipsis too: instead of just 1 character for a space before your permashortlink, save 4 characters for "... ", or 2 characters for "… " (ellipsis entity character). Note Twitter now counts characters in a sensible fashion, they count codepoints in Unicode NFC: https://dev.twitter.com/docs/counting-characters. Ideally elide at a punctuation boundary (like sentence terminator or a comma - the above-mentioned CASSIS function does this too). Examples (which link to original posts after eliding with "... ")

Exception, if your abbreviated tweet already has a ":" (colon) character at the end, no need for an ellipsis, simply append a space " " and your permashortlink. Examples which truncate after a ":"

IndieWebCamp community members who are doing this:

POSSE Notes to Twitter Pseudocode

This is pseudocode based on Barnaby Waltersphp-helpers package, specifically BarnabyWalters\Posse\Helpers class.

Assume that to start with you have some HTML markup representing the content of the note ($text), the canonical URI to link back to from the syndicated tweet ($url) and an optional in-reply-to url associated with the note ($inReplyTo).

  1. Convert any HTML in $text to twitter-like plaintext syntax, or strip it
  2. Truncate $text such that, once all URLs contained within it are replaced with URLs of the current t.co length (21 ATOW) there are enough characters left to contain the canonical URI with brackets around, or preceded by and ellipsis if any truncation has occurred
  3. Trim $text
  4. Append a space and $url, surrounded with brackets if no truncation has occurred or preceded by an ellipsis if truncation has occurred
  5. Create a hashtable representing the body of the POST request to send ($tweet)
  6. Set $tweet['status'] to $text
  7. Check to see if $inReplyTo is the URI of a tweet (see line 58)
    • If it is, grab the ID
    • Set $tweet['in_reply_to_status_id'] to the ID
  8. Send an authenticated POST request to the relevant API endpoint with the contents of $tweet as a urlenoded body

POSSE Articles to Twitter

Similar to POSSEing a note, you can POSSE an article to Twitter, but instead of the first 140 characters of the post, you should POSSE the first 140 characters of the entry name (title) of the article, followed by optionally a ":" character (unless the entry title already ends with punctuation), then a space " ", then the perma(short)link to your original post. Examples:

IndieWebCamp community members who are doing this:

POSSE Replies to Tweets

Similar to POSSEing a note to Twitter, when POSSEing a reply to a tweet, you also set the in_reply_to_status_id field in the API call to the URL of the tweet that your note is in reply to, to enable the threading UI and presentation on Twitter.com.

You can use this regular expression to extract tweet IDs from tweet URLs:

/https?:\/\/twitter.com\/[a-zA-Z0-9_]{1,20}\/status\/([0-9]*)/

Examples, i.e. IndieWebCamp community members who are doing this:

POSSE Replies to Twitter

When posting a comment which is a reply to another indieweb post, the POSSE tweet of your comment should attempt to set the in_reply_to_status_id to the POSSE tweet of the indieweb post you're replying to.

This is similar to how you POSSE Replies to Tweets but with one extra step to discover the POSSE tweet of the original post:

See also: in-reply-to, comments.

If the indieweb post (or other URL) has no equivalent POSSE tweet and is not a tweet URL itself, then:

  • include the non-Twitter in-reply-to URL as the last thing before your permashortlink in the POSSE tweet copy of your reply

POSSE Reposts of Tweets

When posting a repost of a tweet, the proper POSSE behavior should be to do a native retweet (on Twitter) of the tweet that you're reposting on your own site.

Examples:

POSSE Reposts to Twitter

When you do a repost of an indieweb post:

  • look for a rel-syndication hyperlink on the indieweb post permalink page to twitter.com
  • if there is such a rel-syndication hyperlink with href linking to a tweet URL
  • then
    • do a native retweet of that tweet URL on Twitter itself
  • end if

POSSE Favorites of Tweets

When posting a favorite of a tweet, the proper POSSE behavior should be to natively favorite (on Twitter) the tweet that you're favoriting on your own site.

Examples:

POSSE Favorites to Twitter

When you post a favorite of an indieweb post:

  • look for a rel-syndication hyperlink on the indieweb post permalink page to a tweet
  • if there is such a rel-syndication hyperlink with href linking to a tweet URL
  • then
    • natively favorite that tweet URL on Twitter itself
  • end if

Embedding Tweets

Twitter offers an embed link on its site that allows one to embed the appearance/text of a tweet in one's own page, as well as oembed option. As both the oembed and the embed include the text of the tweet in a blockquote, if Twitter shut down, this could continue to display the text minus the appearance.

iOS client

See full article: Twitter-API

The Twitter iOS client has an option to use a different API root, "intended for a Twitter proxy server"

It may be theoretically possible to re-implement enough of the Twitter API on your own site to set your own site as a "Twitter proxy server" and thus use the Twitter client as a client to post directly to your own site (and perhaps read from it as well).

Criticism

Bad or failing behavior on the part of Twitter.

Downtime

Live: http://downrightnow.com/twitter

Most recent first:

spammy cross-posting to facebook

This may be a side effect of both Twitter and Facebook doing arbitrary photo cropping (FB likes square images, Twitter likes 16:9 wide ones, both often crop rather than fit to space with background). Kevin Marks posted screenshot of Facebook to twitter. fbupcomingclone0.png

When this was replicated to facebook by Twitter's app it looked like this:

fbupcomingclone.png

cropping the screenshot and requiring a clickthrough back to twitter to make it readable, with explicit text added by Twitter when cross-posting:
Get the whole picture - and other photos from Kevin Marks
Emphasis added - which implies deliberate behavior of a spammy sort (using your cross-post to advertise broader use of Twitter than just that one permalink).

Issues

Correctable issues with Twitter.

Email Identity Removal

This 2013-05-17 screenshot seems to demonstrate an odd interaction and vulnerability on Twitter:

2013-05-17-twitter-com-new-email-warning.png

Screenshot shows a logged in Twitter profile page (http://twitter.com/t) with a yellow warning box just below the global black toolbar that says:
New email address required. Twitter has removed the email address from your account, by request of the email owner.
Please enter a new email address where you can be reached. Learn more ›

Hyperlinks as present in original. No such warning on m.twitter.com equivalent page when logged in (i.e. on a mobile device).

It's not clear how this happened, how the email address was revoked, how to avoid having it revoked in the future etc.

Appears to be an identity threat/vulnerability.

Related posts:

History

Early User Interface

Twitter's early user interface (circa 2006-2007?) was quite simple and minimal:

Features

  • Simplicity
  • Easy to navigate, one click from your stream to someone's profile
  • Could opt to see all @-replies, even if you only followed one side of it.
    In the early days, half of how I found new interesting people to follow was catching half a conversation and checking out who the other person was.-- gRegor Morrill on IRC

Lots of UI revisions

Look for #newtwitter and #newnewtwitter

UI 2014-04

As of 2014-04, Twitter has yet another UI revision, this time nearly cloning Facebook profile pages' look & feel:

https://twitter.com/genmon/status/459257786875265025/photo/1

Bl-chggIMAAGyBZ.png

Facebook on the left, new Twitter UI on the right.

Additional Resources

Profile Image URLs

Programmatically, the official way to get a Twitter user's profile image is to use the profile_image_url and profile_image_url_https fields returned by the users/show v1.1 API call. More details.

Unfortunately, those fields are absolute URLs for the *current* profile image, and usually break when the user changes their profile image.

Unofficial alternatives: these both 302 redirect to the current profile picture. Replace username with the Twitter username without the @.

  • https://twitter.com/username/profile_image
  • http://avatars.io/twitter/username

Ryan thinks the first one might be based on the users/profile_image v1 API call. If so, it will probably disappear soon, since the v1 API is already disabled. That's just speculation though.

Twitter profile images can also be found by choosing "View Image" on context menu on a profile, those URLs may end with:

  • _normal.png - 48px square
  • _200x200.png - 200px square
  • _400x400.png - 400px square
  • or no "_" and just .png - actual uploaded size

You can change any of the above suffixes to any others in order to get a few different sizes.

See Also