trendpreneur

With the immense prevalence of Twitter apps, hacks and mashups out there, the concept of a new idea is fast becoming a fallacy. However, I have a problem, and I’ve not yet found a handy solution.

This particular problem, as alluded to here and with various Tweets I’ve sent, is one of catching up. Before the anti-Twitter brigade get going, let’s be clear here: this isn’t just a Twitter-related problem. I subscribe to users on FriendFeed who post items there, and 95% of my Facebook friends post everything on Facebook; I’ve also got a (long unused) set of Livejournal contacts who routinely post interesting stuff, not to mention the number of blogs and del.icio.us feeds I subscribe to and don’t bother reading.

This is a slightly different problem to the one the professional blogger faces — my motivations for reading this stuff are mostly personal, although as one of today’s digital nomads there’s a heavy professional element as well. It isn’t the end of the world if I miss a posted item or two, and I’m fairly interested in the chatter between friends as well as the meat of what they post, hyperlinking themselves into the World Wide Web.

Still. Let’s distill this problem down a little. First off: my friends post too much stuff for me to reasonably attempt to keep up with it using the original services’ interfaces. Second off: I care what my friends find interesting. Third off: I like having a ton of links to open first thing in the morning over coffee. Things that will enrich me, make me smile, interest me, teach me, or inspire me.

How do I currently deal with this? Well, how does anybody? We all have different approaches to this personal level of information overload (oh, how I am coming to detest that phrase).

Step 1 – Cut the fluff

I don’t read things I don’t need to. My Facebook stream is easily caught up with by logging in once a day, or every other day. The level of content there is fun and usually worth reading – finding out what my friends and colleagues are up to does have its worth! – but it’s not going to kill me if I don’t find out about it right this minute.

Similarly, the way I personally use FriendFeed leads to a lot of duplicated content (Tweeted links from people I already follow on Twitter, for example) and it’s really only useful for me because interesting stuff from friends of friends floats up. Your mileage probably varies; I’m by no means a powerful FriendFeed user, as I haven’t quite got the point of it yet, though I like some of the concepts it uses. Either way, I get by checking my FriendFeed purely from the single email a day.

Talking of email, I filter my mailing lists quasi-religiously and use Gmail’s multiple inboxes (in Labs) so I can more easily ignore the volume of email coming through certain labels. I can catch up with these at my leisure; they’re not clamouring for my attention 24/7. But due to the multi inbox placing, I don’t forget about them either.

Twitter is the biggest source of noise for me, and I do a wee bit of filtering on this by not following people whose tweets simply don’t interest me. However, a lot of people post a lot of cool stuff, so my follow list — small though it might seem — is still pretty loud and leaves a lot of catching up to do in the morning since I have the misfortune to be 5-8 hours ahead of half my subscriptions.

RSS is something of an interesting one. I subscribe to loads of RSS feeds, but I’ve stopped reading them. When I have downtime I load up my iPhone and catch up with my ‘key’ feeds – others I subscribe to on Twitter. The rest of them I download every couple of days and save for long journeys so I can look busy when there’s no ‘net connection. I don’t take long journeys often. I used to be a total RSS junkie; I find I get the same level of ‘cool stuff’ out of the other networks I tap into that I used to get out of RSS. However, there are still a couple of feeds from particular people that I check by manually visiting their blogs every few days.

Step 2 – Filter the rest

Okay, so I’ve already cut out a lot of the ‘problem’ by completely ignoring the realtime web and checking things at my own leisure. But I still have Twitter jumping up and down crying for attention in the corner.

This is where I evangelise TweetDeck, as so many do. Three key features: groups, filter, and search.

Groups allow me to set up columns of people I want to read more than others. It’s almost an inverse of my Gmail mailbox, where I cluster and separate stuff I don’t want to read. Unless I’ve totally missed something, when you add a new follow you have to manually add them to relevant groups, so I try to keep my groups organised to save me this effort; most of them won’t grow. I use groupings like ‘RL friends’, ‘Edinburgh people’, ‘Bloggers’ and ‘RSS bots’, although due to reinstalling recently the only group I currently have set up differentiates people I’ve known and followed for a long time from people I just started following. As I follow more people, my stream gets polluted with stuff that’s less and less relevant, so cutting that out helps with the headache.

I still spend a lot of time reading All Tweets, though, and this is where Filter is quite fun. Try filtering your tweets column with the term ‘http’. Instantly you just get the day’s links to peruse at your leisure. Good stuff.

Search is also cool, although it can be overwhelming. It’s just a shortcut to search.twitter.com which allows you to keep everything within TweetDeck; I used it to track the #edtwestival, and would’ve used it for the conferences I was at, had it existed back then.

Step 3 – Pipes, RSS and mashups

This is where it all gets a little kooky and theoretical. Here’s the deal: Twitter provides your friends’ updates via RSS, as well as via its API. Blogs, del.icio.us and poor abandoned Livejournal also have handy RSS feeds. A quick search implies you can get at least some FriendFeed content via RSS. Facebook? Maybe not, but I already decided Facebook content can wait. Still, there’s a theme here.

Many RSS feeds plus the desire to do something cool to them? Enter Yahoo Pipes, as inspired by this Twitter-Pipes URL extractor mashup. (Aside: I’m pretty impressed by the stuff Yahoo is offering for mashups and Internet tomfoolery. BOSS, Pipes and Term Extractor are all fun to play with.)

Using Pipes, it’s theoretically possible to amalgamate a dozen or more RSS feeds of your choosing, strip out all but the relevant http:// links and display them in a handy ‘Here’s your link bag for the night, madam’ format. (I’m imagining Stephen Fry presenting me a stack of virtual newspapers on a virtual silver platter. Too far?)

This is where it gets a little problematic. Firstly, my brief experimentation with Pipes involved having to call a URL of the format http://username:password@twitter.com/statuses/… in order to get my friends timeline. The geek in me won’t touch cleartext with a bargepole. Secondly, I don’t really think this is intelligent enough to produce the level of results I want. I don’t think it’s too hard to beef it up a little, though.

Here’s the deal. Using the API, or RSS, or messenger pigeons, show me: the URLs tweeted by my Twitter friends in the last 24 hours, sorted by interestingness and longURL’d to prevent duplicates. Let me add in other RSS feeds that commonly contain links such as ‘best of’ blogs, delicious feeds, etc. Once I open them up, give me some way of tracking where they originally came from, so I don’t end up confused and unable to attribute if I retweet.

It seems to me that mashing up a delicious or similar service on top of this URL extractor would work really nicely. (Why roll your own when you can overload someone else’s?). When someone tweets a link it gets added to this bucket o’ URLs with tags extracted from the tweet itself, and metadata indicating its source. When I go to retweet, that info is saved, so my retweet writes itself (of course, I can edit it if I really don’t want to give the source credit). Either automatically or manually, I bookmark/star/’like’ items that appeal to me and my friends do the same, thus giving the ‘interestingness’ ranking I handwaved about a paragraph earlier. Perhaps everything I click on gets +1 to interesting, and stuff I explicitly like gets +5. If n is the number of times tweeted/retweeted, interesting = n (clicks + 5 like).

I would also be able to mimic some popular tweet-universe services such as retweet tracking and URL popularisation but to keep them within my friends cloud. What’s the point of limiting my information sources if I don’t take advantage of those limits? Of course, I care about what The Internet thinks (I wouldn’t be in this business if I didn’t), but I also want to know what people I respect and listen to think.

If I ever get a spare weekend, I’ll code this up. If someone’s already beaten me to it, all the better!