Capturing base Twitter discussions

These are some of the steps Truffle uses to semi automatically capture Twitter discussions like #Agchat. They are not perfect and do require some understanding some information technology (LAMP, Ruby, regex). Ideally this would have a simple interface where you supply the Twitter search term and magically returned is the HTML for your web page... That's for the next version.

This whole post is mainly the technical / manual steps. What is missing is the editorial process steps and discussed in later post.

FYI, These steps rely on your having Ruby plus Hpricot (install and info) to interpret the support files.

Grab the tweets in a big batch.
  • GetFromTwitter.rb contains the Ruby code that helps extract from Twitter specific tweets.
  • GetThisDiscussion.sh (below) is a linux bash command file that wraps around GetFromTwitter.rb to better organize the capture of Tweets.
    # $1 hash tag without hash sign and no spaces, no quotes on command line
    # $2 description of captured conversation, in double quotes on command line
    # bash GetThisDiscussion.sh co_health "CoHealth discussion"
    datestamp=`date +'%Y-%m-%d-%s'`
    # date search only works on computers that are using UTC time, like Twitter server
    datesearch=`date +'%Y-%m-%d'`
    ruby GetFromTwitter.rb 2010-01-11 $datesearch "$1" "$2" TruffleWebAnalytics.html > $1$datestamp.html
    
  • AmiGetThisDiscussion.sh (below) wraps around GetThisDiscussion.sh to provide logging/monitoring information. This is also the bash command tool that is put into an automated process on a linux server to capture Tweets at specific times (using Cron for those that are interested).
    # $1 hash tag without hash sign and no spaces, no quotes on command line
    # $2 description of captured conversation, in double quotes on command line
    # example use
    # sudo bash AmiGetThisDiscussion.sh agchat "#agchat update"
    datestamp=`date +'%Y-%m-%d-%s'`
    datestamplog=`date`
    # date search only works on computers that are using UTC time, like Twitter server
    datesearch=`date +'%Y-%m-%d'`
    cd /vol/var/truffle/discussions/
    echo ================
    echo start $datestamplog
    echo start get discussion $1$datestamp.html
    ruby GetFromTwitter.rb "$1" "$2" TruffleWebAnalytics.html > $1$datestamp.html
    echo end get discussion $1$datestamp.html
    datestamplog=`date`
    echo end $datestamplog
    echo ================
    

What would be better is a visual Twitter publishing tool that allows one to drag/drop, move, organize, etc tweets from a specific Twitter conversation. Any takers? Like I said, next version :)