Once upon a time, I viewed capturing 1 million tweets as a challenge. Now that that’s in the rearview mirror, I’m pondering new achievements. This one is really audacious.
1 BILLION TWEETS! (curling lip ala Doctor Evil and chuckling diabolically)
That would actually be something outstanding for a single individual. Consider that to do it in a calendar year, you’d have to average just a little south of 2.75 *million tweets per day, 114K per hour, 31 per second. I’m not even sure that’s possible with Twitter’s streaming API. And then there’s the bandwidth and storage issues. Not to mention maintaining pretty high annual uptime to stay on target.
Now that would just be the capture. What the heck would you then do with all that data? Imagine the possibilities.
I think a reasonably well funded individual could pull it off with careful exploitation of Amazon Web Services, but it would definitely be non-trivial. I can dream can’t I!