Database for consuming twitter streaming API

What would be a good database to use for developing a site with twitter streaming API?

It depends on what you wan’t to store away, how much you store away, if you wan’t to have it locally or on a dedicated DB server or even clustered. Also licensing of the database system can have a huge impact on this decission or the customers requirement.

But to answer the question as generic as you have asked it: any.

Thanks @NobbZ…I want to save the tweets as they are coming in…it’s not for a company…do you think PostgreSQL would be a good choice?

PostgreSQL is often a good choice. But unless you really expect loads of data even a plain file would do.

Since you didn’t told that you wan’t to read the data from the DB, I’d even suggest /dev/null, its probably the fastest for Write-Only data :wink:

If though, it is true that you do not expect to read the data back in, I am near to suggest a simple logging DB like LogStash.

If you wan’t to read data back in, I’d stick with PostgreSQL until you hit issues that are provably related to it.


NB: As you can see, you might think your requirements are already clear as you have written them, but if one takes them for granted, it will probably mess up everything :wink: So before telling your requirements, think of them. Perhaps speak them out loud to a Rubber Duck (I have a easter chocolate rabbit on my desk for this), maybe you will discover the gaps in qour requirements then as well :wink:

@NobbZ…I will be saving a lot of data…I am thinking of building something along the lines of this http://birdwatch.matthiasnehlsen.com to give you a better idea.

When it comes to databases, I would ask myself “Why not Postgres?”.

PG is an awesome database and extremely flexible.

Thanks @AstonJ…I absolutely love Postgres

1 Like

So you already know it. You are developing on your own. So keeping the number of things to learn at a minimum seems logical. For a draft I had even considered SQLite if it were the only RDBMS I know.

You can always swap technologies later on. Especially if they are abstracted away very well.

I have done this a couple of times in day job. Wasn’t that easy as it were here, but you get the point I think.

1 Like

Thanks again @NobbZ