Which key-value database should I use and why?

Which key-value database should I use and why?

I have an application that scrapes webpages. Once a webpage is scraped, it has a unique identifier which is a string and it has the associated content which is also a string. I need to persist the webpage content, so that the next time I scrape the data for the same webpage I need to know whether the webpage has changed or not.

So either I might be storing the page identifier mapped with its checksum or simply its content. Anyhow, the basic need of storage is simple – a string to string map.

I think it is a use case for a key-value database. Since, my needs are very simple I don’t need a database full of fancy features. What could be a good open source key-value database that suits my simple needs?

Also, is it even necessary to use a database? I can fall back to a file storing the key-value. Will I lose performance if I use a database over a file?

Go to Source
Author: Navjot Singh