Which key-value database should I use and why?
I have an application that scrapes webpages. Once a webpage is scraped, it has a unique identifier which is a
string and it has the associated content which is also a
string. I need to persist the webpage content, so that the next time I scrape the data for the same webpage I need to know whether the webpage has changed or not.
So either I might be storing the page identifier mapped with its checksum or simply its content. Anyhow, the basic need of storage is simple – a
I think it is a use case for a key-value database. Since, my needs are very simple I don’t need a database full of fancy features. What could be a good open source key-value database that suits my simple needs?
Also, is it even necessary to use a database? I can fall back to a file storing the key-value. Will I lose performance if I use a database over a file?
Go to Source
Author: Navjot Singh