nosql – World Of Nubcraft

How to create an Event-Driven Architecture with many external sources

I work for this Accounting company – as a Product Owner – that uses several external services (one for Contract management, another one for Accounting Demands management, a third one for Financial purposes and so on).

My IT Director wants us to build an Event-Driven Architecture to gather all the events that happen externally and bring them into our NoSQL in as-close-as-possible realtime.

The problem specifically is: how can I gather this data from external sources? What would be the best procedure for hundreds of events every day from different services? Consuming their APIs periodically?

Can you guys shine a light on how this can be properly done?

Go to Source
Author: Otto Neres

What is the best database design for storing survey form with different types of questions and answer formats and branching is possible?

I would like to store the format of the survey form which can branch into different question based on
Questions can be video, audio, text and answer can be text, multiple choice, video, audio, geolocation etc. Also based on the answers of a question branching into different question should be possible. It should also be possible for user to fill the form in multiple session so some state should also be there. So the answers to the columns can be missing due to branching as well as the response being incomplete. There is a need of fast filtering and analysis of the database. Also, it should be possible to extract all the responses of a particular form in CSV file. What would be the best implementation for this problem?

Go to Source
Author: Shrey Paharia

Which key-value database should I use and why?

I have an application that scrapes webpages. Once a webpage is scraped, it has a unique identifier which is a string and it has the associated content which is also a string. I need to persist the webpage content, so that the next time I scrape the data for the same webpage I need to know whether the webpage has changed or not.

So either I might be storing the page identifier mapped with its checksum or simply its content. Anyhow, the basic need of storage is simple – a string to string map.

I think it is a use case for a key-value database. Since, my needs are very simple I don’t need a database full of fancy features. What could be a good open source key-value database that suits my simple needs?

Also, is it even necessary to use a database? I can fall back to a file storing the key-value. Will I lose performance if I use a database over a file?

Go to Source
Author: Navjot Singh