We have a some external databases where we want to poll new data from in every 10 minutes. We want to apply this to some other data sources as well so it is not just limited to database, but you can think of db as 1 data source for this.

In every 10 minutes, we want to check if something happened (e.g. new entries were added).

Now, going through all database table for that is quite pointless as we can use a cursor (like a bookmark) that saves last id or timestamp from previous polling query and queries the newest data using that cursor as the last change pointer. This way we don’t have to query whole table.

This is just an idea that spawned into my mind, are there any other efficient ways to do it (polling)?

submitted by /u/IamJustAWizard
[link] [comments]