Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This page is for discussing ways in which we can accommodate multiple projects using same sensors without duplicating data.

Approach 1- Aggregation and Filtering on backend

  • Have the backends make requests between them and filter the data in the backend before returning to geodashboard.
  • The api returns aggregated data from own database and data recieved from API requests to other projects

...

  • .

Drawio
width
bordertrue
diagramNameApproach A
simpleViewerfalse
linksauto
tbstyletop
lboxtrue
diagramWidth491
revision1

...

Drawio
bordertrue
diagramNameApproach B
simpleViewerfalse
widthlinksauto
tbstyletop
lboxtrue
diagramWidth411
revision2

...

Drawio
bordertrue
diagramNameApproach C
simpleViewerfalse
widthlinksauto
tbstyletop
lboxtrue
diagramWidth481
revision3

How:

Scenario: Project A needs to show all datapoints of a source in Project B.

  • Partition the Project B database tables on sources. i.e. datapoints is partitioned into datapoints_sourceX, datapoints_sourceY.
    • Queries can still be run on the main datapoints table returning data for all the sources
  • Create a partition table in Project A's database `datapoints_sourceX`.
  • Use foreign data wrapper to connect `datapoints_sourceX` partition table to Project B's datapoints_sourceX table.
  • Now, all the data of datapoints_sourceX will be available in datapoints table on Project A as well without it being copied into Project A database.

Drawbacks:

  • Will require restructuring the database schema.
  • The databases on each machine will have to be exposed so that they can communicate. Currently, they are only accessible on the machines locally.

Link for more information:

https://www.percona.com/blog/2019/05/24/an-overview-of-sharding-in-postgresql-and-how-it-relates-to-mongodbs/

https://pgdash.io/blog/postgres-11-sharding.html