vendredi 24 juin 2016

Architecture of a write intensive feature

I use Ruby on rails backed by oracle database and memcached for my current project.

There is a pretty heavy used feature, which relies on a single database views as a datasource, and this data source internally has other database views and tables inside.

It's a virtual db view, to be able to access everything from one place, not a materialized db view.

Users most of the times if they are in the feature they are looking to update, so having data up to date is important.

When obtaining data from this view, I inner join security table to the view (security table is not part of the view itself) which contains some fields that we use to control data access on more granular level. For example security table has user_id, prop_1, prop_2 columns, where prop_1, prop_2 are columns available on a db view and the user_id is a logged in user. Some users have same props in the security table say prop_1 = 1 and prop_2 = 1, but also can have prop_1 like the other user but have different prop_2 like prop_1 = 2 and prop_2 = 1. There are many different combination of prop_1 and prop_2, think about them as a FK to another table, so possible to have many entries.

By now the time to retrieve the records on the app is almost 10 seconds, it's pretty slow. I m considering alternative approach.

First thing I though of was the materialized view, but since the user do frequent updates, it might not be the best choice, as refreshing the view might take time.

Second thing I thought about was the cache, to use prop_1 and prop_2 combination as a composite key to the underlying data, as many users have the same combination and whoever has the same combination can access the same data.

However this approach might require more code rewrites and logic to save and retrieve data in fragments, rather from one location with one query like in the database view.

In your experience, how did you address same/similar issue? Or is there a better approach that I could try?

For those of you about to ask, what have you tried. I m thinking about the solution first, gathering information from reliable resources and more experienced people, then I m gonna make an informed decision and start implementing. Implementing first, thinking second, proved so wrong so many times

Aucun commentaire:

Enregistrer un commentaire