Cache promotion strategies

[ericdmoore]

• Pull to top
• Ahead of time
• evented
• scheduled
• ML based
• ensemble

[ericdmoore]

Some where in the code is a cache interface. Something where you can compose cache layers. Likely fast expensive ones on top, and cheap, slow, large ones on bottom.

After you compose these layers - you are left with pull vs push questions for how to operate the cache.

Pull to top

basically descends the cache cake looking in each layer for the ID, then pulling it out to the top, until there is no more room in the "icing"

Assumptions

1. repeat queries are likely enough to warrant filling up the top layer with "recent" queries.

2. It's too hard to know ahead of time what will be requested so just stay in a reactionary mode.

ML Ordered

Using some ML model - to predict, and then reinforce what will be requested.

Assumptions

1. Assets have natural clumps, due to the semantics within the assets (for example the hrefs available). So when cache gets a request for the file1 in some 8 part series. Go ahead and queue up 2, 3 and the most popular episode or whatever the model predicts will be likely request in the next few minutes.

2. Cache Misses may or may not act like a pull-to-top. Although if there are bandwidth considerations that may also influence the pull to top configuration.