Caching Feature Views in Production

Private Preview

This feature is currently in Private Preview.

This feature has the following limitations:

• Must be enabled by Tecton support. Sdk version must be 0.7.7+.
• Official parameters will be available in Tecton sdk version 0.8.

If you would like to participate in the preview, please file a feature request.

For use cases with high traffic and duplication, caching can provide benefits in both cost and latency for your production deployment.

Latency Benefits

Retrieving from the cache skips the extra overhead of Feature Server computation needed to process feature values including aggregations and joins, along with the overhead required to retrieve the feature value from the online store.

Latency Benchmark

Benchmarks were run for a sample Feature View to measure the latencies with different simulated cache hit rates. The benchmark details are as follows:

1. Feature Service contains 10 Aggregate Feature Views each with 3 Aggregate Features (sum, count, mean).
2. Feature Service uses 3 entity keys.
3. Each aggregate feature has a time window of 30 days and a 1-day aggregation interval.
4. The benchmark was run with 10 minutes of random traffic at a constant 10,000 QPS.

Latency Benchmark Results

Total QPS	Cache Hit Rate	P50 FS Latency	P90 FS Latency	P95 FS Latency	P99 FS Latency
10,000	0%	7ms	12ms	16ms	29ms
10,000	50%	5.5ms	7.5ms	9ms	22ms
10,000	95%	1.5ms	5.5ms	6ms	8ms

NOTE: This does not include the network latency to/from the client, only the time it takes for the Feature Server to compute the feature values.

Cost Benefits

Caching high-duplication use cases can also have a two-fold benefit for cost savings.

1. Reduces the volume of queries to your online store by a number proportional to your average cache hit rate.
2. Reduces the number of Tecton credits you consume.

Below is a scenario with the Feature Service used above for benchmarking. Cost analysis is done assuming AWS region us-west-1 and the default on-demand mode Dynamo database.

Total QPS	Cache Hit Rate	DynamoDB Read Cost/Month
10,000	0%	$36700
10,000	50%	$18350
10,000	95%	$1835

Security

The cache cluster is secured through the following attributes:

1. The cache is managed in your Tecton Account's dedicated Cloud Account & VPC.
2. All connections to the cache are between private endpoints in the VPC and communications will not go through the public internet.
3. All connections are TLS encrypted.
4. The user controls the TTL of the data and the TTL must be less than or equal to 1 day.
5. No data is ever persisted by Tecton.