Billable Usage Logs
Billable Usage Logs contain detailed information about your use of the Tecton platform, and how that use translates into Tecton Credits. By analyzing these logs, you can understand your usage trends, attribute usage within your organization, and optimize costs.
Logs are written as a CSV file into the Cloud Provider Object Storage you have configured during your Tecton deployment.
Accessing Billable Usage Logs​
Billable Usage Logs are available in your Cloud Provider's Object Storage. Data will be available from July 1st, 2023. How you access the Billable Usage Logs depends on how your Tecton account is configured.
- For AWS:
s3://tecton-{DEPLOYMENT_NAME}/tecton-billable-usage/{VERSION}/YYYY-MM-DD/yyyy-MM-dd-HH.csv
- For Google Cloud:
gs://tecton-{DEPLOYMENT_NAME}/tecton-billable-usage/{VERSION}/YYYY-MM-DD/yyyy-MM-dd-HH.csv
- For Tecton on Snowflake: Open a ticket with Tecton Support to configure Object Storage.
Logs will arrive between 24 - 48 hours after usage of the credits. No files will
be written for periods where there is no billable usage. The current VERSION
of the metrics schema is v1.
Usage Metrics CSV Schema​
- v1
Name | Description |
---|---|
accountName | Tecton Account Name |
workspace | Tecton Workspace |
timestamp | UTC Timestamp (At hour granularity) (For Example: "2023-08-11 00:00:00 UTC") The hour of the file is inclusive with the start time and exclusive with the end time of the event. For example: File 2023-08-11-02.csv will contain the usage from "2023-08-11 02:00:00 UTC" to "2023-08-11 02:59:59 UTC". |
tectonObjectName | Tecton Object Name (Feature View or Feature Service Name) |
quantity | Number of units used by this Tecton object during the hour |
unit | Unit of measurement for billable usage |
Example File Contents:
"accountName","workspace","timestamp","tectonObjectName","quantity","unit"
"sampleAccount","workspace_1","2023-07-22 19:00:00 UTC","user_transaction_metrics",10000,"FeatureViewOnlineReads"
"sampleAccount","workspace_1","2023-07-22 19:00:00 UTC","user_impression_counts",3,"FeatureViewOnlineReads"
Analyzing Billable Usage Logs​
Once Billable Usage Logs are available in your environment, you can use many types of tools to query and analyze the data.
The following example illustrates how to read and analyze the logs from a standard Juypter Notebook environment. Note that your environment will need to be able to access the Cloud Storage path containing the logs.
import boto3
import os
import pandas as pd
BUCKET_NAME = "tecton-<DEPLOYENT_NAME>"
VERSION = "v1"
PREFIX = f"tecton-billable-usage/{VERSION}"
s3 = boto3.resource("s3") # assumes credentials & configuration are handled outside python (e.g. in the .aws directory)
local_dir = "<SOME_LOCAL_DIRECTORY>"
bucket = s3.Bucket(BUCKET_NAME)
for obj in bucket.objects.filter(Prefix=PREFIX):
filename = obj.key.split("/")[-1]
bucket.download_file(obj.key, os.path.join(local_dir, filename))
df_list = []
for filename in os.listdir(local_dir):
full_path = os.path.join(local_dir, filename)
df_list.append(pd.read_csv(full_path))
metrics_df = pd.concat(df_list, ignore_index=True)
# Group metrics by Tecton Object Name + Workspace
usage_by_workspace_object = metrics_df.groupby(["unit", "workspace", "tectonObjectName"])["quantity"].sum()