๐ Tecton Quickstart
Tecton helps you build and productionize AI applications by making it easy to define, test, and deploy features for model training and serving.
Let's take a quick look at how you can build a low-latency streaming feature for a fraud detection use case using nothing but Python.
For an end-to-end tutorial that takes you from raw data to a real-time model, check out Building a Production AI Application with Tecton.
This tutorial is designed to work with our Interactive Demo, which you can sign up for at explore.tecton.ai.
โ๏ธ Install Tectonโ
pip install 'tecton[rift]==0.9.0'
โ Log in to Tectonโ
Make sure to hit enter
after pasting in your authentication token.
import tecton
tecton.login("explore.tecton.ai")
tecton.set_validation_mode("auto")
๐งช Define and test a streaming featureโ
Using Tecton's Feature Engineering Framework we will define 3 new features for our fraud detection model:
- A user's total transaction amount in the last 1 minute
- A user's total transaction amount in the last 1 hour
- A user's total transaction amount in the last 30 days
To do so, we will first define a Stream Source which tells Tecton where to retrieve events online and offline. For the online path, we've configured Tecton to accept real-time events via an HTTP ingestion API which we will try sending records to in the next step. For the offline path, we've pointed Tecton to an S3 path that contains a historical record of our stream. Tecton uses this path for offline development and backfills.
Next we define a
Stream Feature View
which can create one or more features via transformations against the Stream
Source. In a Stream Feature View, we can run Python transformations on incoming
records, and optionally apply time-windowed
aggregations
via the aggregations
parameter. These transformations run identically online
and offline which is critical for preventing skew.
Lastly, we can immediately test our features directly in our notebook via the
get_features_in_range
method. Try it out!
from tecton import *
from tecton.types import *
from datetime import datetime, timedelta
transactions_stream = StreamSource(
name="transactions_stream",
stream_config=PushConfig(),
batch_config=FileConfig(
uri="s3://tecton.ai.public/tutorials/transactions.pq",
file_format="parquet",
timestamp_field="timestamp",
),
schema=[Field("user_id", String), Field("timestamp", Timestamp), Field("amount", Float64)],
)
user = Entity(name="user", join_keys=["user_id"])
@stream_feature_view(
source=transactions_stream,
entities=[user],
mode="pandas",
aggregations=[
Aggregation(function="sum", column="amount", time_window=timedelta(minutes=1)),
Aggregation(function="sum", column="amount", time_window=timedelta(hours=1)),
Aggregation(function="sum", column="amount", time_window=timedelta(days=30)),
],
schema=[Field("user_id", String), Field("timestamp", Timestamp), Field("amount", Float64)],
)
def user_transaction_amount_totals(transactions_stream):
return transactions_stream[["user_id", "timestamp", "amount"]]
# Test the feature locally using historical data
df = (
user_transaction_amount_totals.get_features_in_range(start_time=datetime(2022, 1, 1), end_time=datetime(2022, 2, 1))
.to_pandas()
.fillna(0)
)
df.head(5)
user_id | amount_sum_1m_continuous | amount_sum_1h_continuous | amount_sum_30d_continuous | _valid_to | _valid_from | |
---|---|---|---|---|---|---|
0 | user_2210887384 | 0 | 4.55 | 3429.12 | 2022-01-22 00:00:00 | 2022-01-21 00:00:00 |
1 | user_2417164600 | 0 | 1.97 | 2048.18 | 2022-01-10 00:00:00 | 2022-01-09 00:00:00 |
2 | user_9757807451 | 0 | 98.37 | 12365 | 2022-01-07 00:00:00 | 2022-01-06 00:00:00 |
3 | user_1939957235 | 0 | 0 | 3616.4 | 2022-01-20 00:00:00 | 2022-01-19 00:00:00 |
4 | user_1990251765 | 0 | 0 | 7721.29 | 2022-01-24 00:00:00 | 2022-01-23 00:00:00 |
โก๏ธ Ingest data and retrieve updated feature values in real-timeโ
Tecton objects get registered via a declarative workflow. Features are defined as code in a repo and applied to a workspace (like a project) in a Tecton account using the Tecton CLI. This declarative workflow enables productionization best practices such as "features as code," CI/CD, and unit testing.
We've gone ahead and registered these features so you can try ingesting new events and querying for online features. After features get registered with a workspace, Tecton handles the backfilling and ongoing materialization of data to the offline and online store for training and serving.
This step requires generating an API key, which you can do here. Copy the generated key and paste it in below.
import random, string
tecton.set_credentials(tecton_api_key="your-api-key") # replace with your API key
# Fetch registered Tecton objects
ws = tecton.get_workspace("prod")
ds = ws.get_data_source("transactions_stream")
fv = ws.get_feature_view("user_transaction_amount_totals")
# Generate a random user_id for the next step
user_id = "user_" + "".join(random.choices(string.digits, k=7))
print("Generated random user id: " + user_id)
โ Successfully set credentials. Generated random user id: user_7370526
๐ฅ Run this repeatedly and watch the features update!โ
Note: It may take a few minutes for your API key permissions to update.
# Ingest a new transaction with any amount you want
record = ds.ingest({"user_id": user_id, "timestamp": datetime.utcnow(), "amount": 100})
print(f"Ingested record for '{user_id}':")
print(record)
# Read updated features via Tecton's HTTP API
features = fv.get_online_features(join_keys={"user_id": user_id}).to_dict()
print(f"\nUpdated features for '{user_id}':")
print(features)
Ingested record for 'user_7370526': {'workspaceName': 'prod', 'ingestMetrics': {'featureViewIngestMetrics': [{'featureViewName': 'user_transaction_amount_totals', 'onlineRecordIngestCount': '1', 'featureViewId': 'c3c076e11740b141097d5f864bfceb1f'}]}}
Updated features for 'user_7370526': {'amount_sum_1h_continuous': 100.0, 'amount_sum_1m_continuous': 100.0, 'amount_sum_30d_continuous': 100.0}
โญ๏ธ Conclusionโ
Now that you've seen how easy it is to build real-time features with Tecton, check out Building a Production AI Application to see how you can productionize an end-to-end online AI Application in just 30 minutes.
If you want to productionize your own features with your own data, you can sign up for a free trial at tecton.ai/free-trial.