Connect Databricks Notebooks
You can use the Tecton SDK in a Databricks notebook to explore feature values and create training datasets. The following guide covers how to configure your all-purpose cluster for use with Tecton. If you haven't already completed your deployment of Tecton with Databricks, please see the guide for Configuring Databricks.
Supported Databricks Runtimes for Notebooks​
See this page for the list of supported Databricks Runtimes in Tecton.
Note that when using Databricks Runtime 9.1 LTS & 10.4 LTS, Tecton only supports Service Account credentials. When using Databricks Runtime 11.3 LTS or above, Tecton supports both Service Account credentials and User credentials.
As a best practice, use the same version for your Notebook Cluster as is configured for your Feature View materialization.
Install the Tecton SDK​
This step must be done once per notebook cluster.
On the cluster configuration page:
- Go to the Libraries tab
- Click Install New
- Select PyPI under Library Source
- Set Package to your desired Tecton SDK version, such as
tecton==0.8.0
ortecton==0.8.*
.
Authenticate to Tecton Account​
Authenticating to a Tecton instance from a notebook can happen in 3 ways. They are listed here in the order that Tecton searches for credentials to use. For example, credentials set using Option 1 will override any credentials set in Options 2 and 3.
Option 1: User Credentials in Notebook Session Scope​
User credentials configured using tecton.login()
are scoped to the notebook
session, and must be reconfigured when a notebook is restarted or its state is
cleared. User credentials override any credentials set in both
Option 2: Service Account Credentials in Notebook Session Scope
and
Option 3: Service Account Credentials in Databricks Workspace Scope.
tecton.login(interactive=True)
requires the cluster to be on Databricks
Runtime 11.3 or higher.
To authenticate as a user, run the following in your notebook, replacing
"https://example.tecton.ai"
with the URL of your Tecton instance:
tecton.login("https://example.tecton.ai")
Then follow the directions to open the login link in your browser, sign in to the Tecton instance as your user, and copy and paste the authorization code from the Identity Verified web page back into your notebook's input box. Please be aware the authorization code is one-time use only.
Option 2: Service Account Credentials in Notebook Session Scope​
Service account credentials configured using tecton.set_credentials()
are
scoped to the notebook session. They must be reconfigured whenever a
notebook is restarted or its state is cleared. They override credentials set in
Option 3: Service Account Credentials in Databricks Workspace Scope.
Prerequisites​
Please have a Tecton Service Account already set up (and have its API Key secret value accessible). If you don't have one, create a new one using these instructions.