Connect CI/CD
Overview​
We recommend creating a new CI/CD pipeline to automatically run tecton plan
and tecton apply
upon changes to your Tecton feature repo.
Example: Tecton Plan​
For example, to run a tecton plan
and tecton apply
in GitHub actions, you
can add the following file to the .github/workflows
directory underneath your
Github repository, assuming that your features repository starts at the base of
your Github repository. Note that this example uses
Github environments:
name: Tecton Feature Repo CI/CD
on:
push:
branches: [master]
pull_request:
branches: [master]
jobs:
plan:
runs-on: ubuntu-latest
outputs:
planid: ${{ steps.planid.outputs.planid }}
env:
TECTON_API_KEY: ${{ secrets.TECTON_API_KEY }}
API_SERVICE: https://app.tecton.ai/api
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: "3.9"
cache: "pip" # caching pip dependencies
- run: pip install -r requirements.txt
- name: Run tecton plan
run: tecton plan | tee output.log
- name: Extract plan output
id: planid
run: |
PLANID=$(cat output.log | perl -wne '/Generated plan ID is ([0-9a-f]+)/i and print $1')
echo "planid=$PLANID" >> "$GITHUB_OUTPUT"
apply:
runs-on: ubuntu-latest
needs: plan
environment: prod-workspace
steps:
- env:
PLANID: ${{needs.plan.outputs.planid}}
run: echo "$PLANID"
Your requirements.txt
file can look something like the below:
tecton[pyspark]~=0.7.3
To authenticate your Github Action, you'll need to create a Service Account to obtain a Tecton API key, and assign the Service Account the Editor role for the appropriate workspace:
- Create the Service Account to obtain the API key.
tecton service-account create \
--name "cicd-service-account" \
--description "A cicd example"
Output:
Save this API Key - you will not be able to get it again.
API Key: <Your-api-key>
Service Account ID: <Your-Service-Account-Id>
- Assign the Editor role for the workspace to the Service Account.
tecton access-control assign-role --role editor \
--workspace <Your-workspace> \
--service-account <Your-Service-Account-Id>
Output:
Successfully updated role.
Validate Plan with JSON output​
You can output a JSON version of a to-be-applied diff using the --json-out flag.
tecton plan --json-out <path>
This can be useful in a CI/CD pipeline to prevent applying unintended changes by running a custom script on the output.
Example json file output:
{
"objectDiffs": [
{
"transitionType": "DELETE",
"objectMetadata": {
"name": "transaction_user_has_good_credit",
"objectType": "FEATURE_VIEW",
"owner": "john@doe.com",
"description": "Whether the user had a good credit score (over 670) as of the time of a transaction."
}
},
{
"transitionType": "RECREATE",
"objectMetadata": {
"name": "continuous_feature_service",
"objectType": "FEATURE_SERVICE",
"owner": "john@doe.com",
"description": "A Feature Service providing continuous features."
}
}
]
}
See Types of Repository Changes doc to help understand the plan output.
Apply Generated Plan​
When a plan is successfully generated with tecton plan
, an ID for that plan is
printed to the console after the plan contents.
...
↑↑↑↑↑↑↑↑↑↑↑↑ Plan End ↑↑↑↑↑↑↑↑↑↑↑↑
Generated plan ID is a25e9516ebde475690ef3806e1f12e1e
If tecton plan
was run with the --json-out
flag, the plan ID is also
included as a field in the JSON file:
{
"objectDiffs": [
...
],
"planId": "a25e9516ebde475690ef3806e1f12e1e"
}
After this plan is approved through your team's workflow (whether automated or
manual), you can directly apply the plan by passing the plan ID through the
--plan-id
parameter:
tecton apply --plan-id=a25e9516ebde475690ef3806e1f12e1e
This will apply the plan directly without recomputing a new plan.
If any changes have been made to the feature repo since the plan was generated
(i.e. someone ran tecton apply
), then you will get an error and must generate
a new plan on top of the current repo state.
Protecting Critical Objects from Destruction​
Changes to Tecton objects may trigger destructive changes which can lead to
serving downtime or expensive rematerializations. You can set the
prevent_destroy
on critical objects to avoid accidentally destroying them. See
this guide for more
information about prevent_destroy
.