Introduction to the toolkit

The Toolkit lets you control the composition of Cognite Data Fusion capabilities as code for your projects.

It enables you to roll out changes in a reliable and controlled manner and to use pre-built configurations to deliver faster.

Cognite Data Fusion has powerful features that need to be configured to load, process and bring data to users. By configuration we mean both the setup of each individual feature, resource, but also how they are logically grouped to solve specific tasks. For example, a "data pipeline" is typically a logically connected set of resources like an Extractor, Extraction Pipeline, Raw table, Transformations and a Data model. Resources that belong together should be configured and deployed together, and naming standards should make the logical grouping clear.

The CDF Toolkit is a command-line tool (cdf-tk) used for safely automating this process. It comes with pre-built templates that can get you started on your Cognite Data Fusion project quickly.

It has three main roles:

1. An interactive command-line tool used alongside the Cognite Data Fusion web application to retrieve and push configuration of the different Cognite Data Fusion services like data sets, data models, transformations, and more. If you want to make sure that you can always recreate the state of a Cognite Data Fusion project, you can use the tool interactively like this. This is also the typical way to use the tool for development and iterations on a development project.
2. Project life-cycle by scripting and automating configuration and management (CI/CD) of Cognite Data Fusion projects where CDF configurations are kept as yaml-files that can be checked into version control. Using the tool this way also allows you to do DevOps workflows with development, staging, and production projects.
3. A tool to deploy official Cognite project templates to your Cognite Data Fusion project. The tool comes bundled with templates useful for getting started with Cognite Data Fusion, as well as for specific use cases delivered by Cognite or its partners. You can also create your own templates and share them.

Install the CDF Toolkit

The tool is available as a Python package. To install the cdf-tk tool, you need a working Python installation >=3.9 (recommended 3.11). Run the command pip install cognite-toolkit in your terminal (or use a package manager like poetry).

You can then run cdf-tk --help to see the available commands. Each of the commands has comprehensive help accessible using the --help option, e.g. cdf-tk init --help.