What is a source ?
Data sources are data stored in your organization, it's an endpoint and the central elements of QALITA Platform. They are referenced in the platform, and the original data is never stored in the platform.
Reference a Source
Referencing a source is a way to tell Platform that you have data accessible at this endpoint, it can be a file path, or database credentials.
You can reference a source two ways :
- You can do it using the QALITA CLI source add command.
- Your can do it using the QALITA CLI UI
When you reference a source, QALITA CLI creates a configuration file sources-conf.yaml in the ~/.qalita/ directory, which contains the connection information for the source.
version: 1
sources:
- config:
path: ~/desktop/data/heart/
description: 11 clinical features for predicting heart disease events.
id: 1
name: Heart Failure Prediction Dataset
reference: true
sensitive: true
type: file
validate: valid
visibility: internal
In this example, the source is a local file, but it could also be a database, a remote file, a data stream, etc.
| Property | Description |
|---|---|
version | Configuration version |
sources | List of sources |
config | Source configuration, properties vary depending on the source type. |
id | Unique source identifier, this ID is obtained from the platform after finalizing the source referencing with the qalita source push command. |
name | Source name |
type | Source type |
reference | If the source is a reference, it allows comparison with other sources. |
sensitive | If the source is sensitive, special permission is required to access it see permissions |
validate | If the source is valid, it means the client has proper access to the source, and it can be referenced on the platform. |
visibility | Source visibility, can be public, internal, or private. Allows restricting access to different user profiles on the platform. |
Source Types
Sources can be of any type, as long as there are packs to process them. QALITA Platform imposes no limitations on source types.