How to 'Pack' ?

A pack is an entity run by the agent, it can be created by anyone.

It's purpose is to process the source and retrieve usefull informations about it to feed back into the platform.

Init

To create the base pack, see : qalita pack init.

At runtime

The entrypoint of the pack is the run.sh file that is located at the root path of the temp local folder created by the agent.

run.sh Example :

#/bin/bash
python -m pip install --quiet -r requirements.txt
python main.py

The pack is feed by a source_conf.json file containing the source's config: data. This file is located alongside the run.sh entrypoint.

source_conf.json Example :

{
  "config": {
    "path": "/home/lucas/desktop"
  },
  "description": "Desktop files",
  "id": 1,
  "name": "local_data",
  "owner": "lucas",
  "type": "file",
  "reference": false,
  "sensitive": false,
  "visibility": "private",
  "validate": "valid"
}

Note : The pack is responsible for managing itself it's source type compatibility by checking the source type in the source_conf.json file.

Post runtime

A the end of the pack run, the agent searchs for :

• logs.txt : File uploaded to give feedback logs to the platform in the frontend.

logs.txt Example :

2023-07-21 11:51:12,688 - qalita.commands.pack - INFO - ------------- Pack Run -------------
2023-07-21 11:51:15,087 - qalita.commands.pack - INFO - CSV files found:
2023-07-21 11:51:15,222 - qalita.commands.pack - ERROR - Summarize dataset:   0%|          | 0/5 [00:00<?, ?it/s]
...

• recommendations.json

Recommendations file contains the recommendations given by the pack about the source.

recommendations.json Example :

{
    [
        {
            "type":"<type of recommendation>",
            "scope":"<scope : is a json>",
            "content":"<any content>"
        },
        {
            ...
        }
        ...
    ]
}

• metrics.json

Metrics file contains the metrics given by the pack about the source.

metrics.json Example :

{
    [
        {
            "scope":"<scope : is a json>",
            "key":"<metric key>",
            "value":"<metric value>"
        },
        {
            ...
        }
        ...
    ]
}

Metrics & recommendations are pushed to the platform and are then available to the source's pack run view.