2.1. Preparation

Qunomon uses AIT (AI Test package), an executable AI/ML evaluation method, to evaluate and assure the ML components under test according to the guideline/standards adopted by your organization.

Qunomon comes with several AITs and guideline definitions from the beginning, but you will need to install new AITs and guideline definitions according to your ML components under test and organization’s quality policies.

To install these new packages, there are two main different ways: (1) Search and install packages through AIT Hub which is hosted by the AIST for sharing AITs and guideline definitions. (2) Locally install manually created AITs and guideline definitions (actually a JSON file).

2.1.1. Install new packages through AIT Hub

2.1.1.1. Preparation: establish a connection with AIT Hub

Click the “AIT-HUB login” button on the Setting screen on your Qunomon.

2.1.1.2. Install and register a new guideline definition

You can see the list of guideline which can be installed to your Qunomon from the “Guidelines” screen. Choose guideline to be installed with radio button, and then click the “Guideline install” button.

※Guideline that have already been installed cannot be reinstalled.If necessary, click the delete icon in the “Install” column to delete the guideline, and reinstall it.

2.1.1.3. Install and register a new AIT

You can see the list of published AITs through “AITSearch” screen. You can filter the AITs to be listed by search boxes diplayed at top of the screen.

If you want to know more detailed information (such like authors, detailed description, I/O requirements…) of the AIT, you can press the icon displayed on the “Detail” column on the table to see them.

If the “×” mark are displayed on the “Installed” column, they are AITs currently not installed. Please check the detail and install them as needed. Click the icon in the “Install” column of the AIT you wish to install. This will automatically download the selected AIT from AIT Hub and registration to your Qunomon.

※AIT that have already been installed cannot be reinstalled. If necessary, please go to the Installed AITs screen, remove it, and reinstall it.

Alternatively, you can check and install popular AITs from “AITHub Ranking” screen.

2.1.2. Local Installation (without AIT Hub)

If you want install AITs and/or guideline definitions locally because you have created the new AIT, your organization adopted a private guideline/standards, etc., please read this section.

These procedure are for advanced users and may require some computer science knowledge.

2.1.2.1. Register Guideline (not using AIT-HUB)

You must analyze guideline/standards and identify quality characteristics defined in them. After that, you required to write “Guideline JSON” conform to our Guideline Schema defined as JSON Schema.

Click the “Guideline Create” button on the “Guidelines” screen.

A sample of guideline definition compliant with the schema will be displayed on the “guideline creation” screen. Modify them accordingly and click the “Register” button.

If you want modify them over a wide range, it is advised to read guideline schema carefully and edit them on the other editors. We recommend that you test your guideline JSON for schema compliance beforehand.

Note

If the JSON format is not correct, an “x” mark will appear to the left of the line number. Please correct the area around the “x” mark and register the data with all the “x” marks gone.

2.1.2.2. Register AIT (not using AIT-HUB)

Move to the “AITLocalInstall” screen. Press the “Select Upload File” and then you are choose zipped AIT to be installed. “AIT Create” button will register the AIT into your Qunomon.

To create AIT and its zipped archive, you should read AIT Development Guide as reference.

※AIT registration from local will take some time. Please wait for a while and make sure that the “Status” column becomes “Registered” before using it for the TestDescription. If the “Status” column does not change from “Processing” after a while, please reload the screen.