3.3. AIT development¶

After completing the experiment and deciding the purpose, functionality, I/O specifications of an AIT, move the part of the codes to (my_ait.ipynb) and prepare for packaging. my_ait.ipynb contains codes to ensure reusability in other environment and codes which enable interaction with Qunomon. Even if you don’t know the specification of Qunomon in detail, you can develop AIT by editing some cells in the template while following the rules.

3.3.1. Edit my_ait.ipynb¶

Open my_ait.ipynb in the AIT development environment (Jupyter Lab) and port your AI evaluation program onto this template. Please refer 3.3.3 The strcure of my_ait.ipynb for detailed procedure.

Tip

This file is mounted on the docker container, you can refer them both from host environment.

Path to the template seen from the host : {YOUR_AIT_ROOT}/develop/my_ait.ipynb
Path to the template seen from the Jupyter Lab: /root/develop/my_ait.ipynb

The rest of the section describes the detailed strcuture of ‘my_ait.ipynb’.

3.3.2. The structure of my_ait.ipynb¶

The my_ait.ipynb consists of multiple areas; and each area consists of multiple cells. AIT development involves breaking down the AI evaluation program created during the experimental phase according to areas, and then transferring it into each respective area.

Note

Some cells contain essential information and code for packaging and are not editable in Jupyter Lab. Please do not edit these cells.

Tip

Some of the areas and cell are devided for understandability reason. So even if the number of cells or contents of the cell are not completely conforming to the definition of areas, they may work as expected.

Tip

The python code on this page is identical to https://github.com/aistairc/ait-template/blob/main/develop/my_ait.ipynb.

The table below is the structure of my_ait.ipynb.

The meaning of columns:
- #: Number of the area.
- Name: Name of the area.
- cells: Number of the cells contained in that area.
- description: explains the meaning/purpose of the area.
- for_dev: If the value is True, this area executed only on the Jupyter Lab for the packaging purpose.
- edit: Whether the area required to be edited or not editable.

#	Name	cells	for_dev	edit	description
1	Environment detection	1	No	uneditable	detect whether the notebook are invoked for packaging or in production
2	Preparing AIT SDK	1	Yes	uneditable	download and install AIT SDK
3	Dependency Management	3	Yes	required(cell #2))	generate requirements.txt for Docker container
4	Importing Libraries	2	Yes	required(cell #1))	import required libraries
5	Manifest Generation	1	Yes	required	AIT Manifest definition
6	Prepare for the Input	1	Yes	required	AIT Input JSON definition(inventory mapper)
7	Initialization	1	No	uneditable	initialization for AIT execution
8	Function definitions	N	No	required	define functions invoked from Main area. also define output functions.
9	Main Algorithms	1	No	required	area for main algorithms of an AIT
10	Entry point	1	No	uneditable	an entry point where Qunomon invoke this AIT from here
11	License	1	Yes	required	generate license information
12	Deployment	1	Yes	uneditable	convert this notebook to the python file for packaging purpose

3.3.2.1. #3 Dependency Management area¶

Since an AIT will be executed on the Docker container, there is a need for prepare requirements.txt to successfully install dependencies on the container. In this area, we automatically generates a requirements.txt and place them to the appropriate locations by using AITRequirementsGenerator class in the AIT SDK. You should write down all required dependencies in this area. Please write the OSS name in the first argument and the version in the second argument to add_package.

Note

It is possible to generate requirements.txt by omitting the second argument of add_package and using only the OSS name, but this is not recommended as there is a concern that the dependencies may break down as the OSS version is updated.

Example

When your AIT depends on the pandas 2.0.3 and seaborn 0.13.2 and numpy 1.26.4:

if not is_ait_launch:
## sample ##
    requirements_generator.add_package('pandas', '2.0.3')
    requirements_generator.add_package('seaborn','0.13.2')
    requirements_generator.add_package('numpy','1.26.4')

requirements_generator is an instance of the AITRequirementsGenerator class. For detailed information about AITRequirementsGenerator, please see here.

Important

The AIT SDK itself are automatically added to the requirements.txt. Don’t explicitly add them in this area.

Tip

“is_ait_launch” is False when experimenting on Jupyter Lab. Commands that you want to execute only in Jupyter Lab experiments (AITRequirementsGenerator etc) should be enclosed in if not is_ait_launch:.

3.3.2.2. #4 Importing Libraries area¶

Import classes, functions and/or other entities from libraries used in your AIT here.

3.3.2.3. #5 Manifest Generation area¶

AIT Manifest contains metadata such as name, author, description and/or I/O definitions of an AIT for ensure reusability and searchability of it. In this area, we automatically generates ait.manifest.json file and place them to the appropriate locations by using AITManifestGenerator class in the AIT SDK.

#	Method name	Details	Possibility of multiple definitions	Submethod	Code sample
1	set_ait_name	Set AIT name	×	None	`manifest_genenerator.set_ait_name('Name')`
2	set_ait_description	Set AIT description	×	None	`manifest_genenerator.set_ait_description('Description')`
3	set_ait_source_repository	Set AIT source repository URL	×	None	`manifest_genenerator.set_ait_source_repository('URL')`
4	set_ait_version	Set AIT version	×	None	`manifest_genenerator.set_ait_version('Version')`
5	set_ait_quality	Set quality URL for AIT	×	None	`manifest_genenerator.set_ait_quality('Quality_URL')`
6	add_ait_keywords	Add AIT keyword	〇	None	`manifest_genenerator.add_ait_keywords('Keyword_1')` `manifest_genenerator.add_ait_keywords('Keyword_2')`
7	add_ait_references	Add AIT reference	〇	None	# Pattern that sets all arguments `manifest_genenerator.add_ait_references(bib_info='bib_info_1', additional_info='additional_info_1',url='URL_1')` # Pattern that only sets required arguments `manifest_genenerator.add_ait_references(bib_info='bib_info_2')`
8	add_ait_licenses	Add AIT license	〇	None	`manifest_genenerator.add_ait_licenses('License_1')` `manifest_genenerator.add_ait_licenses('License_2')`
9	add_ait_inventories	Add AIT inventory * Requirement variable must be generated using the format_ait_inventory_requirement submethod in advance	〇	- format_ait_inventory_requirement	# Pattern that sets all arguments `manifest_genenerator.add_ait_inventories(name='Name_1', type_='Type_1', description='Description_1', requirement=inventory_requirement_1, depends_on_parameter='Parameter_1')` # Pattern that only sets required arguments `manifest_genenerator.add_ait_inventories(name='Name_2', type_='Type_2', description='Description_2', requirement=inventory_requirement_2)`
10	- format_ait_inventory_requirement	Format inventory requirement items	〇	- create_inv_req_compatible_packages - create_inv_req_additional_info	# Pattern that sets all arguments `inventory_requirement_1 = manifest_genenerator.format_ait_inventory_requirement(format_=['Format_1', 'Format_2'], compatible_packages=compatible_package_1, additional_info=additional_info_1, min='min_1', max='max_1')` # Pattern that only sets required arguments `inventory_requirement_2 = manifest_genenerator.format_ait_inventory_requirement(format_=['Format_1'])`
11	-- create_inv_req_compatible_packages	Create compatible_packages item for inventory requirements	〇	None	# Pattern that sets all arguments `compatible_package_1 = manifest_genenerator.create_inv_req_compatible_packages(name='Name_1', version='Version_1', additional_info='additional_info_1')` # Pattern that only sets required arguments `compatible_package_2 = manifest_genenerator.create_inv_req_compatible_packages(name='Name_2')`
12	-- create_inv_req_additional_info	Create additional_info item for inventory requirement	〇	None	`inv_req_additional_info_1 = manifest_genenerator.create_inv_req_additional_info(key='Key_1', value=['Value_1', 'Value_2'])`
13	add_ait_parameters	Add AIT parameter	〇	None	# Pattern that sets all arguments `manifest_genenerator.add_ait_parameters(name='Name_1', type_='Type_1', description='Description_1', default_val='Default_val_1', min_value='Min_value_1', max_value='Max_value_1', depends_on_parameter='parameter_1')` # Pattern that only sets required arguments `manifest_genenerator.add_ait_parameters(name='Name_2', type_='Type_2', description='Description_2')`
14	add_ait_measures	Add AIT measure	〇	None	# Pattern that sets all arguments `manifest_genenerator.add_ait_measures(name='Name_1', type_='Type_1', description='Description_1', structure='Structure_1', min='Min_1', max='Max_1')` # Pattern that only sets required arguments `manifest_genenerator.add_ait_measures(name='Name_2', type_='Type_2', description='Description_2', structure='Structure_2')`
15	add_ait_resources	Add AIT resource	〇	None	`manifest_genenerator.add_ait_resources(name='Name_1', type_='Type_1', description='Description_1')` `manifest_genenerator.add_ait_resources(name='Name_2', type_='Type_2', description='Description_2')`
16	add_ait_downloads	Add AIT download	〇	None	`manifest_genenerator.add_ait_downloads(name='Name_1', description='Description_1')` `manifest_genenerator.add_ait_downloads(name='Name_2', description='Description_2')`

Example

if not is_ait_launch:
## sample ##
    from ait_sdk.common.files.ait_manifest_generator import AITManifestGenerator
    manifest_genenerator = AITManifestGenerator(current_dir)
    manifest_genenerator.set_ait_name('dev_ait_template_for_aithub')
    manifest_genenerator.set_ait_description('アヤメの分類精度(sample)')
    manifest_genenerator.set_ait_source_repository('https://github.com/aistairc/ait-template')
    manifest_genenerator.set_ait_version('0.1')
    manifest_genenerator.add_ait_licenses('Apache License Version 2.0')
    manifest_genenerator.add_ait_keywords('Template')
    manifest_genenerator.set_ait_quality('https://ait-hub.pj.aist.go.jp/ait-hub/api/0.0.1/qualityDimensions/機械学習品質マネジメントガイドライン第三版/A-2データ設計の十分性')
    inventory_requirement_iris_data = manifest_genenerator.format_ait_inventory_requirement(format_=['csv'])
    manifest_genenerator.add_ait_inventories(name='iris_data', 
                                            type_='dataset', 
                                            description='アヤメの分類データです', 
                                            requirement=inventory_requirement_iris_data)
    manifest_genenerator.add_ait_parameters(name='mean_column_name', 
                                            type_='str', 
                                            description='sepal.width\nsepal.length\npetal.width\npetal.length', 
                                            default_val='sepal.width')
    manifest_genenerator.add_ait_measures(name='mean', 
                                        type_='float', 
                                        description='mean of select column', 
                                        structure='single',
                                        min='0')
    manifest_genenerator.add_ait_resources(name='pairplot', 
                                            type_='picture', 
                                            description='pairplot')
    manifest_genenerator.add_ait_downloads(name='Log', 
                                            description='AIT実行ログ')
    manifest_path = manifest_genenerator.write()

AITManifestGenerator can validate the format of attributes and check whether required attributes are set. So if there’re missing or wrong attributes in this area, execution of my_ait.ipynb may fail at that time. For detailed information about AITManifestGenerator class, please see the document AIT_SDK_API Specification.

Tip

The defined AIT manifest is saved as ait.manifest.json.

Tip

Please refer to my_ait.ipynb and ait.manifest.json in “1.3. Published AIT” for registration information of AIT manifest. Because it depends on the characteristics and purpose of the AI evaluation program.

3.3.2.4. #6 Prepare for the Input area¶

Define the input data (data to be evaluated by the AIT) and parameters (settings for executing the AIT).

Example of input data settings

    if not is_ait_launch:
    ## sample ##
        from ait_sdk.common.files.ait_input_generator import AITInputGenerator
        input_generator = AITInputGenerator(manifest_path)
        input_generator.add_ait_inventories(name='iris_data',
                                            value='iris_data/tableconvert_csv_4nryby.csv')
        input_generator.set_ait_params(name='mean_column_name',
                                        value='petal.width')
        input_generator.write()

AITInputGeneratorFor details about the class, please see here.

The name argument specified to the function is linked to the inventory parameter name in the AIT manifest generated in #5 Manifest Generation Area.
- The name argument of the add_ait_inventories function should be the same as the inventories/name attribute in the AIT manifest.
- The name argument of the set_ait_params function should be the same as the parameters/name attribute in the AIT manifest.
- Set the value argument of the add_ait_inventories and set_ait_params functions by referring to the Unit Testing chapter.

3.3.2.5. #8 Functions definitions area¶

In Function definitions area, you can define any number of user defined functions which is invoked from the following main algorithms area (#9). Of the user defined functions, the functions that output the execution results of AIT are annotated and defined as result output functions.

Tip

The output value of the result output functions is automatically associated with the corresponding report download name in the AIT manifest generated in #5 Manifest Generation area.

The format of the user-defined function is arbitrary. There are three types of output functions:

Report/Measures (Output evaluation/analysis results as measured value)
Report/Resources (Output evaluation/analysis results as image, graphs or specified file format)
Download (Output supplemental and/or intermediate results as image, graphs or specified file format)

Each result output function is described in detail below.

3.3.2.5.1. result output functions: Report/Measures¶

You can define the execution result of the user-defined function as the AIT measurement value. You must add following two annotations to your output function.

@log(logger) annotation
@measures(ait_output, '{NAME_OF_REPORT/MEASURE}') annotation
- {NAME_OF_REPORT/MEASURE} must be matched to the one of the name in the report/measures defined in the AIT Manifest.

Important

The return value of this function will be treated as an evaluation value of {NAME_OF_REPORT/MEASURE}. So the function must return a value.

When your AIT output mean value as a result:

## sample ##
@log(logger)
@measures(ait_output, 'mean')
def calc_mean(iris_data, col_name):
    return iris_data[col_name].mean()

In the #5 Manifest Generation area, predefine the AIT manifest(report/measures) corresponding to the annotation.

    manifest_genenerator.add_ait_measures(name='mean', 
                                        type_='float', 
                                        description='mean of select column', 
                                        structure='single',
                                        min='0')

3.3.2.5.2. result output functions: Report/Resources¶

You can define an output resource (e.g. image file) generated by a user-defined function as an AIT resource. you must add following two annotations to your output function.

@log(logger) annotation
@resources(ait_output, path_helper, {NAME_OF_REPORT/RESOURCES}, {FILE_NAME}) annotation
- {NAME_OF_REPORT/RESOURCES} must be matched to the one of the name in the report/resources defined in the AIT Manifest.
- {FILE_NAME} specifies the file name of the AIT resource.

Important

This function requires a file_path argument because the resource file path is automatically specified in the file_path argument of the result output function.

When your AIT outputs pairplot image:

## sample ##
@log(logger)
@resources(ait_output, path_helper, 'pairplot', 'pairplot.png')
def save_pair_plot(iris_data, file_path: str=None) -> str:    
    sn.pairplot(iris_data, hue='variety')
    plt.savefig(file_path)

In the #5 Manifest Generation area, predefine the AIT manifest(report/resources) corresponding to the annotation.

    manifest_genenerator.add_ait_resources(name='pairplot', 
                                        type_='picture', 
                                        description='pairplot')

3.3.2.5.3. result output functions: downloads¶

In addition to user-defined function resources and measures, you can define user-defined function supplemental and/or results as download files. you must add following two annotations to your output function.

@log(logger) annotation
@downloads(ait_output, path_helper, {NAME_OF_DOWNLOADS}, {FILE_NAME}) annotation
- {NAME_OF_DOWNLOADS} must be matched to the one of the name in the downloads defined in the AIT Manifest.
- {FILE_NAME} specifies the file name of the download file.

Important

This function requires a file_path argument because the resource file path is automatically specified in the file_path argument of the result output function.

Example of function definition when outputting the execution log of the AIT as a reference.

## sample ##
@log(logger)
@downloads(ait_output, path_helper, 'Log', 'ait.log')
def move_log(file_path: str=None) -> str:
    shutil.move(get_log_path(), file_path)

In the #5 Manifest Generation area, predefine the AIT manifest(downloads) corresponding to the annotation.

    manifest_genenerator.add_ait_downloads(name='Log', 
                                            description='AIT実行ログ')

3.3.2.6. #9 Main Algorithms area¶

In this area, you should write the main algorithms of the AIT while reffering I/O and/or functions defined before this area. This area has some rules to follow.

you must add following two annotations to your output function.
- @log(logger) annotation
- @ait_main(ait_output, path_helper, is_ait_launch) annotation
The name of the function must be main()
Paremetes must be got from the get_method_param_value method of ait_input variable (AITInput class).
Path to the inventory mus be got from the get_inventory_path method ait_input variable (AITInput class).

Example of the main algorithms area

@log(logger)
@ait_main(ait_output, path_helper, is_ait_launch)
def main() -> None:
    pass
## sample ##
    # インベントリを読み込み
#     iris_data = pd.read_csv(ait_input.get_inventory_path('iris_data'))
#     calc_mean(iris_data, ait_input.get_method_param_value('mean_column_name'))
#     save_pair_plot(iris_data)
#     move_log()

Important

In the example of the main algorithms area, the actual processing of the main function is commented out.

If you want to try it uncommented out, create a iris_data directory under /root/local_qai/inventory and create `` Create tableconvert_csv_4nryby.csv under /root/local_qai/inventory.

tableconvert_csv_4nryby.csv should follow the following format.

"sepal.length","sepal.width","petal.length","petal.width","variety"  
1.0,1.1,1.2,1.3,"Setosa"  
2.0,2.1,2.2,2.3,"Versicolor"  
3.0,3.1,3.2,3.3,"Virginica"  
.......  
.......  
.......

3.3.2.7. #11 License area¶

Some attributes must be set for automatic generation of your AIT license information. Currently you must set the value to the ait_owner attribute and the ait_creation_year attribute.

Important

The AIT template applies the Apache 2.0 license by default. While Apache 2.0 is recommended, modify or replace the LICENSE.txt as necessary.

3.3.3. Check program execution¶

Execute the my_ait.ipynb and check that the ported program works properly.

The Output stored in /root/top_dir/local_qai/mnt/ip/job_result/1/1. AIT manifest stored in /root/develop/ait.manifest.json