3.2. Experimentation within Jupyter Lab.

Using the launched Jupyter Lab environment, you can explore ideas for AI evaluation programs and conduct experiments related to AI evaluation.

Important

When considering AI evaluation, it is recommended to define specifications with future AIT implementation in mind.

Specifications to Consider for Converting AI Evaluation Programs into AITs:

  • What to evaluate?

    • For example, measure the accuracy of a classification model, use Accuracy to measure that accuracy, and use algorithm XX to calculate Accuracy.

    • At this time, referring to quality guidelines to determine which quality aspect this measurement relates to can help define the direction of quality assessment.

    • It is desirable for an AIT to measure a single quality metric.

    • The quality metric should be a specific numerical value.

  • What to input?

    • AIT can accept multiple assets (data, models, etc., that are the subjects of evaluation) and parameters (runtime settings).

  • What to output?

    • AIT can output multiple reports (metrics of evaluation results or images). Additionally, intermediate results and preliminary evaluation information generated during the evaluation can also be included in the output.

Tip

You’ll be moving the AI evaluation program you experimented with on Jupyter Lab to the “my_ait.ipynb” template starting from section 3.3 and onwards.

3.2.1. Environmental information

Development environment information:

  • Python 3.9.13

  • jupyterlab 2.2.9

  • matplotlib 3.7.3

Important

The environment information may change with updates to the AIT template. Before use, please ensure to check the latest AIT template Dockerfile.

3.2.2. Jupyter Lab Usage

Access “http://localhost:9888/” from a browser like Chrome.

Important

For the location of input/output data for Jupyter Lab notebooks and Jupyter Lab notebooks, please use the location directly under the root directory.

  • Path to the template seen from the host: qunomon/ait_repository/ait/{YOUR_AIT_ROOT}/

  • Path to the template seen from the Jupyter Lab: /root/

  • You can create and run excutable notebook like normal jupyter notebook or jupyter lab.

Create a new Python3 notebook by selecting Python3 in the Notebook category located on the launcher tab.

create notebook

Then the new notebook is opening at the right pane, and now you are able to do some experiments and implementation freely. write notebook

  • Environment & Version Information:

    • JupyterLab 2.2.9 running on Python 3.9.13 Docker image.

    • For more details, refer to {YourAITname}/tool/docker/jupyter/dockerfile.