# Centralized Suites

Centralized Suites are a way to centrally and modularly define groupings of
tests that together capture a certification goal when run against a device (e.g.
FSI, CQ, Release). Centralized Suites consist of:

* **Suites** - which contain tests
* **SuiteSets** - which contain other Suites and SuiteSets

This README details the process of authoring Suites and SuiteSets. For more
details on the motivation and design behind Centralized Suites see the design
doc at [go/cros-centralized-suites].

## Authoring Centralized Suites

Centralized Suites are defined in Starlark across two repositories: [config] and
[config-internal]. Only Suites with private Tests and SuiteSets with private
Suites/SuiteSets should live in the private config-internal repo, the rest
should live in the public config repo. See [Centralized Suite Build Process] for
more details on how Centralized Suites interact across these two repos.

To define a new Suite/SuiteSet in either of these repos, do the following:

1. Create a owners specific subdirectory

    * Create a subdirectory in the [suite_sets] directory and add an `OWNERS`
    file to it with individuals/teams that should sign off on changes to the
    Suite/SuiteSet

        * If a subdirectory with the same owners already exists, use that
        subdirectory instead of creating a new one

1. Define your Suite/SuiteSet

    * In your specific subdirectory, define your Suite/SuiteSet as demonstrated
    in [example_suites.star] and [example_suite_sets.star]

1. Add Suite/SuiteSet to compilation list

    * If you defined a Suite, import and add it `compiled_suites` list in the
    [compiled_suites.star] file

    * If you defined a SuiteSet, import and add it to the `compiled_suite_sets`
    list in the [compiled_suite_sets.star] file

1. Validate your Suite/SuiteSet

    * Run [generate.sh] script

        ```
        ./generate.sh
        ```

        * Fix any linting errors it returns

        * Validate your Suite/SuiteSet is included in the generated proto file

1. Upload a CL with your changes
    * See this [reference CL] for an example of what a change might look like

## Migrating from Legacy Suites

If you want to migrate a pre-existing legacy suite (where the suites are
attributes in the tests themselves) to a Centralized Suite, you can use the
[suite_migration.py script] as follows:

1. Complete step 1 of [Authoring Centralized Suites]

1. Build the test metadata so it is up-to-date
    ```
    cros_sdk '$CROS_WORKON_SRCROOT'/src/third_party/autotest/files/contrib/suite_migration.py --update-metadata
    ```

1. Migrate the suite of interest
    ```
    cros_sdk '$CROS_WORKON_SRCROOT'/src/third_party/autotest/files/contrib/suite_migration.py --suite <suite_name> --output <output_starlark_path>
    ```

1. Fill out missing metadata

    * The starlark file generated by the script will have TODO messages for all
    the missing metadata; fill in the relevant information

1. Complete steps 3-5 of [Authoring Centralized Suites]


## Centralized Suite Build Process

As of January 30, 2023, Centralized Suites are built per board by combining the
information across the [config] and [config-internal] repos. Specifically, the
logic is as follows:

1. Load the Centralized Suites from the [config] repo

1. If the [config-internal] repo is present (will not be  available to public
builders) do:

    1. Load the Centralized Suites from the [config-internal] repo

    1. Merge Centralized Suites with the same name:

        * Suites will be merged by unioning of the test lists

        * SuiteSets will be merged by unioning the Suite and SuiteSet lists

1. Filter tests from Suites that are not relevant to the Board

<!-- Insert links below -->
[go/cros-centralized-suites]: https://docs.google.com/document/d/1Qr7oW2kCeCsxsyu2vWBSXjwCHlhMEaSWSme7zoALnWc/edit?resourcekey=0-U91BRPcjbxKGdR_TpJsHBA&tab=t.0#heading=h.7zgnj8bwqfld
[config]: https://chromium.googlesource.com/chromiumos/config/+/refs/heads/main/test/suite_sets/
[config-internal]: https://chrome-internal.googlesource.com/chromeos/config-internal/+/refs/heads/main/test/suite_sets/
[example_suites.star]: https://chromium.googlesource.com/chromiumos/config/+/refs/heads/main/test/suite_sets/suite_sets/example/example_suites.star
[example_suite_sets.star]: https://chromium.googlesource.com/chromiumos/config/+/refs/heads/main/test/suite_sets/suite_sets/example/example_suite_sets.star
[reference CL]: https://chromium-review.googlesource.com/c/chromiumos/config/+/5171723
[Centralized Suite Build Process]: #centralized-suite-build-process
[suite_migration.py script]: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/refs/heads/main/contrib/suite_migration.py
[Authoring Centralized Suites]: #authoring-centralized-suites
[test/suite_sets/suite_sets]: suite_sets
[compiled_suites.star]: compiled_suites.star
[compiled_suite_sets.star]: compiled_suite_sets.star
[generate.sh]: generate.sh