MAIA Agents (MA) are usually installed on systems separated from the core web app. Part of the installation and operation is managing environment and configuration files and data.
Defines the values of the variables used by MAIA agent.
The environment variable BLT_ENV defines the path to the environment file (*.yml), and is set in the command shell that executes the MAIA Agent commands.
Defines the gems used by MAIA Agent.
The environment variable BUNDLE_GEMFILE defines the path to the Gemfile, and is set in the command shell that executes the MAIA Agent commands.
Defines the behavior of a component when MAIA Agent performs commands on the component. The file must be present in the root directory of each checked out component. The file can be:
The name of the file (*.yml) is specified by the parameter COMPONENT_CONF_FILE in the environment file. The default name of the file is MAIA.yml.
Defines the names of the repositories that are monitored by a Git scanner.
The repository file contains plain text with each repository name on separate lines. The name of the file is user defined. The repository file is used when scanning multiple repositories.
If a single repository is scanned the name of the repository is specified from the command line.
The environment file (environment.yml ) defines the information needed for MAIA Agent (MA) commands associated with scan, build, deploy and test.
The file must be stored in the directory where MAIA Agent is installed.
This chapter contains examples of the environment file and parameter descriptions.
Open the directory where MA is installed.
Create the environment file for MA, environment.yml
touch environment.ymlOpen the environment file for editing.
Add parameters to the file (see parameters and example below).
Export the variables in the environment file to BLT_ENV.
export BLT_ENV=*path_to_maia_agent*/environment.ymlThe column headed M/C/O in the table refers to whether the parameter/attribute is mandatory, conditional or optional. The following notation is used:
| Meaning | |
|---|---|
| M | Mandatory – always present. |
| C | Conditional – present under certain conditions (as specified in the description). |
| O | Optional – can be present. |
MAIA Agent can be used scanning (SCAN) or/and Build/Deploy/Test (BDT). The activity controls the usage of parameters as indicated in the parameter list below.
| Parameter | M/C/O | Description |
|---|---|---|
| AFTER_CHECKOUT | O (BDT) | The commands performed on a component directly after the command maia checkout. If a suite component is checked out together with its descendants, any commands for the suite are run before the descendants are checked out. The AFTER_CHECKOUT parameter can contain one or more mappings between a component and a set of commands. Each mapping is a key-value pair with the syntax component_name: set_of_commands |
| BUILD_ROOT | M (BDT) | Path to the directory where finished builds are stored. Used by command maia build. |
| COMPONENT_CONF_FILE | M (BDT) | Name of the component configuration file located at the root of each component. The file has .yml-format. The default name of the file is MAIA.yml |
| COMPONENT_STATE_FILE | M (BDT) | Name of the file that stores states for a component. The file is used by MAIA agent to correctly perform a number of commands. The file is created by the command maia checkout and is updated by the commands maia build and maia fetch. |
| DELIVERABLE_ROOT | O (BDT) | The path to the temporary directory used by maia fetch. |
| ENVIRONMENT_NAME | O | Specify name to be used for build or test environment. Default value is user@hostname |
| ENVIRONMENT_TAGS | O | Array of strings to identity the build or test environment. Default value is [user, hostname] |
| GIT_SCAN_TRACKED_FILE | M (SCAN) | The file used to store a list of commits that are scanned. The file is created and updated by the command maia git_scan. |
| KAFKA_BROKER | M | A list of the hosts that Kafka sends events to. A broker will be created for each host defined. |
| KAFKA_TOPIC | M | A list of the topics to that a consumer subscribes to. The topics defined in MAIA Agent (producer) must correspond the topics in MAIA WebApp (consumer). Events with topics not defined in MAIA WebApp will not be received (consumed). |
| LOG_ROOT | M | Path to the directory where logs for builds and tests are stored. The path is used by the commands maia build and maia test. |
| MAIA_CORE_HOST | M | Address to the host where MAIA WebApp is installed. |
| MAIA_CORE_HTTP_PORT | M | Port number used to access MAIA WebApp. |
| MAIA_CORE_APP_CONTEXT | M | A part of the MAIA application URL. The full MAIA application URL is: https://MAIA_CORE_HOST:MAIA_CORE_HTTP_PORT/MAIA_CORE_APP_CONTEXT |
| MAIA_CORE_API_TOKEN_FILE | M | The local path/name to the file where the token is stored. |
| MAIA_CORE_USE_SSL | M | Defines whether SSL is used to access the MAIA WebApp. The parameter can be true or false. |
| REPOSITORIES | M | The mapping between repository names and their source code. The REPOSITORIES parameter can contain one or more mappings to either Git or Subversion repositories. Each mapping is a key-value pair with the syntax repository_name:repository_adress |
| SRC_ROOT | C (BDT) | Path to the directory that all components are checked out to. Used when MAIA Agent is controlling the cloning/updating of repositories. |
SRC_ROOT: /opt/examples/packages
BUILD_ROOT: /opt/examples/builds
DELIVERABLE_ROOT: /opt/examples/deliverables
LOG_ROOT: /opt/examples/logs
MAIA_CORE_HOST: host.example.com
MAIA_CORE_HTTP_PORT: 7777
MAIA_CORE_APP_CONTEXT: /maia
MAIA_CORE_API_TOKEN_FILE: /home/username/.maia-api-token
MAIA_CORE_USE_SSL: true
COMPONENT_CONF_FILE: MAIA.yml
COMPONENT_STATE_FILE: .maia_state.yml
GIT_SCAN_TRACKED_FILE: tracked_commits
REPOSITORIES:
ir: https://cm.example.com/svn/ir
dev: https://cm.example.com/svn/dev
manuals: https://cm.example.com/svn/manuals
(.+): ssh://user@192.168.1.199:29418/$1
AFTER_CHECKOUT:
maia_base: 'setup_suite maia_base maia.FC7'
maia_platform: 'echo maia_platform'
'maia_.*': 'setup_component maia_base --build-makeflags="ENCODE_RUBY=no"'
'.*': 'setup_component maia_base --build-makeflags="ENCODE_RUBY=no"'
KAFKA_BROKERS:
- stream.example.com:9092
KAFKA_TOPIC: MAIA
ENVIRONMENT_NAME: centos7-host3
ENVIRONMENT_TAGS:
- centos7.1
- maia
- production
The Component Configuration File is a YAML file with erb-syntax where you configure specific instructions for MAIA Agent. It defines the steps needed to perform and/or send information about checkout, build, fetch, test and deploy a specific component and track to MAIA web app.
The file must be stored at the root of your repository. If more than one track is built, a new configuration file must be checked out before the track is built.
The default name name of the file is MAIA.yml after installation, but can be redefined by editing the parameter COMPONENT_CONFIG_FILE in the environment file.
The column headed M/C/O in the table refers to whether the parameter/attribute is mandatory, conditional or optional. The following notation is used:
| Meaning | |
|---|---|
| M | Mandatory – always present. |
| C | Conditional – present under certain conditions (as specified in the description). |
| O | Optional – can be present. |
MAIA Agent is used for scanning (SCAN) or/and Build/Deploy/Test (BDT), which activity is indicated in the parameter list below.
| Parameter | M/C/O | Description | |
|---|---|---|---|
| version | M | The protocol version of the component configuration file. The protocol version must be 5.0. | |
| component_name | O | The name of the component in MAIA application. | |
| rev | M | The revision of the component. The revision can be a string or a call to a shell script. | |
| subcomponents | C | The list of all the sub components the component has in MAIA application, if the component is a collection. The sub component can contain collections in any number of levels. | |
| label | C | The label of a previously built sub component. If the label is defined, the same code that was used in the previous build will be checked out again to be built. The label cannot be used together with the parameter revision or the parameter track. | |
| name | M | The name of a sub component. | |
| rebuild | O | Defines whether the sub component, together with any sub component, will be checked out and rebuilt. If true the sub component, together with any sub component, will be checked out and rebuilt. (Default) If false, the sub component will not be checked out, but its sub component will be checked out. The sub component will not be rebuilt. | |
| revision | C | The client defined revision of a previously built sub component. If the revision is defined, the same code that was used in the previous build will be checked out again to be built. The revision cannot be used together with the parameter label or the parameter track. | |
| track | C | The name of a track if the last commit of a sub component is fetched. The track cannot be used together with the parameter label or the parameter revision. | |
| test | O | Defined whether tests on sub components should be run. Test is set to true by default. | |
| clean | O | Section with instructions for the build clean phase. | |
| cmd | O | The commands that will be executed during the build clean phase. | |
| prepare | O | Section with instructions for the build preparation phase. | |
| cmd | O | The commands that will be executed during the build preparation phase. | |
| build | O | Section with instructions for the build phase. | |
| cmd | O | The commands that will be executed during the build phase. | |
| dependency_generator | O | The instruction for how to collect information about external software dependencies during the build phase. See information below. | |
| fetch | O | Section with instructions for the fetch phase. | |
| cmd | O | The commands that will be executed during the build fetch phase. | |
| deploy | O | Section with instructions for the deploy phase. | |
| cmd | O | The commands that will be executed during the deploy phase. | |
| deploy_path_generator | O | The scripts called in the parameter cmd. The scripts generate the names and the paths to the deliverables specified in the section deliverables, and post the information to MAIA WebAPP. If the section deliverables is missing, the information will not be posted to MAIA WebApp. See information below. | |
| test | O | Section with instructions for the test phase. | |
| cmd | O | The commands that will be executed during the test phase. | |
| prepare_cmd | O | Defines the commands that will be used to prepare for a test. | |
| suites | O | A list of of tests suites that will be run. | |
| deliverables | O | Section with instructions for which deliverables to include in the build. The deliverables can be built or fetched. | |
| name | O | The name of a deliverable. The name is the same as the deliverable name generated when running the instructions in parameter deploy_path_generator. | |
| unpack | O | The commands used to unpack the deliverable during the fetch stage. |
references one ore more scripts that fetches external software for the build. The script must print to standard output (stdout) and use CycloneDX json format.
Example:
{
"bomFormat": "CycloneDX",
"specVersion": "1.3",
"version": 1,
"components": [
{
"type": "library",
"name": "libxml2",
"version": "2.9.9",
"purl": "pkg:generic/libxml2@2.9.9",
"externalReferences": [
{
"type": "website",
"url": "http://xmlsoft.org/sources/libxml2-2.9.9.tar.gz"
}
],
"licenses": [
{
"license": {
"name": "MIT",
}
}
}
]
}
]
}must reference a user defined script that returns the deliverables on a build that:
Example of script printed to standard output.
Image http:/mynexus.com/repositories/r-abc133/base.iso
Signature http:/mynexus.com/repositories/r-abc133/base.iso.sigThere are a number of predefined variables that can be used within the <%%> signs in the Component configuration file:
Example :
| Parameter | Description |
|---|---|
| component_src_root | The directory that the source files of the component are checked out to. The component_src_root is the default working directory when executing commands. The default value is defined by the expression 'SRC_ROOT/ |
| top_src_root | The directory where source files of the top component are checked out. |
| build_root | The directory that stores the result of a build. It is defined by the variable BUILD_ROOT in the environment file. |
| log_root | The directory that temporary files, log files, debug files and similar files are written to. It is defined by the variable LOG_ROOT in the environment file. |
| deliverable_root | The directory that deliverables are unpacked to. It is defined by the variable DELIVERABLE_ROOT in the environment file. |
| rev | The revision of the component. The revision can be a string or a call to a shell script. |
| grev | The global revision of the Subversion repository. |
| build_label | The build label, if code has been checked out by using the parameter label. |
| tested_rev | The revision used in tests to define the system under test. |
| tested_component | The name of the component used in tests to define the system under test |
| maia_core | The URL to the package maia_core. |
---
version: 5.0
rev: <%= `make -s version`.chomp %>
prepare:
cmd: make clean
build:
cmd: make build
dependency_generator: make -s dependency_list---
version: 5.0 # Protocol version, must be 5.0
component_name: template # Optional. If defined it must be the same as the name of the component.
rev: 1.0.2 # User-specified revision
subcomponents: # If this is a collection, specify sub component to builds before this component
- rf-common # Default, checkout and build with latest in track 'main'
- name: ps-common # Checkout and build with latest in a specified track
track: bug-fix4
- name: camel-rf-net # Do not checkout or build, reuse from build label
label: CAMEL_RF_NET_1.2-4
rebuild: false
- name: rf-common # Do not checkout or build, reuse from revision
revision: 4.0.5
rebuild: false
- name: rf-common # Checkout and rebuild, with label given by environment
label: <%= ENV['RF_COMMON_LABEL'] %>
rebuild: true
test: false # Don't test
clean:
cmd: rm -rf <%= build_root %>
prepare:
cmd: make clean
build:
cmd:
- mvn clean install
- rm -f <%= log_root %>/depend.txt
- mvn dependency:list > <%= log_root %>/depend.txt
dependency_generator:
- grep-dependencies <%= log_root %>/depend.txt <%= src_root %>/pom.xml <%= src_root %>/features.xml
deploy:
cmd: mvn deploy |tee > <%= log_root %>/deploy.txt
deploy_path_generator: grep-nexus-path <%= log_root %>/deploy.txt
test:
prepare_cmd:
- make clean_test
suites:
- name: system_test
cmd:
- do_test --suite regression.xml <%= rev %> <%= tested_component %> <%= tested_rev %> <%= maia_core %>
deliverables:
- name: mm-features # Specify deliverables and how to unpack them
unpack:
cmd: 'tar xf <%= deliverable_root %>/mm-features-<%= rev % >.jar'---
<% test_arg = "TEST_SUITE_ID=$test_suite_id API_TOKEN=#{api_token} ARTIFACT_ID=#{artifact_id} KAFKA_BROKERS=#{kafka_brokers} KAFKA_TOPIC=#{kafka_topic}" %>
<% set_env = "export PATH=#{build_root}/bin:$PATH; export LD_LIBRARY_PATH=#{build_root}/lib:$LD_LIBRARY_PATH;" %>
version: 5.0
rev: '<%= rev=`ruby $BLT_IPV_ROOT/bin/read_revision.rb local_repository/Makefile`.chomp %>'
prepare:
cmd: cd local_repository; make clean prepare
build:
cmd: cd local_repository; make build install
dependency_generator:
- <%= set_env %> ruby gem_dependencies.rb local_repository/maia/Gemfile.prod
- <%= set_env %> cd local_repository/maia && ruby npm_dependencies.rb
system_test:
prepare_cmd: cd local_repository/test; make clean
suites:
<% ['models', 'presenters', 'views', 'validations', 'features', 'lib', 'helpers', 'services', 'requests'].each do |r| %>
- name: rspec <%= r %>
cmd: cd local_repository/test; make rspec_test <%= test_arg %> PATTERN=spec/<%= r %>/*
<% end %>
<% ['models', 'jobs', 'helpers', 'lib', 'validations'].each do |m| %>
- name: 'mt <%= m %>'
cmd: cd local_repository/test; make mini_test <%= test_arg %> SUITE=<%= m %>
<% end %>