Advanced Usage

The purpose of this document is to give an introduction to advanced usage patterns of the MythX CLI as well as functionality the user might not have been aware of.

Automatic Group Creation

The basic workflow for attaching analyses to groups can be divided into the following steps:

  1. Open a new group with mythx group open
  2. Submit analyses in reference to the new group mythx analyze --group-id <ID>
  3. Close the group when done mythx group close <ID>

This can become very annoying and hinder automation tasks leveraging the MythX CLI. Because of that, the --create-group flag has been introduced. Passing this flag to mythx analyze will automatically open a group before submitting new analysis jobs, and close it after the submission process has been completed. This functionality encapsulates all targets passed to the analyze subcommand.

A short example:

$ mythx analyze --create-group dir1/ test.sol test2.sol

This command will initialize an empty group, add all analyses coming from dir1/ and the two test Solidity files into it, and close it once submission has been completed. The analyses will then show up in their dedicated group inside the dashboard.

File Output

Especially in scenarios of automation the need often arises to persist data and store it as files. Since version 0.4.0 the base mythx command carries the --output option. This allows you to take the stdout from any subcommand and store it as a file. This can be very helpful for storing analysis job and group IDs for long-running asynchronous jobs - which we will also outline in this document.

The --format option is fully supported with --output and allows the user to define what is written to the file. Furthermore, it can be combined with every subcommand mythx supports.

Examples:

  1. mythx --output=status.json --format=json-pretty status <id>: Output the status of an analysis job in pretty-printed JSON format to status.json.
  2. mythx --output=report.json --format=json report report <id>: This is equivalent as above with the difference being that now the analysis job’s report is fetched and directly written to the file. This is especially useful for testing new formatters and other integrations with static input.
  3. mythx --output=analyses.txt analyze --async <targets>: This performs a quick analysis on the defined targets (e.g. truffle projects, Solidity files) and instead of waiting for the results, simply writes the analysis job IDs to the analyses.txt files. This can be helpful for when you need to persist long-running analysis IDs and cannot wait for them to finish, e.g. when running full-mode analyses on a CI server.

Filtering Reports

The MythX CLI offers a variety of flags to filter report output. These are available in the analyze and report subcommands respectively and identical in their behaviour. Report filtering can be used to hide unwanted issues detected by MythX, e.g. a floating Solidity compiler pragma (SWC-103). There are three flags with report filtering capabilities available:

  1. --swc-blacklist: Ignore specific SWC IDs, e.g. --swc-blacklist SWC-104 will display all found issues except for unchecked call return values.
  2. --swc-whitelist: Ignore all SWC IDs that do not match the given ones, e.g. --swc-whitelist 110,123 will only display requirement and assert violations.
  3. --min-severity: Ignore issues below the given severity, e.g. --min-severity medium ignores all low severity issues.

The SWC black- and whitelist parameters support a variety of formats. Besides individual SWC IDs, the user can also pass in a comma-separated list of multiple IDs to submit a list. Furthermore, the inputs can be expressed in different manners, i.e. swc-110, SWC-110, and 110 all target the same SWC ID. List submission works in the same way, so --swc-blacklist=SWC-101,102,swc-103 is a valid way to exclude IDs 101-103 from the report, even though it is definitely recommended to stick with a consistent notation for readability purposes.

Property Verification Mode

To leverage MythX for property verification, simply pass the --check-properties flag to the analyze subcommand. This will instruct the MythX analysis backend to only look for assertion violations and optimize execution.

Asynchronous Analysis

As mentioned before, the analyze subcommand offers the --async flag. This will prevent the CLI from blocking after the submission of one or multiple analysis jobs and waiting for them to finish. Instead, for each analysis job the job’s UUID will be printed to stdout, or to a file, if the --output option is passed as well. This feature is often used in CI scenarios, where the server job is not expected to run for very long. It is strongly recommended that this flag is used in combination with MythX plans that offer higher analysis times. These scans are more exhaustive, deliver more precise results, but as a trade-off take longer to process. In the CI server scenario, a developer might choose to submit analyses asynchronously and check on the job later in the MythX dashboard. Alternatively, the job IDs could also be stored in a file as a CI artifact and later retrieved by another part of the pipeline, e.g. to kickstart a security or QA pipeline.

An example is the submission of a large truffle project:

$ mythx analyze --async my-project/

This flag is also best friends with the --create-group flag for the analyze subcommand. Together they help keeping the MythX dashboard overview tidy.

The CI Flag

MythX is designed to support developers throughout the process of building their product by providing continuous security checks. It is self-understood that CI use cases present their own set of challenges, and the MythX CLI aims to support this process by providing the –ci flag in the base command. This flag sets the application’s return code to 1 if any issues were found in the analysis.

The true power of this flag becomes apparent when taking into consideration that it is fully integrated with the options available for report filtering. This means that the return code can be set depending on the input provided to the swc-blacklist, swc-whitelist, and min-severity options. A use case is to make CI jobs only fail on high-severity issues, but excluding a subset of them because they are already in the process of being fixed, or insignificant relating to the business logic.

The filtering options can be freely combined with the --ci flag to achieve the desiged behaviour. A simple example is excluding the (fairly common) floating pragma issue type, and assert and requirement violations for testing purposes:

$ mythx --ci analyze --swc-blacklist 110,123,103 my-project/

Import Remapping and Relative Paths in solc

When given one or more Solidity files as argument, the MythX CLI will try to compile them using solc to submit the resulting bytecode, AST, and source mappings. Especially in more complex smart contract systems, contract dependencies such as zOS and OpenZeppelin libraries are pulled in using NPM. These can then be imported using import remappings in the solc call. These remappings are supported by the MythX CLI as well. Given some example Solidity imports that would make standard compilation fail:

import "openzeppelin-zos/contracts/token/ERC721/ERC721Token.sol";
import "openzeppelin-zos/contracts/token/ERC721/ERC721Receiver.sol";
import "openzeppelin-zos/contracts/ownership/Ownable.sol";

These imports will have to be remapped. This can be done by passing the --remap-import parameter to the analyze call:

$ mythx analyze --remap-import "openzeppelin-zos/=$(pwd)/node_modules/openzeppelin-zos/" myContract.sol

This parameter can be defined multiple times to declare various import remappings in the context of the same call. If no remappings are given, the MythX CLI tries to make the user’s life as easy as possible by defining a set of remappings that should act as a sane default:

openzeppelin-solidity/=<pwd>/node_modules/openzeppelin-solidity/
openzeppelin-zos/=<pwd>/node_modules/openzeppelin-zos/
zos-lib/=<pwd>/node_modules/zos-lib

This does not affect relative imports such as

import "../interfaces/MyInterface.sol";

These are supported by default through the MythX CLI by adding the current working directory the call was made from to the allowed solc paths. Please note that if compilation fails on a relative import, the current working directory was not the project root that results in correct import resolution.

Configuration using .mythx.yml

Using import remappings, contract filters, SWC black-/whitelists, and various other configuration options can result in large CLI commands. The .mythx.yml file can remediate this by providing the user with an easy-to-read and -update YAML configuration file.

Top-level parameters (included after the mythx command) are defined on the top level of the configuration file, while analysis-specific parameters (included after the analyze subcommand) are included under the analyze key. For example:

output: mythx.json
format: json

analyze:
    mode: quick
    create-group: true
    group-name: My fancy analysis
    solc: 0.5.16
    remappings:
        - "@openzeppelin/=/my/path/node_modules/@openzeppelin/"
        - "@nomiclabs=/my/path/node_modules/@nomiclabs/"
    contracts:
        - Contract1
        - Contract2
        - Contract3
        - Contract4
        - Contract5

This will execute a quick analysis on the five specified contracts. Compilation is done using solc version 0.5.16 and the specified import remappings are passed to the compiler. Additionally, a new group will be opened for this submission under the name My fancy analysis. After submission, the CLI will wait until all contracts have been analyzed and output the resulting report in JSON format. This report will be written into the mythx.json file. In a CI scenario, this report could for example be stored as an artifact for later retrieval and further processing.

The currently supported top-level configuration keys are:

  • ci: Boolean indicating whether to return 1 if any severe issue is found (equivalent to --ci)
  • output: Name of the file to write output data into (equivalent to --output)
  • format: The output format to return (equivalent to --format)
  • confirm: Boolean indicating the automatic confirmation of multiple file submissions (equivalent to --yes)
  • table-sort-key: The column name to sort the default table output by (equivalent to --table-sort-key)

The analyze configuration keys currently supported are:

  • mode: The analysis mode to run MythX on (equivalent to --mode)
  • create-group: Boolean indication whether to create a new group for the submission (equivalent to --create-group)
  • group-id: The group ID to add the submitted analyses to (equivalent to --group-id)
  • group-name: The name to attach to the newly created group (equivalent to --group-name)
  • min-severity: Ignore SWC IDs below the designated level (equivalent to --min-severity)
  • blacklist: A comma-separated list of SWC IDs to ignore (equivalent to --swc-blacklist)
  • whitelist: A comma-separated list of SWC IDs to include (equivalent to --swc-whitelist)
  • async: A boolean indicating whether to submit the analyses asynchronously (equivalent to --async)
  • solc: The solc version to use for Solidity file compilation (equivalent to --solc-version)
  • remappings: A list of import remappings to pass to the solc compiler (equivalent to one or multiple --remap-import parameter(s))
  • contracts: A list of contracts to include in the submission (equivalent to one or multiple --include parameter(s))
  • check-properties Enable property verification mode (filter out everything other than assertion violations in the backend and optimize for property verification)
  • targets: A list of targets to analyze. This is equivalent to passing an argument directly to the analyze command - whether it’s a Solidity file, a directory, a Truffle project, or a mix of all.

Property Validation with Scribble

Scribble is a tool develped by ConsenSys Diligence that aims to facilitate verification of invariants over smart contract systems. This is done by annotating parts of the Solidity code with instrumentation comments. Scribble as a tool then translates the user-provided specifications into an instrumented smart contract that can be sent to MythX. The MythX analysis tools will then specifically look for assertion violations that break the defined invariants and generate a report outlining how they were broken.

Scribble is currently in its early alpha version. The MythX CLI still aims to natively provide a Scribble integration to make property validation of Solidity smart contracts as easy as possible. At the moment it is integrated into the analyze subcommand and can be accessed as follows:

$ mythx analyze --scribble annotated_token.sol
Found 1 job(s). Submit? [y/N]: y
  [####################################]  100%
Report for scribble-annotated_token.sol
https://dashboard.mythx.io/#/console/analyses/9a2c4e24-0c9d-4834-b11f-fb99061ba910
╒════════╤══════════════════╤════════════╤═══════════════════════════════════════╕
│   Line │ SWC Title        │ Severity   │ Short Description                     │
╞════════╪══════════════════╪════════════╪═══════════════════════════════════════╡
│     41 │ Assert Violation │ Medium     │ An assertion violation was triggered. │
╘════════╧══════════════════╧════════════╧═══════════════════════════════════════╛

To view the details on how the invariant was violated, check out the report details on the dashboard, or switch to a different output format, such as json-pretty.

Custom Report Rendering

The MythX CLI exposes a subcommand render, which allows the user to generate HTML reports of the analyses inside a group, or an individual analysis job. The --template flag allows the user to submit their own report template. This bears the question: How is a custom template written? This section aims to explain the two ways of writing a custom template:

  1. Write a new template from scratch
  2. Extend the default layout.html or layout.md with the pre-defined blocks

Writing a New Template From Scratch

Is the default layout too complex? Do block names confuse you? No worries! The MythX CLI of course also support completely user-defined templates. These templates can be specified using the Jinja2 syntax. With basic knowledge of HTML, CSS, Jinja, and possibly also JavaScript (if you feel fancy), it is fairly easy to write a template. Explaining the inner workings of Jinja and the core principles of web design are out of scope for this section. It is relevant to know what context MythX provides for user-defined templates. There are two core items that are rendered onto the template. The report_context, and the target.

The target is a string containing either the analysis group ID, or the analysis job UUID that the user has passed to the :code`render` subcommand.

The report_context is a dictionary containing a mapping from file name to a list of line objects. Each line object has the following schema:

{
  "content": "        emit OwnershipTransferred(address(0), _owner);",
  "issues": [],
  "line": 19
}

The issues key can contain zero or more issue objects that have the following schema:

{
  "description": {
    "head": "Any sender can withdraw Ether from the contract account.",
    "tail": "Arbitrary senders other than t..."
  },
  "severity": "High",
  "swcID": "SWC-105",
  "swcTitle": "Unprotected Ether Withdrawal",
  "testCases": []
}

Generating a simple report is as easy as iterating over the report_context parameter and displaying the properties of each issue object in the desired way:

{% for filename, file_data in report_context %}
{# my template code #}

Extending the Default HTML Template

The MythX CLI default template is generated from two files: layout.html and default.html. The former defines the overall structure of the report page, namely the CSS grid and the components built on top of it. The latter template extends the layout file and adds the default theme’s color scheme and fonts.

In Jinja2, the templating language used by the report renderer, templates can be extended by defining so-called blocks in the template file to be extended. Blocks can contain content already to define a sane default. Otherwise, the extending template can choose to overwrite specific blocks of the extended templates to inject customized content. This is a powerful mechanic that is extensively used by the report rendering engine. A short example:

Let’s assume we have a base template base.html that defines the following code in its HTML head tag:

<head>
    <title>{% block title %}Default{% endblock %}</title>
</head>

An extending template extended.html might them contain the following code:

{% extends "base.html" %}
{% block title %}My Extended Title{% endblock %}

In the final template, we will get the combined code:

<head>
    <title>My Extended Title</title>
</head>

The advantages here are obvious: By providing a sane default template with reasonable block definitions, the MythX CLI can allow the user to make quick and rather deep updates to the final HTML template without them needing to go through the hassle of reading and understanding the HTML, CSS, and Jinja statements written in the overall default template - even though this knowledge becomes more useful the deeper the user aims to change things up.

More details can be found in the official Jinja template inheritance docs.

All blocks in the default template are scoped, meaning that the extending template has access to all context variables around the block in the base template file. This allows the user to e.g. access report objects inside the block from the extending layout to customize the way things are displayed. The blocks defined in the layout template are as follows:

  • head
    Defines the head HTML tag. This will overwrite all default content including CSS styles and the site title.
  • style
    Defines the CSS styles. If you want to keep the default template’s style, consider using {{ super() }} insite the extending block definition to insert the styles from the parent template.
  • title
    Defines the site title as it appears in the Browser tab and header.
  • extra_html
    This is a block that is empty by default. It allows the user to insert extra HTML tags at the beginning of the body element - before anything else is defined. This is expecially useful for overlays, but with the flexibility of custom CSS styles for the inserted element, it can be positioned elsewhere in absolute terms,
  • navigation
    Defines the content of the navigation bar on the left-hand side of the page. It should contain an overview of all the reports inside the template and allow the user to click on a navigation link that jumps directly to the selected analysis report.
  • navigation_header
    Defines the heading (h2) of the navigation bar. By default it is defined as Overview.
  • main_header
    Defines the content of the main header (a header tag with class main-head). This tag is displayed on top of the main page’s report listing. If only the name needs to be customized, it is recommended to use the main_header_name block instead.
  • main_header_name
    Defines the main header name. It is displayed on top of the main page’s report listing. By default it is MythX Report for {{ target }} where the target variable is the group or analysis job ID submitted by the user to the render subcommand.
  • report_header
    Defines the report header. This is the section on top of each analysis report inside the main page’s report listing. By default it contains a heading with the analysis job’s main source file(s), and a small link to the official MythX dashboard’s analysis report labelled with the analysis UUID. More fine-grained customization can be done using the blocks below.
  • report_header_name
    Defines the report header name. This is the heading on top of each report, containing the main source file(s) of the analysis job. By default, this heading has the analysis job’s UUID as ID. This is done so a user can reference the tag’s ID in the navigation bar to quickly jump to specific report listing entries.
  • section_report
    Defines the central report section of an analysis job in the main page’s report listing. By default this section displays a table is displayed showing the SWC-IDs of the found vulnerabilities along with its verbose name, the file name it was found in, and location information carrying line and column number. More fine-grained customization can be done with the blocks below.
  • section_report_id
    Defines the SWC-ID column name in the report issues overview table. This block can be used to e.g. change the column name to its equivalent in another language.
  • section_report_name
    Defines the SWC title column name in the report issues overview table. This block can be used to e.g. change the column name to its equivalent in another language.
  • section_report_location
    Defines the issue location column name in the report issues overview table. This block can be used to e.g. change the column name to its equivalent in another language.
  • section_code_name
    Defines the name of the collapsible to display the source code. This block can be used to e.g. change the column name to its equivalent in another language.
  • section_case_name
    Defines the name of the collapsible to display the issue test case. This block can be used to e.g. change the column name to its equivalent in another language.
  • section_code_step_name
    Defines the name of the collapsible to display a test case’s step name. This block can be used to e.g. change the column name to its equivalent in another language.
  • footer
    Defines the content of the footer. By default, the footer carries the class main-footer, which by default has an absolute fixed position at the bottom. This block by default gives credits to MythX CLI, which was used to generate the report. It can be customized with the user’s own branding. Kudos to the MythX CLI is not required, but always appreciated. :)

Extending the Default Markdown Template

Extending the default Markdown template is considerably easier compared to extending the HTML one. This is mainly due to the fact that Markdown is a simpler language and the resulting report does not contain any interactive elements such as expanding sections to hide the code, or even a navigation bar to quickly jump to reports.

To allow flexibility without rewriting the whole templates/layout.md file are as follows:

  • heading
    This block contains the overall report’s heading, such as “MythX Report for …”.
  • preamble
    This text sits right below the heading and is empty by default. It can be used to add a disclaimer, custom branding, report owners, timestamps, etc. to the report; Any meta information that is deemed to be useful in the report’s context.
  • header
    This is the header that is displayed for each report in the analysis group - or the single analysis job (depending on the user input). It stands above the report status and issues overview and should describe the job displayed below.
  • report
    This block should give detailed information about the issue that has been found. By default, it contains the vulnerability title, SWC ID, Severity, the corresponding source lines, and a short source listing, containing one line before and after the source position. If the source line decoding failed, it will display undefined as line locations and omit the source snippet.
  • no_issues_name
    This block should contain the message that is displayed when no issues were found in the report. It is displayed instead of the above status and report blocks.
  • postamble
    Similar to the preamble block, this text is displayed at the end of the report listing. It can be used for displaying license texts and more verbose information that might be needed in the future but are not essential to the report itself. By default it displays a little heart and a link to the MythX CLI Github repository. Kudos are always appreciated and you have my thanks if you keep the credit intact during your awesome customization work. :)