Remote Workers

The Remote Workers Configuration is a parallelism technique in Continuous Integration (CI) testing environment, where testing operations are not performed on the main orchestrator, but are delegated across various worker nodes. These worker nodes act as remote workers, enabling the simultaneous execution of multiple tests, thereby reducing feedback time.

In this configuration, the orchestrator (main node) is responsible for distributing tests among the worker nodes. Each worker node independently executes the assigned tests, ensuring quicker CI test execution.

Install

Follow the instructions in the Quick Start ⏱️guide to install Redefine.

Configure

Basic Configurations

Configure the relevant configurations on the orchestrator following the instructions in the quick start guide. Note that configuration changes on worker nodes are not required for this process.

Configure a shared session ID

To use Redefine with the remote worker architecture, it is important to configure a shared session ID between the orchestrator and the worker nodes. This allows Redefine to connect the prediction and the execution of the tests.

Follow these steps to configure the shared Session ID:

1. It is crucial to generate the session ID once and provide it to both the orchestrator and worker nodes, to avoid misalignment between the orchestrator and the workers.

2. You can use the CLI command redefine get session_id to generate the session ID, which represents the current session.

3. Export the environment variable REDEFINE_SESSION_ID with the same value on both the orchestrator and each worker node.

export REDEFINE_SESSION_ID=$(redefine get session_id)

Verify

Make sure that the verify command ends with success both for the orchestrator and the worker nodes, that will help ensure that the CI environment is valid for installation.

Orchestrator

redefine verify --<testing-framework>

Worker

redefine verify --<testing-framework> --worker

Follow the Verify guide for instructions and troubleshooting.

Run

Orchestrator

Execute the redefine predict command to receive an ordered list of tests, which would be saved to a file in the desired path without actually running the tests. Once you have the list, you should distribute the tests in the existing logic, ensuring you follow the specified order.

Pytest

redefine predict --pytest --<selection_mode> --output-path="/file/path" --command="pytest tests"
export OPTIMAL_TESTS_LIST=$(cat /file/path)

# Use OPTIMAL_TESTS_LIST in your existing test distribution logic
# All specs will be ordered by priority in <output_path>,
# separated with a new-line ('\n').

Cypress

redefine predict --cypress --<selection_mode> --output-path="/file/path" --command="cypress run"
export OPTIMAL_TESTS_LIST=$(cat /file/path)

# Use OPTIMAL_TESTS_LIST in your existing test distribution logic
# All specs will be ordered by priority in <output_path>,
# separated with a new-line ('\n').

Mocha

redefine predict --mocha --<selection_mode> --output-path="/file/path" --command="npx mocha"
export OPTIMAL_TESTS_LIST=$(cat /file/path)

# Use OPTIMAL_TESTS_LIST in your existing test distribution logic
# All specs will be ordered by priority in <output_path>,
# separated with a new-line ('\n').

Workers

Run the redefine install command to run redefine using --worker as the selection mode. Following this, you can execute the specified tests that have been assigned to the worker.

Pytest

redefine install --pytest --worker

# Run the tests assigned to the current worker 
# based on the existing tests distribution logic

Cypress

redefine install --cypress --worker

# Run the tests assigned to the current worker 
# based on the existing tests distribution logic

Mocha

redefine install --mocha --worker

# Run the tests assigned to the current worker 
# based on the existing tests distribution logic

Maintain the test execution order

After the orchestrator completes, an optimized test execution order is provided. This test execution order is a crucial part of our orchestrator's function. The orchestrator creates an optimized list that maps out how the tests should be run for the most efficient workflow. Change this order, and you could see a significant drop in efficiency and potential miss of important tests. Keep in mind, modifying the test order does more than just reduce efficiency — it could also compromise the accuracy of our test predictions. Therefore, it's essential to adhere to the initial order.

Session check command

In Redefine, tests will only be executed until the is reached. Any remaining tests will be automatically skipped. However, it's important to be aware that the setup and teardown processes may impose a significant overhead.

To mitigate this overhead, it is highly recommended to periodically check if the test execution exceeds the time limit. If the time limit is exceeded, no more tests should be executed.

To determine if the time limit is exceeded, use the following command:

Pytest

# Make sure REDEFINE_SESSION_ID is available
if redefine get session_check; then
    redefine install --pytest --worker
    # Run the tests assigned to the current worker 
fi

Cypress

# Make sure REDEFINE_SESSION_ID is available
if redefine get session_check; then
    redefine install --cypress --worker
    # Run the tests assigned to the current worker 
fi

Mocha

# Make sure REDEFINE_SESSION_ID is available
if redefine get session_check; then
    redefine install --mocha --worker
    # Run the tests assigned to the current worker 
fi

This is especially important when iterating over a large list of tests, as the overhead may increase dramatically.

Example

The example below shows a complete run with pytest running in Optimize mode.

Orchestrator

export REDEFINE_SESSION_ID=<new_session_id>
redefine config set time_limit=300
redefine predict --pytest --optimize --output-path="/file/path" --command="pytest tests"
export OPTIMAL_TESTS_LIST=$(cat /file/path)

# Use OPTIMAL_TESTS_LIST in your existing tests distribution logic

Worker

export REDEFINE_SESSION_ID=<new_session_id>
redefine install --pytest --worker

# Run the tests assigned to the current worker 
# based on the existing tests distribution logic