Remote Workers

The Remote Workers Configuration is a parallelism technique in Continuous Integration (CI) testing environment, where testing operations are not performed on the main orchestrator, but are delegated across various worker nodes. These worker nodes act as remote workers, enabling the simultaneous execution of multiple tests, thereby reducing feedback time.
In this configuration, the orchestrator (main node) is responsible for distributing tests among the worker nodes. Each worker node independently executes the assigned tests, ensuring quicker CI test execution.
Install
Follow the instructions in the Quick Start ⏱️guide to install Redefine.
Configure
Basic Configurations
Configure the relevant configurations both on the orchestrator and on the workers following the instructions in the quick start guide.
Configure a shared session ID
To use Redefine with the remote worker architecture, it is important to configure a shared session ID between the orchestrator and the worker nodes. This allows Redefine to connect the prediction and the execution of the tests.
Follow these steps to configure the shared Session ID:
1.It is crucial to generate the session ID once and provide it to both the orchestrator and worker nodes, to avoid misalignment between the orchestrator and the workers.  
2.You can use the CLI command redefine get session_id to generate the session ID, which represents the current session.
3.Export the environment variable REDEFINE_SESSION_ID with the same value on both the orchestrator and each worker node.

export REDEFINE_SESSION_ID=$(redefine get session_id)
Verify
Make sure that the verify command ends with success both for the orchestrator and the worker nodes, that will help ensure that the CI environment is valid for installation. 

Follow the Verify guide for instructions and troubleshooting.
Run
Orchestrator
Run the redefine start command to run redefine with the desired selection mode.
Use the testing framework-specific method of retrieving the list of collected tests without executing on the orchestrator (see the example for more information). Instead, use the optimized list of tests intended for distribution across the worker nodes. 

With the returned tests in hand, you should then leverage your existing logic to distribute the tests accordingly, adhering to the prescribed order.
Pytest
Cypress
redefine start --pytest --<selection_mode>
export OPTIMAL_TESTS_LIST=$(pytest tests --collect-only -q | head -n -2)
​
# Use OPTIMAL_TESTS_LIST in your existing test distribution logic
redefine start --cypress --<selection_mode> --dry-run --output-path=</file/path>
export OPTIMAL_TESTS_LIST=$(cat /file/path)
​
# Use OPTIMAL_TESTS_LIST in your existing test distribution logic
# All specs will be ordered by priority in <output_path>,
# separated with a new-line ('\n').
Workers
Run the redefine start command to run redefine using --worker as the selection mode.

Following this, you can execute the specified tests that have been assigned to the worker.
Pytest
Cypress
redefine start --pytest --worker
​
# Run the tests assigned to the current worker 
# based on the existing tests distribution logic
redefine start --cypress --worker
​
# Run the tests assigned to the current worker 
# based on the existing tests distribution logic
Maintain the test execution order
After the orchestrator completes, an optimized test execution order is provided. This test execution order is a crucial part of our orchestrator's function. The orchestrator creates an optimized list that maps out how the tests should be run for the most efficient workflow. Change this order, and you could see a significant drop in efficiency and potential miss of important tests.

Keep in mind, modifying the test order does more than just reduce efficiency — it could also compromise the accuracy of our test predictions. Therefore, it's essential to adhere to the initial order.
Session check command
In Redefine, tests will only be executed until the Time Limit is reached. Any remaining tests will be automatically skipped. However, it's important to be aware that the setup and teardown processes may impose a significant overhead.
To mitigate this overhead, it is highly recommended to periodically check if the test execution exceeds the time limit. If the time limit is exceeded, no more tests should be executed.
To determine if the time limit is exceeded, use the following command:
Pytest
Cypress
# Make sure REDEFINE_SESSION_ID is available
if redefine get session_check; then
    redefine start --pytest --worker
    # Run the tests assigned to the current worker 
fi
# Make sure REDEFINE_SESSION_ID is available
if redefine get session_check; then
    redefine start --cypress --worker
    # Run the tests assigned to the current worker 
fi
This is especially important when iterating over a large list of tests, as the overhead may increase dramatically.
Example
The example below shows a complete run with pytest running in Optimize mode.
Orchestrator
Worker
export REDEFINE_SESSION_ID=<new_session_id>
redefine config set stable_branches=dev
redefine config set time_limit=300
redefine start --pytest --optimize
export OPTIMAL_TESTS_LIST=$(pytest tests --collect-only -q | head -n -2)
​
# Use OPTIMAL_TESTS_LIST in your existing tests distribution logic
export REDEFINE_SESSION_ID=<new_session_id>
redefine config set stable_branches=dev
redefine config set time_limit=300
redefine start --pytest --worker
​
# Run the tests assigned to the current worker 
# based on the existing tests distribution logic
Last modified 4mo ago