• Overview
• ARPA Node Client
  • Usage
• ARPA Node Config Checker
  • Usage
• ARPA Node CLI
  • Usage
  • REPL Commands
    • SubCommands
• Management grpc server
• Dependencies
• Troubleshooting
• Node Config
• Local Test

Arpa Node

Overview

This crate provides a set of tools on the node side of the ARPA BLS Threshold Signature Scheme (BLS-TSS) Network, including Threshold-BLS based on-chain randomness service(Randcast).

It consists of:

• ARPA Node Client
• ARPA Node Config Checker
• ARPA Node CLI
• Management grpc server

ARPA Node Client

The ARPA Node Client is a long-running program to run the ARPA node.

If the data path in the config file doesn't exist, as the first time to run the node, the client will generate a DKG keypair(served as the identity during a grouping process).

Usage

cd crates/arpa-node

cargo run --bin node-client

To print help, use -- -h:

cargo run --bin node-client -- -h

To specify a config file, use -- -c <config_file>:

cargo run --bin node-client -- -c conf/config.yml

ARPA Node Config Checker

The ARPA Node Config Checker is a tool to check the correctness of the node config file. It will print the checksum encoded address of the node identity(wallet) if the config file is correct, otherwise it will print the error message.

Usage

To specify a config file, use -- -c <config_file>:

cd crates/arpa-node
cargo run --bin node-config-checker -- -c conf/config.yml

ARPA Node CLI

The ARPA Node CLI is a fast and verbose REPL for the operator of a ARPA node. The same node config file as ARPA Node Client will be used. As a supplement to ARPA Node Client, it provides a set of commands to inspect the node status and interact with the on-chain contracts, e.g. register node to the network manually as an Eigenlayer EOA operator.

Usage

To print help, use -- -h:

cd crates/arpa-node

cargo run --bin node-shell -- -h

To specify a config file, use -- -c <config_file>:

cargo run --bin node-shell -- -c conf/config.yml

To set the history file path, use -- -H <history_file>:

cargo run --bin node-shell -- -H node-shell.history

REPL Commands

Commands:
  show      Show information of the config file and node database
  call      Get views from on-chain contracts
  history   Show command history
  send      *** Be careful this will change on-chain state and cost gas ***
                Send trxs to on-chain contracts
  generate  Generate node identity(wallet) corresponding to ARPA node format
  inspect   Connect to the node client and inspect the node status
  help      Print this message or the help of the given subcommand(s)

Options:
  -h, --help  Print help

SubCommands

Show information of the config file and node database

Usage: show [COMMAND]

Commands:
  address  Show address of the node identity(wallet) [aliases: a]
  config   Print node config [aliases: c]
  node     Print node info from node database [aliases: n]
  help     Print this message or the help of the given subcommand(s)

Options:
  -h, --help  Print help

Get views and events from on-chain contracts

Usage: call [COMMAND]

Commands:
  block                        Get block information [aliases: b]
  current-gas-price            Get current gas price [aliases: cgp]
  trx-receipt                  Get transaction receipt [aliases: tr]
  balance-of-eth               Get balance of eth [aliases: boe]
  last-randomness              Get last randomness [aliases: lr]
  pending-request-commitment   Get pending commitment by request id [aliases: prc]
  controller-config            Get controller config [aliases: cc]
  adapter-config               Get adapter config [aliases: ac]
  last-assigned-group-index    Get last assigned group index in randomness generation [aliases: lagi]
  randomness-count             Get randomness count [aliases: rc]
  cumulative-data              Get cumulative data(FlatFee, CommitterReward and PartialSignatureReward) of randomness generation [aliases: cd]
  fulfillments-as-committer    Get all fulfillment events as committer in history [aliases: fac]
  fulfillments-as-participant  Get all fulfillment events as participant in history [aliases: fap]
  node                         Get node info by id address [aliases: n]
  group                        Get group info by index [aliases: g]
  valid-group-indices          Get valid group indices which are ready for randomness generation [aliases: vgi]
  group-epoch                  Get global group epoch [aliases: ge]
  group-count                  Get global group count [aliases: gc]
  belonging-group              Get the group index and member index of a given node [aliases: bg]
  member                       Get group member info by group index and member index [aliases: m]
  coordinator                  Get group coordinator during a running dkg process by group index [aliases: c]
  node-withdrawable-tokens     Get node withdrawable tokens(eth and arpa rewards) by id-address [aliases: nwt]
  stake                        Get node staked arpa amount [aliases: s]
  delegation-reward            Get node delegation reward [aliases: dr]
  delegates-count              Get eligible nodes count [aliases: dc]
  balance-of-arpa              Get balance of arpa [aliases: boa]
  frozen-principal             Get frozen principal and unfreeze time [aliases: fp]
  help                         Print this message or the help of the given subcommand(s)

Options:
  -h, --help  Print help

*** Be careful this will change on-chain state and cost gas as well as block time***
Send trxs to on-chain contracts

Usage: send [COMMAND]

Commands:
  approve-arpa-to-staking          Approve arpa to staking contract [aliases: aats]
  stake                            Stake arpa to staking contract [aliases: s]
  unstake                          Unstake(then freeze) arpa from staking contract and claim delegation rewards instantly after exit [aliases: u]
  claim-frozen-principal           Claim frozen principal from staking after unstake [aliases: cfp]
  register-as-eigenlayer-operator  Register node as Eigenlayer operator [aliases: raeo]
  activate-as-eigenlayer-operator  Activate node after exit or slashing as Eigenlayer operator [aliases: aaeo]
  quit                             Quit node from Randcast network [aliases: q]
  change-dkg-public-key            Change dkg public key(recorded in node database) after exit or slashing [aliases: cdpk]
  withdraw                         Withdraw node reward to any address [aliases: w]
  help                             Print this message or the help of the given subcommand(s)

Options:
  -h, --help  Print help

Generate node identity(wallet) corresponding to ARPA node format

Usage: generate [COMMAND]

Commands:
  private-key  Generate private key(not recommended) [aliases: pk]
  keystore     Generate keystore file [aliases: k]
  hd-wallet    Generate hierarchical deterministic wallet and save the mnemonic to a file [aliases: hw]
  help         Print this message or the help of the given subcommand(s)

Options:
  -h, --help  Print help

Connect to the node client and inspect the node status

Usage: inspect [COMMAND]

Commands:
  list-fixed-tasks  List fixed tasks of the node [aliases: lft]
  help              Print this message or the help of the given subcommand(s)

Options:
  -h, --help  Print help

Management grpc server

This server supports inspecting states and interacting with a running node.

Please see management.proto for detailed apis.

Dependencies

Install protoc and foundry, then run

cargo build

Troubleshooting

"error: linker cc not found"... when running cargo build

sudo apt install build-essential
sudo apt install pkg-config
sudo apt install libssh-dev

Node Config

Configuration items in conf/config.yml are listed here:

Note: To protect secrets, several items can be set with literal `env` as placeholder. Their env keys are:

    - ARPA_NODE_MANAGEMENT_SERVER_TOKEN (node_management_rpc_token)
    - ARPA_NODE_ACCOUNT_PRIVATE_KEY (account, private_key)
    - ARPA_NODE_ACCOUNT_KEYSTORE_PASSWORD (account, keystore, password)
    - ARPA_NODE_HD_ACCOUNT_MNEMONIC (account, hdwallet, mnemonic)

    Items below can also be set with arbitrary environment variables starting with `$`:

    - $<CUSTOMIZED_ENV_VARIABLE_KEY>(provider_endpoint / relayed_chains.provider_endpoint)
    - $<CUSTOMIZED_ENV_VARIABLE_KEY>(node_management_rpc_token)
    - $<CUSTOMIZED_ENV_VARIABLE_KEY>(account, private_key)
    - $<CUSTOMIZED_ENV_VARIABLE_KEY>(account, keystore, password)
    - $<CUSTOMIZED_ENV_VARIABLE_KEY>(account, hdwallet, mnemonic)

• node_committer_rpc_endpoint: Endpoint that this node will use to create server socket to expose committer grpc services. Once this get changed, the node MUST re-activate itself to the controller so that the controller can update the endpoint by re-grouping. (example: "0.0.0.0:50060")

• node_advertised_committer_rpc_endpoint: Endpoint that other members in the group will use to connect to this node. If this setting is not set, then value of node_committer_rpc_endpoint will be used here and published to other nodes. Note: This setting is updated every time the node starts, but it will not be broadcasted to other nodes until next re-grouping. (example: "10.0.0.1:50060")

• node_management_rpc_endpoint: Config endpoint to expose management grpc services. (example: "0.0.0.0:50099")

• node_management_rpc_token: Config token phrase for authenticaing management grpc requests by authorization header. (example: "arpa_network")

• node_statistics_http_endpoint: Config endpoint to expose statistics http services. (example: "0.0.0.0:50081")

• provider_endpoint: Config websocket endpoint to interact with chain provider. (example: "ws://127.0.0.1:8546")

• is_eigenlayer: Config whether the node is registered as an eigenlayer operator, or a native staking operator. (example: false)

• is_consistent_asset_and_node_account: Config whether the node's asset account is consistent with the node account. (example: false)

• chain_id: Config chain id of main chain. (example: 31337)

• controller_address: Config Controller contract address to manage nodes and groups. (example: "0x0000000000000000000000000000000000000001")

• controller_relayer_address: Config ControllerRelayer contract address to relay groups to relayed chains. (example: "0x0000000000000000000000000000000000000001")

• adapter_address: Config Adapter contract address to request and fulfill randomness task. (example: "0x0000000000000000000000000000000000000001")

• adapter_deployed_block_height(Optional, used for ARPA Node CLI): Config the block height when adapter contract is deployed to accelerate the query of events. (example: 100000)

• arpa_address(Optional, used for ARPA Node CLI): Config on-chain ARPA token contract address. (example: "0x9fe46736679d2d9a65f0992f2272de9f3c7fa6e0")

• data_path(Optional): Config DB file for persistence. (example: "data.sqlite")

• logger(Optional): Config logger settings.

  • example(default):

    logger:
      context_logging: false
      log_file_path: log/running/
      rolling_file_size: 10 gb
    

  • context_logging: Set whether to log context of current node info and group info. Since the log size will get a significant boost with this setting enabled, it is recommended to set it to false in production.

  • log_file_path: Set log file path. The node-client will create a node.log as well as a node_err.log under log_file_path, then log to them with info level and error level respectively.

  • rolling_file_size: Log file will be deleted when it reaches this size limit. The following units are supported (case insensitive): "b", "kb", "kib", "mb", "mib", "gb", "gib", "tb", "tib". The unit defaults to bytes if not specified.

• account: Config node identity in the network. There are three available account types.

  • example(not recommended): private_key: "4c0883a69102937d6231471b5dbb6204fe5129617082792ae468d01a3f362318"

  • example:

    keystore:
        password: env
        path: test.keystore
    

  • example:

    hdwallet:
        mnemonic: env
        path: "m/44'/60'/0'/0"
        index: 0
        passphrase: "custom_password"
    

    Path and passphrase are optional.

• time_limits(Optional): Config time limits for different tasks. All the time limits are in milliseconds or block numbers.

  • example:

    time_limits:
      block_time: 3
      dkg_timeout_duration: 40
      randomness_task_exclusive_window: 10
      listener_interval_millis: 10000
      dkg_wait_for_phase_interval_millis: 10000
      provider_polling_interval_millis: 10000
      provider_reset_descriptor:
        interval_millis: 5000
        max_attempts: 17280
        use_jitter: false
      contract_transaction_retry_descriptor:
        base: 2
        factor: 1000
        max_attempts: 3
        use_jitter: true
      contract_view_retry_descriptor:
        base: 2
        factor: 500
        max_attempts: 5
        use_jitter: true
      commit_partial_signature_retry_descriptor:
        base: 2
        factor: 1000
        max_attempts: 5
        use_jitter: false
    

  • block_time: Block time of the chain. This value is used to calculate the max pending time of a randomness task. (example: 3)

  • These values need to be set according to config of on-chain Controller contract.

    • dkg_timeout_duration: Block numbers between DKG start and timeout. (example: 40)
    • randomness_task_exclusive_window: Block numbers when a randomness task can be only fulfilled by the assigned group. (example: 10)

  • These values can be set by node owner or administrator according to the rate limitation of the provider. Setting a small value would be to node's advantage in responding tasks. It's recommended to set a value no larger than the block time of the chain.

    • listener_interval_millis: Milliseconds between two rounds of re-trying when a listener fails. (example: 10000)
    • dkg_wait_for_phase_interval_millis: Milliseconds between two rounds of polling for the next DKG phase. (example: 10000)
    • provider_polling_interval_millis: Milliseconds between two rounds of polling pending transactions. (example: 10000)

  • We use fixed interval to reset the provider when it can't be reconnected.

    • provider_reset_descriptor: (interval sequence by default: 5s, 10s, ..., 24h)

  • We use exponential backoff to retry when a transaction or view call fails, or a rpc request to the committer fails. The interval will be an exponent of base multiplied by factor every time, and it will be reset when the interaction succeeds.

    • interval = factor * base ^ attempt

  • A jitter is added to the interval to avoid the situation that all the tasks are polling at the same time. It will multiply a random number between 0.5 and 1.0 to the interval.

    • contract_transaction_retry_descriptor: (interval sequence without jitter: 2s, 4s, 8s)
    • contract_view_retry_descriptor: (interval sequence without jitter: 1s, 2s, 4s, 8s, 16s)
    • commit_partial_signature_retry_descriptor: (interval sequence without jitter: 2s, 4s, 8s, 16s, 32s)

• listeners(Optional): Config listeners to run with node client to customize services. By default all the listeners will be enabled. All of them can be disabled by setting an empty value explicitly.

  • example:

  listeners:
    - l_type: Block
      interval_millis: 0
      use_jitter: true
    - l_type: NewRandomnessTask
      interval_millis: 0
      use_jitter: true
    - l_type: PreGrouping
      interval_millis: 0
      use_jitter: true
    - l_type: PostCommitGrouping
      interval_millis: 10000
      use_jitter: true
    - l_type: PostGrouping
      interval_millis: 10000
      use_jitter: true
    - l_type: ReadyToHandleRandomnessTask
      interval_millis: 10000
      use_jitter: true
    - l_type: RandomnessSignatureAggregation
      interval_millis: 2000
      use_jitter: false
  

  • Block, NewRandomnessTask, PreGrouping, PostCommitGrouping, PostGrouping, ReadyToHandleRandomnessTask, RandomnessSignatureAggregation are the types of listeners. We use a fixed interval to retry when a listen round fails. The interval_millis and use_jitter are the same as the time_limits.

    • The polling intervals of Block, NewRandomnessTask and PreGrouping are decided by provider_polling_interval_millis in time_limits.

    • The polling of PostCommitGrouping, PostGrouping, ReadyToHandleRandomnessTask are triggered by view calls on the chain, so the interval_millis should be set to a value no larger than the block time of the chain.

    • The polling of RandomnessSignatureAggregation is triggered by the node itself, so the interval_millis can be set relatively small.

• relayed_chains: Config chain_id, description, contract addresses, endpoint, time_limits and listeners for all relayed chains we support.

  • example:

  relayed_chains:
  - chain_id: 901
    description: "OP"
    provider_endpoint: "ws://127.0.0.1:9546"
    controller_oracle_address: "0xCf7Ed3AccA5a467e9e704C703E8D87F634fB0Fc9"
    adapter_address: "0x5FC8d32690cc91D4c39d9d3abcBD16989F875707"
    adapter_deployed_block_height: 14224644
    arpa_address: "0xA129BEA1a5d9E37Eb2C505c8D302231A28B0A82b"
    listeners:
      - l_type: Block
        interval_millis: 0
        use_jitter: true
      - l_type: NewRandomnessTask
        interval_millis: 0
        use_jitter: true
      - l_type: ReadyToHandleRandomnessTask
        interval_millis: 1000
        use_jitter: true
      - l_type: RandomnessSignatureAggregation
        interval_millis: 2000
        use_jitter: false
    time_limits:
      block_time: 2
      randomness_task_exclusive_window: 10
      listener_interval_millis: 1000
      provider_polling_interval_millis: 1000
      provider_reset_descriptor:
        interval_millis: 5000
        max_attempts: 17280
        use_jitter: false
      contract_transaction_retry_descriptor:
        base: 2
        factor: 1000
        max_attempts: 3
        use_jitter: true
      contract_view_retry_descriptor:
        base: 2
        factor: 500
        max_attempts: 5
        use_jitter: true
      commit_partial_signature_retry_descriptor:
        base: 2
        factor: 1000
        max_attempts: 5
        use_jitter: false
  

  • The node share the same identity with the main chain on all relayed chains, so the node MUST be registered on the main chain first(will automatically execute on the new-run).

  • Currently latest grouping info are relayed from the main chain to relayed chains, so the listeners of PreGrouping, PostCommitGrouping and PostGrouping are not needed.

  • Time limits of relayed chains are independent of the main chain. The way to set them is the same as the main chain.

Local Test

# unit tests
cargo test --all -- --test-threads=1 --nocapture

Start the local testnet by anvil:

# produces a new block every 1 second
anvil --block-time 1

Deploy the Controller and the Adapter contract:

cd contracts
# controller address 0xdc64a140aa3e981100a9beca4e685f962f0cf6c9
# adapter_address: 0xa513e6e4b8f2a923d98304ec87f64353c4d5c853
# user contract address 0x712516e61C8B383dF4A63CFe83d7701Bce54B03e
forge script script/ControllerLocalTest.s.sol:ControllerLocalTestScript --fork-url http://localhost:8545 --broadcast

Add operators, start the Staking pool and stake for a user and some nodes:

# nodes addresses are generated from index 10 by mnemonic "test test test test test test test test test test test junk"(anvil default)
# offset and length can be set by STAKING_NODES_INDEX_OFFSET and STAKING_NODES_INDEX_LENGTH in .env
forge script script/StakeNodeLocalTest.s.sol:StakeNodeLocalTestScript --fork-url http://localhost:8545 --broadcast -g 150

Run 3 nodes to make a group:

cd crates/arpa-node
cargo run --bin node-client -- -c test/conf/config_test_1.yml
cargo run --bin node-client -- -c test/conf/config_test_2.yml
cargo run --bin node-client -- -c test/conf/config_test_3.yml

Deploy the user contract(GetRandomNumberExample) and request a randomness:

cd contracts
# this should be executed after we have an available group as logging e.g."Group index:0 epoch:1 is available, committers saved." in node terminal
forge script script/GetRandomNumberLocalTest.s.sol:GetRandomNumberLocalTestScript --fork-url http://localhost:8545 --broadcast

The nodes should sign the randomness and one of the committers in the group will fulfill the result, check the results by cast:

# check the randomness result recorded by the adapter and the user contract respectively
cast call 0xa513e6e4b8f2a923d98304ec87f64353c4d5c853 \
  "getLastRandomness()(uint256)"

cast call 0x712516e61C8B383dF4A63CFe83d7701Bce54B03e \
  "lastRandomnessResult()(uint256)"

# the above two outputs of uint256 type should be identical