Using Datasource

Building a topic

In the RuntimeConfig struct you may notice that there is a parameter that accepts a List[str] called datasource_topics. This parameter is how you would 'inform' the runtime as to what datasource you'd like to receive.

Refer to this to see the currently available providers and endpoints.

An example of using this parameter is shown below.

permutation = Permutation(RuntimeConfig(
            ...
            datasource_topics=[
              "coinglass|1m|funding",
              "coinglass|5m|indicator/open_interest_ohlc?ex=Binance&pair=BTCUSDT&interval=h4"
            ],
            ...
        ))
 
hyper_parameters = {}
async def start():
    await permutation.run(hyper_parameters, Strategy)
 
asyncio.run(start())

Notice the structure of the provided datasource_topics in the example above, the format follows a schema.

⚠️

The {interval} value in the schema below does not affect anything, however, it is still required to be included in the topics until further notice. This value will be removed and deprecated in an upcoming future Cybotrade update

{provider}|{interval}|{endpoint}

Currently the available intervals are based on the candle intervals this interval refers to amount of time elapsing before invoking the data. This is not related to any form of interval or window that you might encounter when going through provider documentation, and should not be treated as such.

The interval mentioned in the topic is only relevant in Live and can be assigned to any value in Backtest.

The power of datasource lies in the endpoint field of the schema, this allows you to retrieve data from the providers based on the parameters you desire.

Coinglass Topics

Example of using coinglass endpoint

Note that for all topics, you only have to use the endpoint denoted after '...api/'. As shown in the image above, avoid the green-lined value and include the red-lined value. This topic's '{endpoint}' will be

futures/openInterest/ohlc-history...

Looking at the example provided earlier, we can see that there are two endpoints

futures/fundingRate/exchange-list

as well as

futures/openInterest/ohlc-history?exchange=Binance&symbol=BTCUSDT&interval=4h

Notice that futures/fundingRate/exchange-list does not have any parameter passed in as a query string, whilst the open_interest_ohlc endpoint does.

It is recommended that users do not use the start_time, end_time and limit parameters as the cybotrade runtime will hijack the topics and manage it by basing it off of the RuntimeConfig's start_time, end_time and data_count values, this is only relevant in Backtest. In Live, the topics will be hijacked with the current time when strategy is started and the data_count value.

If a parameter and/or endpoint is passed in that the provider does not recognize, the runtime will reject and promptly end.

You may refer here (opens in a new tab) for coinglass documentation.

Looking back at the open_interest_ohlc endpoint, have a look at the coinglass documentation (opens in a new tab) for it.

Notice that in the example, the query params should align with the required parameters in the documentation.

futures/openInterest/ohlc-history?exchange=Binance&symbol=BTCUSDT&interval=1d

coinglass example

As long as the provider's documentation mentions that the endpoint accepts the parameter, it can be included in the topic.

CryptoQuant Topics

Example of using cryptoquant endpoint

Looking at the image above, you will only have to include the path provided from cryptoquant, into the {endpoint`} value. For an example:

cryptoquant|1m|btc/exchange-flows/inflow?...

The parameters available for your desired endpoint can be found in cryptoquant's documentation here (opens in a new tab).

cryptoquant example

Based on CryptoQuant's API specifications it is highly recommended to specify the desired window parameter else it defaults to day when running.
The to, from and limit parameters will be handled by the cybotrade runtime so this can be ignored.
The format parameter here should be entirely avoided and not be included when creating the datasource topic.

An example using this topic would be as such:

cryptoquant|1m|btc/exchange-flows/inflow?exchange=binance&window=block

Accessing the data in code.

In order to access the data you retrieve, make sure you refer to the models. Shown below is a simple use of the on_datasource_interval api. This is a very simple strategy that places an order if the data closes higher than when it opened.

Do note that this example is assuming that the datasource_topic being used is open_interest_ohlc?ex=Binance&pair=BTCUSDT&interval=h4

async def on_datasource_interval(self, strategy, topic):
 
  # refer to datasource models
  ohlc_history_open = float(super().data_map[topic][-1]["open"])
  ohlc_history_close = float(super().data_map[topic][-1]["close"])
 
  # if the candle closed higher than when it opened, we open a short order
  if ohlc_history_open < ohlc_history_close:
    strategy.open(OrderSide.Sell, quantity=0.01, symbol=Symbol(base="BTC", quote="USDT"))

You are also able to retain the data retrieved by storing it within the Strategy class.

class Strategy(BaseStrategy):
    datasource_close = 0.0
 
    async def on_candle_closed(self, strategy, topic, symbol):
        candle_close = float(super().data_map[topic][-1]["close"])
        if candle_close > self.datasource_close:
          # place order or close position 
        ...
 
    async def on_datasource_interval(self, strategy, topic):
      self.datasource_close = float(super().data_map[topic][-1]["close"])

Validate Cybotrade API Models