Data Feed Handling

Data Feeds and Data Sources

This chain supports low frequency as well as higher frequency data updates. It is not meant for billions of ticks per second, but should easily handle a few million ticks per second during market events.

The ideal data feed comes directly from the data source, such as exchanges or any other marketplace where the data is created (trade, weather station, …)

Data Feed Problems

There are a number of problems that can arise, which need to be addressed:

1. What data provider gets rewarded?
2. Is it stake based or is it quality based? What determines the “best quality”?
3. When do data providers get slashed for bad data?
4. Piggybacking on data feeds: How to find out if data is merely a delayed feed of someone else?
5. Data Flooding: How to prevent a forced average by flooding a specific rate with data sources?
6. Malicious validators: What if a validator maliciously
7. Contract piggybacking relay: A Contract pays for the feed and relays the data to other contracts.

Data Provider Rewards and Slashing

In a first iteration we will provide commission based on the nearest stake-weighted average data provider with a linear decay. E.g. there are 5 data providers for EUR/USD rate. The weighted average for the price based on the stake is

That is an example of calculating the ground truth for the price and considered the “TruePrice”. Commission is paid based on the difference between the price given by the data provider and the weighted average price with a decay between them and a slashing function if the price diverges too far.

Ideally, the price difference, where rewards and slashing are coming into effect, are calculated dynamically based on the amount of data providers, asset volatility etc.

The “TruePrice” can also be dependent on the time decay since the data provider joined, the reliability of the price feed over time (how many times was the price close to the average price), or other metrics.

This is something that needs to be tested and addressed in a test environment.

Piggybacking on Data Feeds

Another problem is the piggybacking on data feeds. That is, someone is listening to incoming ticks and then providing the ticks with a slight delay back into the system.

It might occur, but not necessarily be problematic if rewards vs. slashing is done right.

Data feeds will be encrypted so that only validators who are in charge of creating the next block will be able to read the tick data. This means, the data from someone who is piggybacking on the data feed is already delayed by, at least, one block, or one second. If a market event happens, the data might be already so far off from all other data providers who provide the same tick, that slashing will be far more impactful as any potential commission for being close to the average price.

This is something that needs to be tested and addressed in a test environment.

Data Flooding

Another problem is data flooding.

That is, someone registers itself as the majority of data providers, controlling which will be the average price. This is either by having the largest stake in a stake-based weighted average reward system, or by having the biggest amount of data sources providing the same price feed over and over again.

To circumvent this, a potentially large enough stake is necessary to become data provider in the first place. This is something that needs further active research.

Contract Piggybacking Relay

Another problem is contract piggybacking and relaying the data to other contracts.

Consider a Smart Contract A, which then relays the tick data to Contract B, C and D. Instead of getting rewarded for the price tick 4 times, the data provider now only gets rewarded one time.

To circumvent this, its might be possible to pay a commission based on the used gas amount during that transaction in addition to a baseline commission. This way, even during a piggybacking, a data provider is financially rewarded.

This is something that needs further active research.

Signing of Data Feeds

To verify the authenticity of data feeds, the data needs to be signed. That is, each data point needs to be signed using the same signature mechanism as a usual Ethereum transaction has, including an ever increasing nonce (which could be the timestamp in microseconds).

This allows the verification that the data feeds origin can be trusted and nobody can inject malicious ticks in the name of another data provider. It’s also possible to directly recover the address of a tick data provider and forward the commission gas fee to the staking contracts balance with the data provider as the beneficiary.

Encryption of data feeds

Furthermore, the data feeds are encrypted with the public key of the current set of validators. This means, the data is fully private and invisible for the rest of the network until it is needed inside a smart contract as a callback. At this point the data is paid for automatically.