We run a replay engine, so this page is written against our own product: where replaying a channel's signals is real evidence, where it stops, and which of the documented ways to flatter a track record apply to us.

What is a backtest, and how is a signal replay different?

A channel that shows you a track record is asking you to trust a simulation. A backtest takes a trading rule and runs it over past prices to see what it would have earned. A signal replay starts from calls somebody already published, with the entry, targets and stop named at the time, and resolves each one against the exchange candles that followed. The first tests an idea you are free to adjust. The second tests a claim already fixed in a timestamped message.

That difference matters because the classic criticism of backtesting is about the freedom to adjust. When a rule can be tuned until the curve looks good, the result describes the tuning rather than the market. A replay of published signals removes most of that freedom, since the levels are whatever the channel posted and nobody gets to move them afterwards.

Most of this page is about what survives that difference. Several of the worst problems in backtesting apply to a replay too, and a few of them apply specifically to us.

What can a replay actually prove?

A replay settles one question well: what would have happened to somebody who took the published call exactly as written. The channel controls its screenshots, its marketing and its win-rate claim, and it does not control the candles its old posts get compared against. That is the entire evidential value of the exercise.

As of 7 August 2026 we have parsed 3,227 signals from the channels in our index and resolved 3,167 of them. Under the one-take rule, 1,554 of those outcomes (49%) end as cancelled, meaning the market ran past the target before the published entry could fill. Under the separate as-published replay, which uses each channel's own stop, 1,653 outcomes cannot be resolved at all, because no stop was ever named. Those are two different rule sets applied to the same signals, and the counts are not additive. Both are described on the Methodology page.

A replay can settle A replay cannot settle
Whether the entry was reachable after publication Whether any subscriber actually traded it
Whether price hit the stop or the target first, subject to candle granularity What a follower's fill price would have been
Whether a channel's claimed record matches its own posted levels Whether the channel will keep doing this
How many posts contained a complete, checkable trade Whether the next call is a good one

The left column is worth having. The right column is where most readers want the answer to be.

Why does the number of attempts decide the result?

Because a track record is a maximum rather than an average. Search enough variations and one of them looks skilful on pure noise, which is the finding behind the best-known paper on the subject. Bailey, Borwein, Lopez de Prado and Zhu showed that trying just ten configurations on a random walk is expected to produce an in-sample Sharpe ratio of 1.57 whose true out-of-sample value is zero. The Sharpe ratio there is simply a return-per-unit-of-risk score, and a value above 1 is the kind of number a fund would advertise.

The same paper puts a ceiling on how much searching a dataset can support. With five years of data, no more than 45 independent configurations should be tried, "or we are almost guaranteed to produce strategies with an annualized Sharpe ratio IS of 1 but an expected Sharpe ratio OOS of zero". IS and OOS there are in-sample and out-of-sample: the run used to find the strategy, and the run that tests it. Their conclusion about disclosure is the practical part:

"A researcher that does not report the number of trials N used to identify the selected backtest configuration makes it impossible to assess the risk of overfitting." — Bailey, Borwein, Lopez de Prado and Zhu, Pseudo-Mathematics and Financial Charlatanism, Notices of the AMS, May 2014

They go further than the familiar disclaimer about past performance: "When investment advisers do not control for backtest overfitting, good backtest performance is an indicator of negative future results." A number that survived heavy searching is evidence of searching.

This applies to a channel even though the channel never ran a backtest. Its published record is the survivor of every format, symbol set and posting habit it tried before, and none of those attempts are visible in the feed. The unstated count of attempts is there whether or not anyone computed it.

What does our own rule set assume?

Four choices, each of them ours, each of them moving the number. We publish them because a replay whose assumptions are hidden is worth about as much as a screenshot.

Our choice What it does to the result
One take-profit at a 2% price move A low bar. It makes reaching a target common and says nothing about how far a winner ran
The channel's own stop, or a 10% fallback when none was published The fallback is wide, so stopless channels are stopped out less and score higher
A 30-day window to resolve Trades still open at the end count as not reaching the take, which penalises slow ideas
Fees charged, one venue's candles Understates cost for anyone trading elsewhere or paying funding on a long hold

The fallback stop most flatters the channels with the least discipline. Under the one-take rule, signals scored with our fallback show a hit rate of 81.3% as of 7 August 2026, against 60.5% where the channel published its own stop. Only 390 of those 1,639 fallback outcomes ever produced a scored verdict at all. A wide stop is rarely touched. That is examined in Why half of all crypto signals come without a stop-loss.

Three of the six documented ways to produce a flattering record apply to us directly. We selected a rule set with the benefit of hindsight. We have not published how many rule variants we considered before settling on this one. And our index is a composite we assembled: 50 channels we chose to track, of which 17 produce every number we publish. Anyone applying this article's standards to this article should start there.

Where does a one-minute candle stop telling the truth?

A candle records four prices and throws away the order they happened in. When both the stop and the target sit inside the range of the same minute, no engine reading that candle can know which came first, so it has to assume — and the assumption is not a detail. On some datasets it drives the answer.

The open-source backtesting project Freqtrade documents its own assumptions plainly, including that within a candle it treats the low as happening before the high when checking a stop. It also states the limit outright: "However, backtesting will never replace running a strategy in dry-run mode" (Freqtrade documentation). A user issue filed against the same project in March 2026 reports the consequence. Because stop exits fill at the candle extreme, a 25% stop and a 40% stop produced identical results, which would mean the fill convention decided the outcome and the strategy did not.

Our corpus contains the category most exposed to this. Under the as-published replay, 331 outcomes hit at least one target and then stopped out, against 293 that walked the full ladder of targets, as of 7 August 2026. Those 331 are exactly the trades an optimistic intrabar assumption would reclassify as clean wins.

A formal treatment exists: Low, Maier-Paape and Platen proposed a correctness test for backtest engines in Correctness of Backtest Engines, opening with the observation that platforms struggle with precisely this problem.

What costs does a replay leave out?

Everything that happens between a decision and a fill. A replay charges what it knows about, and what it knows about is smaller than a real cost sheet.

Funding is the clearest example on perpetual contracts. It is a periodic payment between the two sides of the contract, settled by Binance every eight hours by default at 00:00, 08:00 and 16:00 UTC, and a position only pays if it is open at those timestamps (Binance funding rate documentation). A call that reached its target eleven days later crossed roughly thirty-three funding intervals at a market-determined rate. A replay that ignores funding flatters long holds, particularly crowded ones.

Market impact is the one nobody can model from candle data. A replay assumes the follower's order is the only one arriving, while a channel with a hundred thousand subscribers is moving the book it is calling. On the Quant Stack Exchange, the recurring practitioner answer to modelling slippage from candle data is that it cannot be done without microstructure or intraday order flow, and the most-upvoted advice about live results is blunt:

"Optimism: Your actual profits will likely be only a fraction of your simulated profits. Are you assuming otherwise?" — top answer to What are the key risks to the quantitative strategy development process?, Quant Stack Exchange

Which price, on which exchange?

There is no single price to replay against. Venues diverge, and in stressed markets they diverge enormously, so the exchange whose candles an engine reads is itself an assumption about the answer.

The clearest documented case is the crash of 10-11 October 2025. Forbes reported that 31 tokens on one major venue printed more than 100% below the next-worst exchange, with a further 18 trading 50-75% below peers, while only 53 tokens stayed within 1% across venues (Forbes, 21 October 2025). The same report describes traders unable to place reduce-only orders for more than an hour and a half during the event.

Two consequences follow for anyone reading a replayed track record. A signal scored on one venue's candles can be a win there and a loss elsewhere on the same day. And the exit a replay assumes was available may not have been placeable at all, because a backtest has no way to represent an exchange that is not accepting orders.

Why is survivorship the bias you cannot see?

Because the evidence it removes is not in the dataset to be examined. Everything measurable comes from things that lasted long enough to be measured, and the ones that did not are gone without leaving a number behind.

In crypto specifically, the size of that effect has been estimated. Ammann, Burdorf, Liebi and Stockl studied 3,904 coins between 2014 and 2021 and measured the survivorship and delisting bias two ways. Weighted by market value it comes to 0.93% per year; with each coin counting equally it comes to 62.19%. The second figure is the one that matters for anyone treating small tokens as interchangeable.

The channel version has the same shape. Of the 50 channels in our index as of 7 August 2026, 17 publish signals structured enough to parse at all, and only those 17 can ever contribute a number. Channels that blew up, stopped publishing or deleted their history do not appear as bad ratings. They appear as nothing, so every hit rate we publish is a hit rate among survivors.

Sullivan, Timmermann and White made the wider point in their study of trading rules on a century of Dow data. Data snooping, they noted, "need not be the consequence of a particular researcher's efforts": it accumulates across everyone who ever tried a rule and abandoned it quietly. The channels still publishing today are the survivors of that same process.

What do regulators require of hypothetical performance?

Where these rules apply, they demand specific disclosure, and none of them applies to a Telegram channel posting spot-crypto calls. That is worth stating precisely, because the standards are useful even when they are not binding.

Rule What it covers Reaches a signal channel?
CFTC Regulation 17 CFR 4.41 Commodity pool operators and trading advisors presenting simulated results for commodity interests, defined at 17 CFR 1.3 as futures, swaps and certain leveraged retail transactions No for spot crypto calls; possibly yes for regulated futures
NFA Compliance Rule 2-29 and Interpretive Notice 9025 NFA member firms communicating about their commodity interest business No, for the same reason
SEC Marketing Rule, 17 CFR 275.206(4)-1 Registered investment advisers advertising advisory services regarding securities Normally no, since most spot crypto assets are not securities

The CFTC's required wording is the plainest statement of the problem anyone has written down. Simulated results "do not represent actual trading", may "have under-or over-compensated for the impact, if any, of certain market factors, such as lack of liquidity", and "are designed with the benefit of hindsight".

The NFA's interpretive notice opens by explaining why the rule exists: "Over the years the use of hypothetical performance results has repeatedly produced misleading promotional material." It requires the disclaimer to be shown as prominently as the results, in type at least as large, and it bans hypothetical results entirely for a program that already has three months of actual ones.

Enforcement of the equivalent securities rule is active. In September 2023 the SEC charged nine investment advisers over hypothetical performance in advertising, with $850,000 in combined penalties. Such advertisements, the regulator said, "may present an elevated risk for prospective investors whose likely financial situation and investment objectives don't match the advertised investment strategy".

How is a flattering track record manufactured?

Six methods, five of them documented by regulators who watched firms use them:

  1. Choosing the window. The CFTC warns buyers of trading systems to "be alert for the possibility that the system promoter manufactured results by selecting historical trades that would have yielded the greatest returns".
  2. Calling a simulation live. The NFA records members who "attempted to disguise hypothetical performance results by referring to the performance with terms such as 'live' or 'real-time' results".
  3. Burying the caveat. From the same notice: results in "large, bold-face print while the disclaimer can be read only with a magnifying glass".
  4. Showing the simulation, hiding the reality. Members "touted dramatic hypothetical profits without revealing that their actual performance is much worse".
  5. Assembling a composite after the fact. A record built by selecting which advisers to include, once their results are known, stays hypothetical however real each component was. Our index is a composite in this sense, and so is any ranking site's.
  6. Not saying how many attempts it took. The academic point above, which applies to every published win rate including the ones we compute.

Four of these can be checked against a channel's feed in a few minutes: whether the record is a simulation or a claim about real trades, whether losses are shown beside the wins, whether the caveats are visible at all, and whether the record's start date coincides with the beginning of a good run.

So what is a verified record worth?

It is necessary and not sufficient. A channel that fails replay has been shown to be wrong about its own claims, which is strong evidence against it, though even that verdict depends on our rules being right. A channel that passes has been shown to have published complete, timestamped calls that resolved the way it said, which is real information about the past and no forecast at all.

Our own corpus makes the limit concrete. Under the as-published replay, the 1,514 outcomes carrying a result average -0.65% each as of 7 August 2026, with 475 finishing in profit against 728 in loss. A record can be genuine and still describe a losing proposition, which is the subject of Win rate: why 90% accuracy can still lose you money.

So the honest claim for any replayed track record, ours included, is narrow: these calls were published at these times, and this is what the market did afterwards under this specific set of rules. Everything past that is interpretation. Where the rest of the pre-purchase checks sit is set out in How to verify a crypto signal channel before you pay, and what the unfillable half of our corpus looks like in detail is in Half the trades never happen.

Current channel ratings are at Signal Providers, and what our verification does and does not claim is set out in the Verification Guidelines. Nothing here is financial advice.

Sources