The publicly described study method
The FAQ tells aspiring traders to look back at all listed US stocks, check the charts of every mover as far back as possible, and build a chart database over at least a few hundred hours. The episodic pivot post describes going through all six to seven thousand US stocks in TC2000 with no filters on the monthly chart, picking out big moves, and then studying them on intraday charts and in the news. The three-setups article describes the resulting database as thousands of examples built over seven to eight years and more than 1,000 hours, figures that are self-reported.
The same post is candid about the limits: none of the setups work all the time, and the failure rate is high, even though the risk-to-reward is good.
Survivorship bias
A scan of stocks listed today leaves out every company that was delisted, acquired or went bankrupt. Some of those had setups that looked identical to the winners and then failed. A database built only from today's listings therefore holds a higher share of successes than the market actually produced. The method above, starting from stocks that made big moves, adds a second filter on top: it selects on the outcome.
Hindsight bias
On a finished chart the breakout is obvious, the base looks tight and the volume looks decisive. In real time, the same chart sat next to dozens of similar ones that went nowhere. Hindsight makes setups look cleaner and more frequent than they were, and makes the stop that would have been hit seem unlikely.
Reducing both (our methods)
The methods below are this site's own suggestions.
- Sample by date, not by outcome. Pick a past week and record every stock that met your criteria that week, then look at what happened.
- Hide the right side. Review charts with everything after the trigger day covered, decide, then reveal.
- Include the dead. Where your data allows, include delisted stocks in historical scans.
- Log failures with the same fields. See studying failed breakouts.
- Count, then judge. Write down how many examples you have before drawing any conclusion.
A forward log
The cleanest data is recorded before the outcome exists. Saving each weekly leader scan and every candidate you considered, with the date, creates a sample that neither bias can touch. After a few months, compare it with your historical database; the gap between the two is a measure of how much the biases were inflating the picture. The database itself is covered in build a chart database.
The bias check
- Does the sample include examples that failed?
- Was it drawn by date or by outcome?
- Could delisted stocks be missing?
- Were decisions made with the right side of the chart hidden?
- Is there a forward log to compare against?