Three checkpoints in this guide
Follow the full walkthrough in order, or jump directly to one of its main sections.
Important correction: Filtering your own completed trades is a retrospective ablation, not a bias-free simulation of the trades you would have taken. It is excellent for diagnosis and hypothesis generation. It becomes decision-grade only when the rule is frozen before results are opened and checked on untouched or later evidence.
The Concept: Diagnose, Then Validate
Your trading history contains facts that a chart-only simulation often lacks: the entry you actually received, the exit you actually took, recorded costs, account context, setup label, time and execution mistakes. That makes it powerful evidence about how you traded.
But a post-hoc filter does not recreate the alternate world in which the removed trades never happened. Skipping one trade could change later capital, attention, opportunity, order size and decisions. The filtered P&L is therefore a transparent subset calculation—not proof of the causal value of a new rule.
| Method | What it uses | Best question | Core limitation |
|---|---|---|---|
| Journal ablation | Trades you actually executed | Which recorded slice differed from the rest? | Missing opportunities and changed future decisions are not simulated |
| Historical market replay | Past market path plus an execution model | How would a frozen entry/exit rule have behaved? | Fill, latency, data and look-ahead assumptions can dominate |
| Chronological holdout | Later trades hidden during rule selection | Did the frozen change survive new evidence? | One holdout can still be thin or regime-specific |
| Forward observation | Trades and eligible decisions after the rule is locked | Can the rule be followed and does the effect persist? | Slow, path-dependent and still not a controlled experiment |
Use the methods as a chain: journal evidence finds a candidate, a frozen historical or chronological test challenges it, and forward evidence decides whether the operating rule deserves to stay. The strategy-backtest workflow covers market-data simulation when you need to test entries you did not actually take.
What Your Own Trades Can—and Cannot—Answer
A defensible test starts with a question that maps every eligible trade into one of three states:
- Matching: the trade satisfies the frozen filter.
- Control: the trade is in the same account, date and currency scope but does not satisfy it.
- Unavailable: evidence is missing, ambiguous or incompatible, so membership cannot be decided.
Never force unavailable rows into the losing group or treat missing P&L as zero. The missingness rate is part of the result. A smooth curve built by quietly discarding unmapped or incompatible trades is less useful than an incomplete-looking report that exposes them.
Quick answer: Use your own history to compare one precisely defined slice with non-overlapping trades from the same scope. Treat any improvement as a candidate for another test, not as proof. Log every filter tried, preserve the unfiltered population, freeze the chosen rule and validate it chronologically.
Six Hypotheses Worth Testing
1. Exact Setup Version
Question: did trades linked to one saved setup version have different net results from other eligible trades in the same account and period? The setup identity must exist independently of outcome. If “best setup” is whichever label wins after the search, the test selected its answer.
The setup performance breakdown explains version identity, unknown labels, comparable costs and the difference between a supported candidate and a permanent edge claim.
2. Session
Question: did a predeclared session differ after timestamps were normalized to one timezone? Session labels must not drift with daylight-saving changes or be backfilled from the result. A difference can reflect instrument mix or market regime rather than time of day, so inspect composition before creating a trading-hours rule.
3. Weekday
Question: did one weekday differ under the same setup and market scope? A weak weekday in one short interval may be an event cluster, holiday effect or chance. Keep the unsplit result, declare every weekday test in the trial log and validate any exclusion on later dates.
4. Instrument or Direction
Question: did an exact instrument or long/short slice differ from comparable trades? Currency conversion, tick value, fees, volatility and opportunity count must be reconciled first. Do not interpret “short trades were worse” as a psychological diagnosis or a universal market claim.
5. Recorded Reward-to-Risk
Question: did trades above a predeclared recorded R:R threshold differ? This is valid only when the stored denominator has one definition and existed before outcome. Achieved payoff is not the same as planned R:R, and a threshold chosen after plotting every alternative is a multiple-testing search.
6. Sequence or Trade-Count Rule
Question: what happened after a frozen position in the daily sequence—for example, after the third completed trade or after a documented loss? Timestamps and same-day grouping must be complete. Removing later trades is not proof that the earlier event caused them, and a “revenge” label requires recorded evidence rather than a guessed time gap.
Use the performance-analysis framework to reconcile costs, exposure and denominators before comparing these slices.
Build Your Own Backtest From Journal Data
Step 1: Freeze the Evidence Boundary
Choose one account, timezone, start date, end date and evidence cutoff. Save the raw export before cleaning. A later correction is allowed, but it needs an audit note; silently changing rows after seeing the output rewrites the test.
Step 2: Reconcile the Baseline
Deduplicate stable trade IDs, distinguish closed from open positions, reconcile display currency and costs, and count factual trades whose monetary result is unavailable. Record setup, instrument, direction, session and weekday coverage separately.
Step 3: Write the Hypothesis Before Opening the Result
Specify the subject, exact filter, comparison population, primary metric, minimum evidence condition, expected failure modes and the action each result may authorize. “Find the best filter” is not one hypothesis; it is a search over many trials.
Step 4: Keep Matching and Control Populations Disjoint
Every eligible row belongs to matching or control, never both. Compare the same dates, account and compatible money basis. Preserve unavailable membership as its own count instead of hiding it.
Step 5: Read the Trades Behind the Summary
Open winners, losses, outliers and missing rows from both groups. A positive expectancy difference driven by one event is a different finding from a broadly distributed difference. The equity-curve comparison guide shows why final P&L alone can conceal path risk.
Step 6: Log Every Trial
Record rejected filters as well as the one that looked strongest. Reusing one history for many setup, session, weekday and threshold variations makes the maximum observed result increasingly vulnerable to chance. Do not reset the trial count because a result was inconvenient.
Step 7: Validate the Frozen Rule
Lock the rule and evaluate it on a chronological holdout that was not used for selection, or collect later trades without moving the threshold. If the evidence remains thin or the effect disappears, the correct state is unresolved—not “optimize again.” The edge-filtering protocol turns this into a reversible test with explicit rollback.
How to Read the Output
| Field | Why it matters | Failure to avoid |
|---|---|---|
| Eligible / analyzed / unavailable | Shows coverage and the denominator | Calling visible rows the full sample |
| Matching / control count | Shows the two non-overlapping populations | Comparing a slice with a baseline that still contains it |
| Net expectancy | Average recorded result after included costs | Using gross wins against net losses or mixed currencies |
| Profit factor | Gross profit divided by absolute gross loss | Treating a small loss denominator as certainty |
| Observed cumulative P&L | Exposes chronology, clusters and concentration | Calling the historical line a forecast |
| Trade-order drawdown range | Shows how ordering changed drawdown for the same recorded results | Calling shuffled histories simulated future trades |
There is no universal number of trades that makes a filter reliable. Evidence strength depends on effect size, dispersion, dependence, missingness, number of trials, regime coverage and the decision's cost. A fixed “twenty per group” rule can be too permissive or unnecessarily strict.
Reproduce the Audit in a Spreadsheet
Start with one row per closed trade and keep the raw export immutable. Useful columns are stable trade ID, account, entry/exit timestamps and timezone, setup ID and version, instrument, side, session, net P&L, native/display currency, fees, filter membership, exclusion reason and cumulative net result.
- Create a boolean matches_frozen_rule column from the written hypothesis.
- Create an evidence_state column: eligible, unavailable or excluded—with a reason.
- Sort by the original event timestamp and stable ID; never sort by P&L before building the curve.
- Calculate matching and control metrics from non-overlapping rows.
- Export the hypothesis definition, trial log, trade-level rows and both cumulative series together.
This is sufficient for an auditable spreadsheet reproduction. It is not a blind validation unless the holdout outcomes remained unopened while the rule and stopping condition were fixed.
Common Backtesting Mistakes
- Calling subtraction causal. The removed trades are known; alternative opportunities and later decisions are not.
- Selecting the winner from an undeclared search. Testing many variants and reporting one inflates false-discovery risk.
- Outcome-aware labels. Retagging losses as mistakes and winners as setups manufactures separation.
- Ignoring unavailable evidence. Missing results, timestamps, currencies or setup identity are not zero.
- Mixing rule versions. A changed setup must not borrow the history of its predecessor without an explicit compatibility argument.
- Stacking filters until the curve is perfect. Every additional dimension reduces evidence per cell and increases the search space.
- Projecting trades you never took. Own-history ablation cannot estimate missed opportunity without a separately recorded eligible-opportunity set or market replay.
Run the Evidence Loop in TSB
Ownership disclosure: Trader's Second Brain is our product. Its Strategy Backtester is built for explicit retrospective hypotheses over canonical Journal evidence—not for producing a flattering “optimized” equity curve.
You can freeze account, dates, timezone and evidence cutoff; select a saved setup/version or context filter; compare matching and other trades under one scope; inspect both trade lists; view observed cumulative P&L, composition, period context and missing-evidence limits; and optionally examine how rearranging the same recorded results changes drawdown. The verdict language stays bounded to readiness for further testing.
TSB has processed 600K+ imported trades across its import history, and its canonical registry recognizes 331 exact broker, exchange, platform and prop-export profiles. These figures describe imported trades and recognized routes—not users, a minimum sample, automatic setup labels, or trades already validated by Backtester or Coach.
Coach is where the workflow becomes genuinely powerful. It receives the signed evidence object for the exact hypothesis, can explain which observations support the result, point to missing or incompatible rows, compare the candidate with its control and propose the next bounded test. Its evidence guard prevents a thin slice, post-hoc label or attractive curve from being upgraded into a causal story or future-profit promise. That discipline is what makes the answer useful, not timid.
The Journal also provides authenticated CSV and JSON exports for your own audit. Backtester links the matching and control populations back to their exact Journal rows; a separate spreadsheet reproduction should preserve the same hypothesis and evidence cutoff.
Turn One Observation Into a Testable Rule
Import a representative history, freeze one hypothesis, inspect the trade evidence and let Coach challenge the result.
Open Strategy BacktesterImport trade history · export your audit copy · challenge it with Coach
Methodology and Evidence Limits
This September 10, 2026 fact cycle compared the baseline claims with the current local Backtester evidence contract and primary research on data reuse and backtest overfitting.
- White, A Reality Check for Data Snooping
- Bailey et al., The Probability of Backtest Overfitting
- Novy-Marx, Backtesting Strategies Based on Multiple Signals
One exported search row wrapped a backtesting question in conversational location and response directives. It is classified as DENIED measurement noise: it is not executed, protected as a heading or treated as editorial authority. The spreadsheet section is supported independently by the page's backtesting intent and the inspected TSB export contract.
We did not infer a universal sample threshold, causal psychological label, future return or missed-trade outcome. The article contains no exact external firm or program that materially determines the decision, so no provider/catalog component is added. It preserves Article and BreadcrumbList, keeps FAQPage tied to visible FAQ content, and adds no artificial ItemList, Review, Rating or Product schema.
The Bottom Line
Your actual trades are not a loophole around backtest bias. They are a higher-fidelity record of what happened, which makes them exceptionally good for finding precise, inspectable hypotheses.
The winning workflow is not “remove whatever hurt and trust the prettier curve.” It is: reconcile the history, freeze one question, expose missing evidence, compare disjoint populations, log every trial, inspect the rows and validate the rule on untouched or later trades. TSB makes that loop fast and traceable; Coach makes the result easier to interrogate without pretending the past has already proved the future.
Disclosure: Trader's Second Brain is our product. This guide is educational and does not promise that a historical filter, backtest result or Coach explanation will improve future performance.
