Three checkpoints in this guide
Follow the full walkthrough in order, or jump directly to one of its main sections.
Filtering can reveal where your results differ, but a profitable-looking slice is not automatically a trading edge. The same journal can show one result by setup, another by session, and a spectacular number after enough combinations are tried. The useful question is not “Which filter makes my profit factor look best?” It is “Which pre-defined trading condition has a plausible mechanism, survives costs, and still works on observations that were not used to find it?”
This guide gives you a controlled filter cascade for answering that question. It preserves the practical setup → session → day workflow, but replaces universal trade-count and profit-factor thresholds with a decision log, uncertainty checks, and chronological validation.
Method checked September 9, 2026 against the NIST guidance on multiple comparisons, Bailey et al.'s Probability of Backtest Overfitting, and the CFA Institute's backtesting and simulation framework. These sources support the safeguards; they do not validate any illustrative trading result below.
Quick answer: define the category and the decision before viewing its P&L, inspect one broad dimension first, keep a record of every slice tested, and reserve later trades as a chronological holdout. Promote a filter to a trading rule only if its net result, risk, and frequency remain acceptable outside the discovery sample.
Define “Actual Trading Edge” Before You Filter
For this exercise, an edge is a repeatable rule with positive expected value after material trading costs and tolerable risk. It is not the highest historical profit factor in a table. A filter is simply a condition—such as setup tag, session, direction, or instrument—that lets you compare subsets.
Filtering can do three useful jobs:
- Describe concentration: show where past gains and losses occurred.
- Test a mechanism: check a reasoned claim such as “this setup needs the liquidity available during its intended session.”
- Create an operational rule: define when the setup is eligible for future trades.
Only the first job is completed by an attractive historical table. The second needs a credible explanation; the third needs data that were not used to choose the filter. For the underlying expectancy calculation, use the expectancy formula guide and keep the result in the same unit—currency, points, or R—across every subset.
Why Aggregate Stats Can Hide Useful Differences
An aggregate is not false; it answers a broader question. If one setup is taken in two different market conditions, the combined result describes the mixture. It does not tell you whether either condition behaves differently enough to deserve its own rule.
Consider this hypothetical discovery sample:
| Pre-defined setup tag | Trades | Net R | Expectancy | Profit factor |
|---|---|---|---|---|
| Pullback | 46 | +11.5R | +0.25R | 1.46 |
| Breakout | 39 | −3.9R | −0.10R | 0.86 |
| Untagged | 27 | −5.4R | −0.20R | 0.71 |
Illustration only. The numbers are fabricated to demonstrate the workflow, not presented as TSB customer results, a benchmark, or an expected outcome.
The table supports a new question: does the pullback definition identify something repeatable, or did this sample favor it by chance? It does not support deleting every other trade immediately. First check tag consistency, costs, outliers, and a later holdout period. The setup performance breakdown explains the descriptive Layer 1 view in more detail.
The Filtering Cascade: From Broad to Specific
The cascade is a complexity control, not a law that three layers are always safe. Each additional split creates smaller cells and another opportunity to select a lucky result. Stop when the remaining data cannot support the decision you intend to make.
Layer 1: Setup Type
Start with tags that existed before you saw the outcomes. Confirm that each tag has a written definition and that “untagged” remains visible. Report trade count, net result after known costs, expectancy, profit factor, average win and loss, maximum drawdown, and trade frequency. A single ratio can be dominated by one outlier.
Layer 2: Session or Market Condition
Add a second dimension only when there is a reason it could affect execution or opportunity. Session may matter because liquidity, spread, scheduled events, or the setup's intended opening process changes. Compare the same setup across mutually exclusive session definitions and preserve timezone and daylight-saving rules. The session-performance guide shows how to avoid mixing local clock labels with exchange time.
Layer 3: Day, Direction, or Instrument
Choose the third dimension from the hypothesis—not from whichever column looks best after the first two tables. A day-of-week filter needs a market or process reason; direction needs a definition that handles flat and hedged exposure; instrument needs comparable cost and sizing treatment.
Your Filtered Edge Is Still a Candidate
Name the resulting subset as a candidate rule and freeze it. Example: “Pullback A, European morning, major FX pairs, as defined in plan version 3.” Record the exact inclusion logic, exclusions, discovery dates, number of alternatives tested, and decision threshold. The next observation must be classified using that frozen definition.
Building Your Filter Stack: A Reproducible Process
The 7-Step Filter Process
- Freeze the vocabulary. Write setup, session, direction, instrument, and rule-compliance definitions before opening the results.
- Choose the decision. State what would change: remove a setup, restrict a session, collect more data, or make no change.
- Separate time periods. Use an earlier block for discovery and keep a later chronological block untouched for validation.
- Run the broad comparison. Start with one dimension and retain every category, including zero-trade and untagged states.
- Count the searches. Log every filter and variation tested. A winner selected from 30 attempts deserves more skepticism than one pre-specified comparison.
- Check a metric bundle. Review net result after costs, expectancy, drawdown, outliers, trade count, and frequency—not profit factor alone.
- Validate without tuning. Apply the frozen rule to the holdout and then to future observations. If you alter it, start a new version and a new validation record.
There Is No Universal Sample-Size Floor
A fixed “20 trades is enough” rule ignores payoff dispersion, serial dependence, win/loss asymmetry, costs, and how many alternatives were tested. Twenty nearly identical small outcomes and twenty outcomes dominated by one large win do not carry the same uncertainty. Show the count, full distribution, and outlier sensitivity. If your software supports resampling, preserve time dependence where relevant; do not treat correlated trades as independent evidence merely because they occupy separate rows.
Small cells can still be operationally useful as a data-collection flag. They are not strong evidence for scaling risk or deleting a strategy. “Needs more observations” is a valid decision.
8 Quick Filter Presets—Turn Each Into a Hypothesis
These prompts are a starting library. Do not run all eight and quietly report only the winner. Select the ones justified by your plan, write the expected direction, and log every test.
| Candidate filter | Reason it might matter | Definition to lock |
|---|---|---|
| Long vs short | Different trend or borrow conditions | Exposure direction and hedged trades |
| Trade number in session | Fatigue or changing opportunity set | Session reset and partial fills |
| Plan-compliant vs unplanned | Eligibility rules should change selection | Checklist version and exception handling |
| Session | Liquidity and execution conditions differ | Exchange time, overlap, daylight saving |
| Day of week | Scheduled process or event exposure | Holiday and overnight attribution |
| Post-loss entry | Decision quality may change after a loss | Time window, session boundary, open positions |
| Risk band | Sizing may change execution or behavior | Planned risk, realized risk, account basis |
| Instrument | Costs and setup mechanics differ | Contract mapping and roll treatment |
How to Use the Preset Library
Before calculating, make a one-row test record: hypothesis, categories, metric, decision threshold, discovery window, validation window, and why the relationship should exist. After calculation, add the result even if it is boring. This prevents the research trail from becoming a gallery of winners.
Read the Discovery and Validation Results Together
| Discovery | Validation | Practical response |
|---|---|---|
| Favorable discovery | Favorable with usable frequency | Consider a bounded rule change; continue monitoring |
| Favorable discovery | Weak or opposite | Do not promote; inspect overfit, regime, data, and cost assumptions |
| Weak discovery | Favorable | Keep collecting; do not rewrite the hypothesis after one reversal |
| Weak discovery | Weak | Retain the broader rule or investigate strategy definition and data quality |
Also compare equity paths, not only endpoints. Two subsets can finish at the same net result with very different drawdown and concentration. The equity-curve comparison workflow is useful here, provided the selection rule was fixed before you judged the curves.
Common Filtering Traps
- Inconsistent tags: the category changes meaning during the sample.
- Missing trades: rejected or skipped setups are absent, so the journal cannot measure the opportunity set.
- Cost mismatch: gross results are compared across instruments or venues with different friction.
- One-outlier dependence: a candidate loses its appeal when one trade is removed.
- Rule and regime changed together: a new filter receives credit for a market change or a revised strategy.
- Volume blindness: a higher ratio leaves too few opportunities to meet the actual objective.
Trader's Second Brain is our product. Its role in this workflow is to keep imported trades, tags, rule versions, and filtered views in one evidence trail—not to certify that a selected subset is an edge. The canonical source registry currently recognizes 330 structured broker, exchange, platform, and prop-export profiles. Confirm your exact route in the live import directory, validate a representative file, and reconcile counts, timestamps, quantities, fees, and net result before filtering.
A spreadsheet or another journal can run the same method if it preserves definitions and every tested slice. If you want the TSB workflow, Full Access includes filtering and review tools and offers a lifetime-access route; verify the current terms rather than relying on an amount copied into an article.
Turn a Validated Filter Into One Bounded Change
- Write the old and proposed eligibility rule side by side.
- Specify when the new rule begins and which trades remain eligible.
- Keep risk unchanged while testing the selection change.
- Log eligible trades you skip as well as trades you take.
- Review on the pre-committed horizon; do not stop early because the first results look good or bad.
- Accept, reject, or extend the test in writing, then version the rule.
The journal must preserve enough detail to reconstruct the decision. The trading-journal build guide covers the minimum fields for setup, planned risk, execution, context, and review.
Who Should Delay a Filter Decision
- Changing strategy definitions: stabilize and version the rule before attributing differences to a filter.
- Inconsistent or missing data: reconcile imports and tags before interpreting subsets.
- Tiny candidate cells: label them for collection; do not scale or cut on a fragile estimate.
- No untouched period: treat the result as exploration and reserve future trades for validation.
- Algorithmic optimization: use a formal backtest and model-validation process that handles data leakage, search breadth, costs, and time dependence.
Methodology Note
This framework is intentionally conservative. NIST's multiple-comparison guidance supports defining comparisons in advance or using procedures appropriate to post-hoc comparisons. CFA's backtesting material emphasizes an explicit hypothesis, realistic process, rolling or out-of-sample evaluation, bias controls, and attention to structural breaks. Bailey et al. show why selecting a historical winner from many trials can overfit.
None of those sources creates a universal minimum trade count, profit-factor cutoff, layer limit, or validation duration for a discretionary trader. Those depend on the decision, payoff distribution, dependence, costs, number of trials, and acceptable error. This guide therefore uses transparent counts and frozen validation rules instead of declaring a small historical cell “proven.”
Final Verdict: Filter for a Testable Rule, Not a Flattering Number
Start broad, preserve every category, and use the smallest number of filters needed to test a plausible mechanism. Report the metric bundle and the search count. Then freeze the candidate and ask a later period to confirm or reject it.
A good outcome is not always a narrower trading plan. It may be a cleaner tag definition, a data-quality repair, or a decision to collect more observations. The discipline is valuable precisely because it allows “not enough evidence” to beat an attractive but fragile backtest.
