For most retail traders with multi-setup approaches, removing the worst 1-2 setups from past performance increases total P/L by 30-150% while cutting maximum drawdown by 30-60%. The math is straightforward: most traders' overall results are the sum of strongly profitable setups offset by strongly unprofitable ones. The aggregate net P/L hides the wide divergence between best and worst setups. When the worst contributors are removed in counterfactual analysis, the trader doesn't gain from new strategy — they gain from subtraction. The phrase "do less, earn more" sounds like cliché until you see your own equity curve recomputed without the FOMO trades and counter-trend reversals you knew weren't working.
This guide covers the impact analysis methodology (what counterfactual setup removal actually does), a worked example showing 240 trades across 4 setups with the math of removing the worst two, the equity curve transformation that makes the data viscerally obvious, the decision framework for which setups to actually cut versus investigate further, the post-cut psychological adjustment most traders underestimate, and the survivorship-bias trap that can make naive impact analysis produce misleading conclusions.
Counterfactual analysis methodology is standard in applied performance research. The 80/20 distribution observed in setup performance reflects the Pareto principle documented across financial markets, business, and many other domains. Worked example figures illustrate typical patterns observed across the TSB journal user base; individual trader results vary substantially based on number of setups, sample size, and strategy stability. Past performance does not predict future results — counterfactual analysis describes what would have happened, not what will happen.
The principle in one sentence: Most retail traders' net P/L is the difference between strongly profitable setups and strongly unprofitable ones. The aggregate hides the divergence. Counterfactual analysis (impact analysis) makes the divergence visible by recomputing performance with selected trades removed — turning a "do something different" problem into a "stop doing something specific" problem.
The What-If Question Every Trader Should Ask
Every trader has a nagging feeling about one or two of their setups. The reversal play that "sometimes works great but sometimes gets crushed." The news trade that "is usually profitable except when it isn't." The FOMO entry that "I know I shouldn't take but sometimes it just runs."
Why Memory Lies About These Trades
Traders remember winning trades better than losing ones — a documented memory bias. The occasional 3R FOMO winner is vivid; the eight 1R FOMO losers between it are blurred together. Self-assessment without hard data systematically overestimates the contribution of marginal setups. The only reliable diagnostic is computing actual performance with and without the setup.
What Impact Analysis Actually Does
Impact analysis (counterfactual analysis) takes your real trade history, removes specific trades or setups, and recomputes the resulting metrics: win rate, profit factor, net P/L, max drawdown, and equity curve shape. The simulation uses your actual past trades — only the variable of "did I take this set of trades" changes. Everything else (market conditions, timing, your other trades) stays identical, making the comparison rigorous rather than hypothetical.
The Typical Result
Across observational data, traders running impact analysis on their worst-performing setup discover that removing it would have increased total P/L by 30-80% and reduced max drawdown by 30-50%. Removing the worst two setups frequently doubles or triples net profit. The trader hasn't found a new strategy — they've identified what to stop doing. Subtraction is the highest-leverage adjustment most traders never make.
How Impact Analysis Works
The 4-Step Process
- Start with complete trade history — all trades, all setups, all accounts (or specific account scope if preferred).
- Identify the setup to test — typically the one with the lowest profit factor from your setup performance breakdown. Setups with profit factor below 0.7 across 40+ trades are prime candidates.
- Remove those trades from the dataset — recalculate everything: win rate, profit factor, net P/L, drawdown, equity curve.
- Compare the two versions — original performance vs performance without the removed setup.
Why It's Rigorous, Not Hypothetical
This isn't predictive modeling. It uses actual past trades. The only variable is whether the removed setup's trades exist in the data. Market conditions, timing, your other trades — all stay identical. The output describes what your performance would have been, not what your future performance will be — but past performance correlates strongly enough with future performance for setups with 50+ trade samples that the analysis is actionable.
Important distinction: Impact analysis describes counterfactual past, not predicted future. The assumption is that a setup losing money over 50+ trades is likely to continue losing money on similar future trades. That assumption holds for most discretionary strategies on stable market conditions; it fails when markets undergo regime changes that may favor previously-losing setups.
A Worked Example: 240 Trades, 4 Setups, 6 Months
A realistic example reflecting common patterns observed in retail journal data. Trader has 6 months of data and 240 trades across four setups.
Original Performance (All Setups Active)
| Metric | Value |
|---|---|
| Total Trades | 240 |
| Win Rate | 48% |
| Profit Factor | 1.18 |
| Net P/L | +$2,340 |
| Max Drawdown | −$4,120 |
The trader is profitable but barely. $2,340 over 6 months with a $4,120 max drawdown is not a sustainable business. Profit factor of 1.18 means the edge is razor-thin — a few bad weeks could flip the P/L negative.
Setup-Level Decomposition
| Setup | Trades | Win Rate | Profit Factor | Net P/L | Max DD Contribution |
|---|---|---|---|---|---|
| Trend Continuation | 72 | 58% | 2.1 | +$5,420 | −$1,200 |
| Pullback Entry | 64 | 52% | 1.5 | +$1,880 | −$1,600 |
| Counter-Trend Reversal | 58 | 38% | 0.8 | −$1,240 | −$2,400 |
| FOMO / Impulse | 46 | 30% | 0.4 | −$3,720 | −$2,800 |
The story is now obvious. Trend Continuation and Pullback Entry generate $7,300 in combined profit. Counter-Trend Reversal and FOMO trades lose $4,960. The net $2,340 is really $7,300 minus $4,960 in self-inflicted damage.
Impact Analysis: Remove FOMO Trades Only
| Metric | With FOMO | Without FOMO | Change |
|---|---|---|---|
| Total Trades | 240 | 194 | −46 trades |
| Win Rate | 48% | 51% | +3% |
| Profit Factor | 1.18 | 1.52 | +0.34 |
| Net P/L | +$2,340 | +$6,060 | +$3,720 (+159%) |
| Max Drawdown | −$4,120 | −$2,640 | 36% shallower |
Not taking 46 FOMO trades — trades with no defined setup — would have produced $6,060 instead of $2,340. That's 159% more profit from doing less. Max drawdown 36% shallower means less psychological pain and reduced risk of compounding losses through emotional response.
Impact Analysis: Remove FOMO + Counter-Trend
| Metric | All Setups | Trend + Pullback Only | Change |
|---|---|---|---|
| Total Trades | 240 | 136 | −104 trades |
| Win Rate | 48% | 55% | +7% |
| Profit Factor | 1.18 | 1.82 | +0.64 |
| Net P/L | +$2,340 | +$7,300 | +$4,960 (+212%) |
| Max Drawdown | −$4,120 | −$1,600 | 61% shallower |
Removing both losing setups — 104 trades, nearly half of all trading activity — would have tripled the P/L and cut maximum drawdown by 61%. The trader was spending half their time and energy on activity that actively destroyed their profits.
The uncomfortable truth from this example: The trader would have made $7,300 by only taking 136 trades over 6 months instead of $2,340 from 240 trades. Doing less would have earned 3x more. This is not a hypothetical optimization — it's what the actual past data already contains.
The Equity Curve Transformation
Numbers tell part of the story. The equity curve tells the rest.
Two Visible Changes When You Overlay Curves
- The drawdowns flatten. Deep dips in your original equity curve — the ones that made you question your entire trading career — often correspond to clusters of losing setup trades. Remove those setups and the curve smooths dramatically. A smoother curve produces less psychological damage, which produces better decision-making, which produces even better forward performance — a compounding effect.
- The upslope steepens. Without losing setups dragging down the average, profitable setups' contribution becomes more visible. The curve climbs more consistently because it's no longer being pulled down by periodic clusters of bad trades.
Why Visual Comparison Beats Numbers Alone
The visual comparison is often more persuasive than tabular data. Seeing your equity curve without the FOMO trades — seeing how much smoother and steeper it would have been — makes the decision to stop taking those trades feel less like "discipline" and more like common sense. This is why most traders who initially resist the data on emotional grounds finally accept it after seeing the curve overlay: the visceral evidence overcomes the cognitive defense.
Counterfactual setup analysis requires consistent setup tagging across every trade — which most journals don't enforce. Without tagged data, impact analysis can't run. Manual setup tagging from broker statements is slow and tags drift over time. Automated journals with mandatory setup-tag fields capture the data automatically and produce setup-level decomposition, equity curve overlays, and impact analysis natively. The trading journal comparison covers which journals support setup-level decomposition. The performance analysis guide walks through running the analysis on existing data, and the trade quality vs P/L guide covers the upstream quality-grading framework that feeds setup categorization.
Deciding What to Actually Cut
Not every underperforming setup should be eliminated. The decision framework, broken into three categories:
Definite Cuts
- FOMO / impulse trades. These aren't setups — they're emotional reactions. Profit factor is almost always below 0.5 across observational samples. Cut immediately. No 90-day evaluation period needed.
- Setups with 40+ trades and profit factor below 0.7. Sufficient sample to confirm they don't work. Stop trading them. Re-evaluate if market regime changes substantially.
- Setups generating your deepest drawdowns. Even if net P/L is only slightly negative, a setup that produces your largest losing streaks damages psychology disproportionately. Revenge trading typically follows large drawdowns and compounds losses across other setups.
Investigate Before Cutting
- Setups with profit factor 0.8-1.1 across fewer than 40 trades. Might be in a temporary drawdown rather than structural underperformance. Run combined filters (setup + session, setup + instrument) to check whether the setup works under specific conditions.
- Setups recently modified. If you changed how you trade the setup in the last month, filter for only the recent period. The old version might be a loser while the new version shows improvement.
- Setups with high win rate but low R. A 65% win rate with 0.8R average winner might be fixable by adjusting profit targets or stop placement rather than eliminating the setup entirely.
Keep Despite Marginal Performance
- Setups that perform well in specific conditions. A reversal setup that loses in trending markets but profits in ranging markets might be worth keeping for range-bound periods specifically — as long as you only trade it under those conditions.
- Setups with fewer than 25 trades. Insufficient data for any conclusion. Keep trading and tagging; re-evaluate at 40+ trade threshold.
The 80/20 rule applies: For most multi-setup traders, 20% of setups generate 80% of profits. The other 80% range from break-even to destructive. The goal isn't finding more setups — it's identifying your top 20% and focusing exclusively on them.
3 Mistakes Traders Make With Impact Analysis
Mistake 1: Running Analysis on Insufficient Sample Size
A setup with 15 trades showing profit factor 0.6 may simply be in normal variance, not structural underperformance. Removing it based on hindsight inflates the impact analysis result but doesn't reliably improve forward performance because the next 30 trades on that setup might revert. Minimum 30 trades per setup for moderate-confidence analysis; 50+ for high confidence. Below 30, the analysis is fitting noise.
Mistake 2: Confusing Setup Quality with Setup Execution
A setup with profit factor 0.8 might have a positive-expectancy version that's being executed badly. Cutting winners short, moving stops to break-even prematurely, taking trades outside the planned setup criteria — all degrade per-setup performance without changing the underlying edge. Before cutting a setup, audit the trades: is the edge gone, or is execution destroying a real edge? Trade review separates these two failure modes.
Mistake 3: Cutting Too Many Setups Simultaneously
Removing 3-4 setups at once leaves the trader with 1-2 setups producing far fewer trades. Lower trade frequency creates boredom, which creates overtrading on the remaining setups (compensating activity). The post-cut adjustment is psychological, not just mathematical. Cut one setup at a time, give 30 days for adjustment, then re-evaluate. Cutting too aggressively often produces counter-effect: same total trades, just concentrated on fewer setups, with degraded execution from boredom-driven decisions.
Who Should Skip Impact Analysis (For Now)
- Traders with fewer than 100 total trades. Per-setup samples will be too small (typically 20-30 per setup) for statistically meaningful counterfactual analysis. Wait until 200+ total trades before running setup-level removal analysis.
- Single-setup traders. Impact analysis presupposes multiple setups to compare. If you trade one setup exclusively, the analysis question shifts to "should I trade this setup at all" — addressed via edge measurement rather than counterfactual removal.
- Traders without consistent setup tagging. The analysis requires every trade tagged with its setup category. Untagged trade history produces "unknown setup" buckets that distort results. Tag retroactively from journal notes, or commit to forward tagging for 60-90 days before running analysis.
- Algorithmic traders. Systematic strategies typically don't have categorical "setups" — they have rule-based entry conditions that are tested via backtesting and walk-forward analysis rather than counterfactual setup removal. Different methodology applies.
- Traders mid-strategy-transition. If you've changed entry rules, position sizing, or instruments in the last 30 days, your tagged data blends multiple strategies and the analysis becomes uninterpretable. Stabilize first; analyze second.
What Happens After You Cut
Cutting setups is emotionally difficult. Expected adjustment timeline:
Week 1-2: FOMO About Missed Trades
You will see opportunities in the market that match your old (cut) setup and feel the urge to take them. "This one would have worked" — maybe, but the data says the next 10 would not. The cut is statistically valid only if you maintain it for the full evaluation window; selectively re-entering "the obvious ones" reverses the analysis.
Week 3-4: Boredom
Fewer setups means fewer trades. If you went from 5 setups to 3, daily activity drops noticeably. This is by design — you're cutting low-quality activity, not high-quality activity. Use the downtime to improve execution on remaining setups, journal more thoroughly, or simply step away from the screen during low-opportunity windows.
Month 2-3: Results
Profit factor should visibly improve. Drawdowns should be shallower. Equity curve should be smoother. If these improvements don't materialize, re-run the analysis — market conditions may have shifted, or remaining setups may need refinement.
Ongoing: Monthly Re-Evaluation
Run impact analysis monthly. What works changes over time. A setup that was your core edge six months ago might have degraded. A setup you cut might improve in different market conditions. Use the data, not your memory.
Impact Analysis Beyond Setup Tags
Setup removal is the most common application, but the same technique applies to any filter dimension:
- Remove a trading session: "What would my P/L be if I never traded the Asian session?" If significantly better, stop trading Asian. See session performance comparison.
- Remove a day of week: "What if I never traded on Fridays?" Many traders discover Friday alone accounts for 30-50% of weekly drawdown. See why Fridays kill P/L.
- Remove an instrument: "What if I stopped trading GBP/JPY?" Cross pairs with high volatility are often net negative for traders better-suited to majors.
- Remove low-confidence trades: "What if I only took A-grade entries?" Often the highest-impact single filter. See quality vs P/L analysis.
- Remove a time-of-day window: "What if I never traded after 16:00 GMT?" Late-day fatigue + thin liquidity makes this a common high-impact cut.
The five filter dimensions can be combined: "What if I only took A-grade trend continuation setups during London Open on Tuesday-Thursday?" Multi-dimensional filtering frequently reveals the trader's actual high-edge subset hidden within a much larger noisy dataset.
Methodology Note
- Counterfactual analysis: Standard methodology in performance research and applied statistics. Trades are removed from the dataset; all metrics recomputed; original vs counterfactual compared.
- Sample size requirements: Minimum 30 trades per setup for moderate-confidence per-setup analysis, 50+ for high confidence. Below 30, variance dominates structural signal.
- Pre-declaration requirement: Setup categories must be definable in advance, before knowing trades' outcomes. Post-hoc filter discovery produces survivorship-bias inflated results that don't generalize forward.
- Multiple comparison bias: Running impact analysis on many filters and reporting only the most flattering result is statistical cherry-picking. Pre-declare which filters are candidates; report all results, not just successful ones.
- Forward applicability: Counterfactual past performance correlates with future performance for stable strategies on consistent market regimes. Regime changes can invalidate previously-losing setups (and vice versa); re-run analysis quarterly.
For our full editorial process, see our editorial methodology.
Final Verdict: Subtraction Is the Highest-Leverage Adjustment
Most traders spend energy looking for new setups, indicators, and strategies to add. Impact analysis consistently shows the fastest path to better performance is usually subtraction — removing what doesn't work — not addition. The 80/20 distribution applies in retail trading: 20% of setups generate 80% of profits, and the bottom 20% frequently destroy 30-80% of the gross profit. Removing the destructive setups isn't difficult math; it's difficult psychology.
The legitimate methodology requires three disciplines: pre-declared setup categories (no post-hoc filter discovery), adequate sample size per setup (30-50+ trades), and one cut at a time (avoiding overcorrection). Run impact analysis monthly; act on the data; track forward performance to confirm the cuts produced expected improvements.
Three principles from the framework:
- Setup-level decomposition reveals what aggregate hides. Net P/L is the sum of strongly profitable and strongly unprofitable setups. The aggregate is uninformative without decomposition.
- Cut FOMO trades immediately. They aren't setups — they're emotional reactions. Profit factor across observational data is consistently below 0.5. No 90-day evaluation period needed.
- Pre-declare filters before analyzing. Post-hoc filter discovery is multiple-comparison cherry-picking; the discovered "edge" rarely persists forward. Discipline lives in the methodology, not just the math.
For related analysis: trade quality vs P/L for the grade-based decomposition that feeds setup categorization, edge measurement framework for the underlying expectancy math, why Fridays kill P/L for day-of-week impact analysis, session performance comparison for time-of-day impact analysis, performance analysis guide for running counterfactual analysis on your own data, and why most traders lose money for the failure-mode breakdown that often drives setup-level losses.
