The Power Walk Forward Optimizer (PWFO) is a cutting-edge automatic walk forward testing program that eliminates the ad hoc curve fitting produced by optimization of strategy input values on spurious price movements. The PWFO allows you to perform a K-fold cross-validation on your strategy. This package contains a walk forward optimization add-on program that can be added to any TS2000i, TS8 or Multicharts strategy using TS8 or MultiChart multiple strategy option. The PWFO prints out the test sample performance and the out-of-sample performance results, on one line, for each case or input variable combination that is run by the TradeStation(TS) or MultiChart(MC) optimization module to a user selected spreadsheet comma delimited file. The PWFO can generate up to 500 different test and out-of-sample date optimization files in one optimization run saving the user from having to generate optimization runs one at a time. The PWFO output allows you to quickly determine whether your procedure for selecting input parameters for your strategy just curve fits the price and noise or produces statistically valid positive K-fold cross-validation out-of-sample results.

In addition to the out-of-sample performance results presented for each case, 30+ superior and robust performance metrics (many are new and never presented before) are added to each case line and printed out to the comma delimited file. The PWFO also allows the user to code their own performance metric in EasyLanguage which is then added to the performance statistics on each case line. These 30+ performance metrics allow for a superior and robust selection of input variables that have a higher probability of performing well on out-of-sample data. For Instance, using Excel's filtering capabilities, such as "top 1%", "is less than or equal to", etc. on the various column performance statistics, one can quickly home-in on how this filtered selection will perform on out-of-sample data. This is because the out-of-sample performance is right there on the same line for each case. Using Excel's graphical modules, one can graph the out-of-sample net profits Vs the various performance statistics for all cases and visually see which test performance metrics have any predictive ability for out-of-sample net profits. I wrote about such tests in my papers entitled Hoodwinked by Randomness, Futures Magazine December, 2000 and The Optimization Trap, Active Trader Magazine November, 2001 (also available on our papers page).

Out-of-sample testing is the norm in all mathematical economic disciplines other than technical analysis. The basis of walk forward out-of-sample testing is to take a test sample of data, perform a combinatorial input parameter search (dubbed optimization by current trading platforms) on the test data, choose the input parameters from the combinatorial search based upon some performance metric (net profit, profit factor, etc) in the test sample and then apply the input parameters to data they haven't been tested on to see if they work.

Why use walk forward analysis? Why not just optimize over the total amount of data once and choose the inputs from the best performance parameters? Optimization is a misnomer and should really be called Combinatorial search. That is, we have the software output certain performance metrics like net profit or profit factor for every combination of a range of input parameters. We then sort that range on that performance metric. Whenever you run an combinatorial search over many different combinations of input parameters on noisy data on a fixed time interval, the best performance parameters found are guaranteed be due to “curve fitting” the noise and signal. This curve fitting is also called Data Mining or Data Snooping. A fixed number of prices on a fixed time interval has many spurious movements which is also called noise. When we run, say, 5000 different cases, the best performance parameters will be from those system input variables that are able to produce profits from the price or signal dynamics and the spurious movements . While the signal dynamics, if there, will repeat, the same spurious price movements will not. If the spurious movements that were captured by a certain set of input parameters were a large part of the net profit, then choosing these input parameters would produce losses when traded on future data. These losses occur because the spurious movements will not be repeated in the same way. The input parameters chosen from the test section are cherry picked to perform well on only those exact same spurious movements. This is why optimization or combinatorial searches with no out-of-sample testing cause loses in the out-of-sample section from something that looked great in the test section.

The PWFO can print up to 500 Test and out-of-sample(OOS) files in just one TS optimization run. For instance, in this necessarily short example, suppose we had one minute bar data from 4/9/03 to 6/6/03 loaded into a TS chart with a strategy combined with the PWFO. If we set the PWFO to make a test period of 30 calendar days and an OOS period of 7 calendar days of 1 minute bars, the PWFO would generate the following four test-OOS output files for each case of the strategy's optimization run:

• File1 - Test Dates 4/09 to 5/09   OOS Dates 5/12 to 5/16 (5/10,5/11 = weekend)
• File2 - Test Dates 4/16 to 5/16   OOS Dates 5/19 to 5/23 (5/17,5/18 = weekend)
• File3 - Test Dates 4/23 to 5/23   OOS Dates 5/26 to 5/30 (5/24,5/25 = weekend)
• File4 - Test Dates 4/30 to 5/30   OOS Dates 6/02 to 6/06 (5/31,6/01 = weekend)

If the optimization run consisted of 5000 different cases of the strategy input variables, then each of the PWFO files would have 5000 lines and each PWFO file would represent only the test-oos performance for the indicated dates. Of course if the data loaded into TS was much longer in time, then the PWFO could generate up to 500 different sample test-oos date files.

For each TS run, the user can select the test sample dates he wants to perform the optimization on and the out-of-sample(OOS) dates for out-of-sample performance. After examining fifty or more different test and out-of-sample files, the user rapidly gets a real feel for the statistics and filters that are important for determining good out-of-sample performance. No more guess work. The PWFO is a real time saver. Please note that the median is used for a number of performance statistics instead of the average. The median is a much more robust measure than the average and is not distorted by outliers.

Below are the performance metrics that are printed out for each input variable case.

• Test Data Performance Metrics
  • Input Variables
  • Median Test Sample Net Profit (mtnp)
  • Test Sample Net Profit (tnp)
  • Number of Trades (nT)
  • Percent Profitable Trades (pP)
  • Profit Factor (PF)
  • Trade Standard Deviation (std)
  • Student t-statistic. Used to determine the probability that the Ave Trade Profit is zero (t)
  • Median Losing Bars (mlb)
  • Total Losing Bars (lbr)
  • Median Winning Bars mwb)
  • Total Winning Bars (wbr)
  • Ratio of Median Winning Bars to Median Losing Bars (awl)
  • Ratio of Total Winning Bars to Total Losing Bars (wlb)
  • Median of all Trades{Trade Runup minus Final Trade Profit} (mru)
  • Median of all Trades{Final Trade Profit minus Trade Rundown) (mdd)
  • Maximum Consecutive Winners In-A-Row (wr)
  • Maximum Consecutive Losers In-A-Row (lr)
  • Median Of The Winning Trades (mpft)
  • Median Of The Losing Trades (mlos)
  • Ratio of Median Winning Trades /Median Losing Trades (p/l)
  • Maximum Drawdown (dd)
  • Largest Losing Trade (llt)
  • Slope of Trade Equity Regression Line (eqB0)
  • Trade Equity Regression Trend Line Coefficient of Correlation R² (eqR2)
  • Median of The absolute Values of Each Trade Equity Minus The Equity Regression Trend Line (eqDev)
  • Slope of Trade Equity Least Squares 2nd Order Polynomial Line where Equity=b₀ + b₁*i + b₂*i² (eqB1)
  • Velocity of Equity Curve Least Squares 2nd Order Polynomial Line.(eqDy)
  • Acceleration of Equity Curve Least Squares 2nd Order Polynomial Line. (eqB2)
  • Projected Equity 10 Trades In Future Using Equity Curve Least Squares 2nd Order Polynomial Line.(eq10)
  • Equity Least Squares 2nd Order Polynomial Line Coefficient of Correlation R² (eqR22)
  • Modified K-ratio = Slope of Equity Regression Line/ (The Average of The Absolute Values of The Equity at Each Trade Minus The Equity Regression Trend Line). (mKr)
  • User Performance Metric = The user codes their own performance metric in EasyLanguage and this metric is include with the output. (um)
• Out-Of-Sample Data Performance Statistics
  • Out-Of-Sample Net Profit (osnp)
  • Out-Of-Sample Drawdown (odd)
  • Out-Of-Sample Largest Losing Trade (ollt)
  • Out-Of-Sample Number Of Trades (ont)
  • Out-Of-Sample Average Net Profit (aosnp)

Below is a snippet of the output from a PWFO run sorted by the test section student t statistic. This example shows the partial output for the Acceleration system for the test dates of 3/31/04 to 4/30/04 and out-of-sample dates of 5/3/04 to 5/7/04. Notice how the out-of-sample results are on the same line as the input parameters and test performance variables. This output presentation allows you to see immediately the out-of-sample results from your test performance variable selection techniques. For instance if you wanted to see all those cases that satisfied a Profit Factor greater than 1 and less than 3 with losers in a row of 3 or less, then using Excel's Autofilter feature, you could immediately see all cases that satisfied those criteria and what the out-of-sample results would be. In addition, using Excel's macro creation ability you can create an excel macro to perform your own performance variable screen with a one button click. This allows you to go through the many walk forward files created by the PWFO in minimal time to see how your performance variable screen works over many test/out-of-sample runs. As noted above, the PWFO uses the median instead of the average whenever we can. The median is a much more robust statistic than the average. As an example look at row 5 on the below spreadsheet snippet. The median trade profit is minus -$12.50 while the average trade profit (tnp/nT) is a plus $13.23. The average is controlled by a few outliers while the median is robust and immune to these few outliers.

---

---

How is the use of the PWFO possible with any Strategy? TradeStation 2000i's Strategy builder and TS8's/MultiCharts strategy combining window allows for the combining of any number of systems or signals into one strategy. The PWFO is a strategy that computes and prints out to a user defined directory and Excel file all the performance statistics above using a super fast C++ DLL. For TS2000i, TS8 and MultiCharts you need to access the Power Editor for your strategy and add the line value99=$PWFOinp(input1,input2,input3,etc); at the end of "YourStrategy". Then, do a "save as" with a new name like "YourStrategyPWFO", and you are ready to use the PWFO.

---

Click here to see the Input vs OOS Explorer program, a description of the program inputs and the output file that is generated Included with the PWFO is a Walk Forward Input vs Out-of-Sample Explorer that that allows one to quickly determine which strategy inputs generate the statistically best average out-of-sample returns after the curve fitted performance results have been filtered out. The Walk Forward Input vs OOS Explorer (WFINP) reads all files generated by the PWFO and searches the PWFO test sample input parameters in those files for those inputs that generate the statistically best average out-of-sample returns. The WFINP is a stand alone exe program that is super fast. It can run through 100 PWFO files in less that 2 minutes and automatically display the results in Excel.

---

The Power Walk Forward Optimizer comes with a detailed manual explaining:

• How to setup a TS2000i, TS8 or MultiChart with the PWFO and any strategy/signal
• How to use the PWFO with any strategy.
• An explanation of each of the performance statistics.
• How to enter and code your user performance metric in easy language.
• How to use the generated optimization Excel output file.
• How to use the Walk Forward Performance Excel Filter Add-In.
• Walk Forward Optimization tutorial.
• PWFO TS2000i/TS8/MultiChart code (C++ DLL code not included).

Compatibility Note: It is possible to use the PWFO with any TS2000i/TS8/MultiChart strategy. However, if the strategy is locked, read protected or cannot be accessed and changed by the Power Editor, then the PWFO will not work with that strategy. Otherwise if the strategy is accessible and can be edited by the Power Editor then that strategy will work with the PWFO.

For TradeStation 2000i, TS8 and Multicharts, the Power Walk Forward Optimizer v2 package consisting of manual, Walk Forward Input vs OOS Explorer (WFINP), DLL files and TradeStation ELS/ELD or MultiChart PLA file is being offered, for $595 plus shipping. Notice: The PWFO has a "Key Licence" that only allows it to be installed on two computers. With a special combination package, the Power Walk Forward Optimizer+WFINP package can be combined with the  Walk Forward Performance Explorer (WFPE) or the  Walk Forward Surface Explorer (WFSE). These "combos" are being offered for $895 each, a savings of $95 if purchased separately. In addition the Walk Forward Suite consisting of PWFO + + WFINP + WFPE + WFSE is being offered for $1195, a savings of $190 if purchased separately.

How To Order
To order online click Order Online. To order via Fax or mail using a Visa or Master Card please fill out the order form on the Order Form page and Fax it to the telephone number on the order form or mail it to the address on the order form. If you would like to talk to me about the product, please call me at (312) 280-1687 M-F 12pm to 5pm CST. All E-mail queries can be sent to info@meyersanalytics.com.

Thank you for your Interest....Dennis Meyers

---