The Walk Forward Performance Explorer (WFPE) reads all files generated by the PWFO and searches the PWFO performance metric variables in each of those files for those performance metrics that generate the statistically best average out-of-sample returns. The WFPE eliminates all cases in the PWFO test files that do not meet certain Profit Factor (PF), Losers in a row (LR), Number of trades (NT) and Performance Metrics criteria . The PF, LR, Metric, and NT criteria ranges are user selectable generating many filter searches in one run. The WFPE is a stand alone exe program that is super fast. It can run 100 PWFO files in less that 2 minutes and automatically display the results in Excel. In addition, using modern "Bootstrap" techniques, the WFPE calculates the probability of whether or not the filter's results were due to chance. No other Walk Forward software offers this capability.

Click here to see the WFPE Input Form and a description of the inputs

Data Mining and Curve Fitting. Data mining or curve fitting a price series will always produce the best performance results. If you look hard enough using optimization you will always find patterns in the price data. But they are not real. Why use the PWFO walk forward technique? Why not just perform a TradeStation optimization on the whole price series and choose the input parameters that give the best total net profits or profit factor? Whenever you run a optimization (combinatorial search} over many different combinations of input parameters on noisy data on a fixed number of prices, no matter how many, the best performance parameters found are guaranteed to be due to "curve fitting" the noise and signal. When we run, say, 5000 different input parameter combinations , the best performance parameters will be from those system input variables that are able to produce profits from the price pattern and the random spurious price movements While the price patterns, if there, will repeat, the same spurious price movements will not. If the spurious movements that were captured by a certain set of input parameters were a large part of the total net profits, then choosing these input parameters will produce losses when traded on future data. These losses occur because the spurious movements will not be repeated in the same way. This is why system combinatorial searches with no out-of-sample testing cause loses when traded in real time from something that looked great in the test section.

In order to gain confidence that our system input parameter selection procedure on test data will produce profits "on average" in the future we must perform the walk forward out-of-sample analysis many times. Why not just do the analysis once? Well just as in poker, where there is considerable vagaries in hand to hand luck, walk forward out-of-sample analysis give considerable vagaries in week to week out-of-sample profit "luck". That is, by pure chance we may have chosen some input parameters that did well in the test section data and the out-of-sample section data. In order to minimize this type of "luck", statistically, we must repeat the walk forward out-of-sample (oos) analysis over many test/oos sections and take the average of our weekly results over all out-of-sample sections (we need at least 30 oos sections for statistically significant results). This average gives us an expected weekly return and a standard deviation of weekly returns which allows us to statistically estimate the expected equity and it's range for N weeks in the future.

Walk Forward Performance Explorer (WFPE) Description. As stated above, curve fitting the price series will always produce the best test performance results. If we eliminate the optimization cases with the best performance results, we are sure to eliminate many of the curve fitting system input parameters that fitted the past spurious noise movements of the price series. Here is an example of one of the filters that the WFPE can produce: Very few systems can sustain Profit Factors (PF) above 3 over time. Therefore if we eliminate from the test sample optimization results, all cases that have profit factors greater than 3, we will likely eliminate most of the cases that are due to curve fitting the noise. Also, we like to filter for only those cases in the test section had a positive net profit. In addition, because in real time it is difficult to sustain more than five losses in a row and still keep trading, we will eliminate all those test sample cases that have more than 5 losses in a row. The PWFO generates the metric R2 which is the Trade Equity Regression Trend Line Coefficient of Correlation in the test section. We would like to exclude any cases in the test section that have an R2<60. This particular filter might leave anywhere from 10 to 200 cases or rows in the PWFO file that satisfy these filter conditions. What is left in the PWFO file after applying the above filter are 10 to 200 rows of 31 columns of PWFO metric statistics If we look for the maximum value in each one of the PWFO columns in the rows that are left we have up to 31 different filters for this PF, LR, R2 screen. We call this filter PF<30|5|r2>60 where PF<30 means PF<=3.0, "|5" means LR<=5 and r2>60 means r2≥60. Suppose for this filter, for the PWFO rows that are left we want the row that has the maximum PWFO mtnp metric. This would produce a filter named PF<30|5|r2>60-mtnp. This particular PF<30|5|r2>60-mtnp is calculated for each of the PWFO files that are run by the WFPE and the average out-of-sample performance, plus other important statistics for this filter is summarized along with the hundreds of other filter combinations that are constructed in a similar manner. This filter summary is sorted by total out-of-sample Net profits after costs for all the various filters and written to a comma delimited file by the WFPE. When The WFPE run is done just click on the Excel Icon next to the Run button and the WFPE output file will appear in Excel or your spreadsheet

Bootstrap Probability of Filter Results. Using modern "Bootstrap" techniques, the WFPE calculates the probability of obtaining each PF-LR-Metric-NT filter's total out-of-sample net profits by chance. Here is how the bootstrap technique is applied. Suppose, for this example, the WFPE calculates the total out-of-sample net profits(tONet) of 700 different PF-LR-Metric-NT filters. A mirror filter is created for each of the 700 WFPE filters. However, instead of picking an out-sample net profit(OSNP) from a filtered row, the mirror filter picks a random row's OSNP in each PWFO file. Each of the 700 mirror filters will choose a random row's OSNP of their own in each of the PWFO files. Thus if there are 50 PWFO files, each of the 700 mirror filters will pick a random row's OSNP in each of the 50 PWFO files. At the end, each mirror filter will have 50 random OSNP's picked from the rows of the 50 PWFO files. The sum of the 50 random OSNP picks for each mirror filter will generate a random total out-of-sample net profit(tONet) for each of the 700 mirror filters. The average and standard deviation of the 700 mirror filter's random tONets will allow us to calculate the chance probability of each WFPE filter's tONet. Thus given the mirror filter's bootstrap random tONet average and standard deviation, we can calculate the probability of obtaining the PF-LR-Metric-NT filter's tONet by pure chance alone.

Below is a snippet of the output from a WFPE run sorted by the total out-of-sample net profit statistic (tONet). This example shows the partial output file from a WFPE run on the PWFO files generated by the Nth Order Fixed Memory Polynomial Velocity Strategy that was run on 1 contract of the Japanese Yen 1 minute bar futures for the 50 weeks of 12/10/04 to 11/18/05.

---

---

The WFPE Columns are defined as follows

• Row 1 is the PWFO Stub, File Start Date, File End Date and Number of weeks.


• Filter = The filter that was run. For example, pf<25|4|r2>60:12 is: PF<=2.5, LR<=4, R2>=60, Number of trades/test section>=12. pfxx|4|r2>60:10 is: any PF, LR<=4, R2>=60, Number of trades/test section>=10


• Metric = The PWFO performance metric (defined on the PWFO page). For this WFPE filter, pf<25|4:12 Metric=b1 (Equity curve slope), this PWFO metric produced the following average 50 week statistics on this line.


• tOnp = Total out-of-sample(oos) net profit for these 50 weeks.


• mOsp = median oos net profit for the 50 weeks


• aOsp = Average oss net profit for the 50 weeks


• aanp = Average oos profit per trade


• aOnT = Average number of oos trades per week


• B0 = The 50 week trend of the out-of-sample weekly profits


• %P = The percentage of oos weeks that were profitable


• t = The student t statistic for the 50 weekly oos profits. The higher the t statistic the higher the probability that this result was not due to pure chance


• std = The standard deviation of the 50 weekly oos profits


• llw = The largest losing oos week


• eqDD = The oos equity drawdown


• lr = The largest number of losing oos weeks in a row


• # = The number of weeks this filter produced a weekly result. Note for some weeks there can be no strategy inputs that satisfy a given filter's criteria.


• b00 = The straight line trend of the oos equity curve in $/week.


• Blw = The maximum number of weeks the oos equity curve failed to make a new high.


• BE = Break even weeks. Assuming the average and standard deviation are from a normal distribution, this is the number of weeks you would have to trade to have a 99% probability that your oos equity is above zero.


• ndd = The normalized equity drawdown = 100*eqDD/tONet


• tONet = Total out-of-sample net profit(tOnp) minus the total trade cost. tONet=tOnp - #*aOnT*Cost.


• Prob = The probability that the filter's tONet was due to pure chance. For example, for row 3 where tONet=8316 there is a 1 in 1960 (1/0.00051) chance that the tONet obtained by this filter was due to chance.


• Notice: Past performance is no guarantee of future results

The Walk Forward Performance Explorer comes with a detailed manual explaining:

• How to setup and install the Walk Forward Performance Explorer. The Walk Forward Performance Explorer is a stand alone exe file that can be executed directly from your desktop icon or from the Windows Start Program menu.
• How to use the WFPE with your PWFO files.
• An explanation of each of the performance statistics columns.
• Walk Forward Optimization tutorial.

The WFPE Input Finder Excel Add-In and WFPE Strategy Input finder EXE.

Supplied with the WFPE is an Excel Add-In Filter and a separate WFPE Input Finder EXE. For the Excel Add-In, as shown below, you just click on a special Add-In icon on the Excel Toolbar and a popup window displays. Fill in the WFPE parameters and click on the Run button. The Excel Add-In macro will filter any number of PWFO files loaded in Excel, and will display the PWFO Excel file row that satisfies the typed in criteria. For the WFSE Input finder, as shown below, fill in the WFPE parameters click on Run and the Strategy Inputs are displayed as shown below.

   

The The Walk Forward Performance Explorer package consisting of manual, WFPE EXE file, WFPE Excel Add-In, and WFPE Input finder EXE is being offered, for $395 plus shipping. Please note that the WFPE will only read files generated by the PWFO product. In addition, the WFPE has a "Key Licence" that only allows it to be installed on two computers. With special pricing, the Walk Forward Performance Explorer package can be combined with the  PWFO program and this combo is being offered for $895, a savings of $95.

How To Order
To order online click Order Online. To order via Fax or mail using a Visa or Master Card please fill out the order form on the Order Form page and Fax it to the telephone number on the order form or mail it to the address on the order form. If you would like to talk to me about the product, please call me at (312) 280-1687 M-F 12pm to 5pm CST. All E-mail queries can be sent to info@meyersanalytics.com.

Thank you for your Interest....Dennis Meyers

---