BACKTESTING WITH PAST REAL DATA OR RANDOM DATA
COLLECTING TICKDATA: "Tickgrabber"
G-Bot offers 2 effective ways of testing strategies, apart live trading (forward live testing):
1) Backtesting: Running on past real tickdata
2) Forward testing with random tickdata, generated on the fly
Besides, you can run in 2 modes: "backtest" mode or in "Strategy
Analysis"
mode (more useful), which runs the strategy again and again providing resampling results, to
test the strategy robustness.
Testing with random data is very useful for various reasons:
- A greater variety of situations is checked (overfit is
difficult
against randomized data, also because we randomized volatility too within the
random processess)
- Data is perfectly "clean".
In this folder you will have 1 folder for each instrument, and in that folder all the tickdata files (1 for each "day"):
This folder containing all tickdata files for the specified instruments is created
automatically,
by using the utility "TickGrabber" (free
private
application, just like G-Bot).
Tickgrabber is just a tickdata "recorder" which collects ticks and
store them on your hard disk:
Tickgrabber at work:
How to use TickGrabber: quickstart
Tickgrabber is very easy to use. It can be started once and let run forever
(never shut it down, even if the Gateway is restarted).
It
will automatically populate the "Tickdata" folder with all instrument folders
and files. This data can be immediately used by G-Bot backtesting facility.
- Run Tickgrabber
- Edit the symbol list, if it needs to be changed
- Check the 3 check boxes (Autostart/Autostop/Loop Forever) and click on
"Apply"
- Let it run and forget about it. It will continue recording tickdata for all the specified instruments!
As detail, note that Tickgrabber stores the symbol file in the folder "TickGrabber_Files" in your "Documents" folder. This is similar to G-Bot ,which instead use the folder "GBOT_Files", in your "Documents" folder.
It will automatically load, by the default, the file "Symbols1.txt". Note
that a second instance of Tickgrabber could load another symbol file "Symbols2.txt".
This way you can have separate instances of tickgrabber, using different symbol
file (for instance one for futures, and another one for stocks and ETFs, which
trade only RTH).
How to backtest the tickgrabber tickdata
This is very simple: just switch to to "backtest" tab and, in the symbol list,
add the symbols you wish to backtest (use "Edit symbol" button to add symbols).
(The symbol names as the same as the folder names created by tickgrabber in you
tickdata repository.)
Click on "Get Files" > "Single Backtest" tab > "Start Backtest"
How to make G-Bot process tickdata that you already have
G-Bot can process any tickdata. It clearly expects them in exactly the same format as generated by TickGrabber.
So if you have tickdata that you wish to process with G-Bot, simply convert that data into the tickgrabber format. The Tickgrabber format is very simple and efficient.
Each daily tickdata files will look like:
where the 6 values on a line are: Hour, Min. Sec, Ms, Bid=1/Ask=2, Price
The file name will be of the type: INSTRUMENTNAME_DAY.txt, example: EMD FUT 201006 CME_USD 100_2010-04-23.txt
where the instrument name is in G-Bot symbol format. The name of the folders containing the daily tick file will be the same as the instrument name, example: EMD FUT 201006 CME_USD 100
G-Bot will also process tickdata file with an alternative format where bid and ask are on the same row, and instead of the 1/2 codes, there is a 0 (this is for the cases where you have both BID and ASK, associated with a unique timestamp).
Example:
Before backtesting your own data make sure it is consistent fullfils the following conditions:
- Time must go only forward
- Do not replicate same ticks at same time
- Each price MUST be an exact multiple of the instrument ticksize
Notes:
Tickgrabber assumes a day session to be::
from 18:00 edt to 18:00 edt of next day (24 H, for futures)
from 9:30 edt to 16:00 edt of same day (RTH, for stocks, ETFs)
These are set automatically. In case you need to restart Tickgrabber after a computer reset or whatever,
make sure that the red label ("Target Tick File") matches with the start day of the trading session.In order to set that automatically:
- for futures, click either on "default 24H" or
"default 24H - 1 Day" if it's past midnight, as appropriate.
You can record anything with Tickgrabber, including Futures, Stoks, ETFs,
Forex, Options, etc.
The symbol convention is the same as G-Bot, and follow closely IB notation.
Note that according to our rules and to IB rules, it is forbidden to disseminate
this data. Clearly, no commercial use would be legal.
Use only for personal
research and to improve your performances with G-Bot.