Expanding data mines hierarchy content

This commit is contained in:
Julian Molina committed 2020-02-04 17:16:36 +01:00
1 parent bbd608f460
commit a02e627e77
8 files changed
+254 -10

No files matched your search

+6 -4
View File
@@ -1,12 +1,12 @@
data_mine: "A data mine is a hierarchy that contains definitions regarding sensor bots, indicator bots, and plotters, which extract data from third-party sources, process data, and produce the visualization of data over the charts, respectively."
data_mine: "A data mine is a hierarchy that contains definitions regarding bots, including the trading bot, sensors, indicators, and even plotters. These definitions make up the actual programs of such bots—the source code."
indicator_bot: "An indicator bot is an algorithm that processes available information that other bots may have generated, and produces elaborate datasets for others to consume. The indicator bot node holds all definitions required for an indicator bot to function."
indicator_bot: "An indicator bot is an algorithm that processes information that other bots have generated, and produces elaborate datasets for others to consume. "
process_definition: "The process definition node groups all definitions required for a process to function. The Multi Period Market Process deals with time frames of one hour and above. The Multi Period Daily Process deals with time frames of 45 minutes and below."
process_output: "The process output defines which datasets are impacted by the process."
process_output: "The process output groups the definitions of which datasets are impacted by the process, that is, which datasets the process builds or takes a part in building."
output_dataset: "The output data set is a reference to a dataset definition. The effect of establishing a reference from the output dataset to a product dataset definition is that—upon execution, every time the process finishes a processing cycle—the process triggers an event that may be consumed by other entities. These events indicate that the corresponding datasets (the ones impacted by the process as per the process output definition) have been updated."
output_dataset: "The output dataset is a reference to a dataset definition. By establishing such reference, the process acquires the definitions as of how the dataset is to be constructed."
process_dependencies: "Process dependencies are references to various data structures on which the process depends to function."
@@ -69,3 +69,5 @@ style: "A style is a definition of color and opacity in case of a polygon body,
style_condition: "A style condition is a rule that allows applying an alternative style in cases where the rule validates true."
polygon_body: "A polygon body is is the fill of a shape made out of at least 3 vertices."
pending: "daily files, market files and so on, "
+1 -1
View File
@@ -1,4 +1,4 @@
data_mine: "A data mine is a hierarchy that contains definitions regarding sensor bots, indicator bots, and plotters, which extract data from third-party sources, process data, and produce the visualization of data over the charts, respectively."
data_mine: "A data mine is a hierarchy that contains definitions regarding bots, including the trading bot, sensors, indicators, and even plotters. These definitions make up the actual programs of such bots—the source code."
indicator_bot: "An indicator bot is an algorithm that processes available information that other bots may have generated, and produces elaborate datasets for others to consume. The indicator bot node holds all definitions required for an indicator bot to function."
+2
View File
@@ -43,3 +43,5 @@ data_product: "A data product represents the collection of datasets generated by
dataset: "A data set is a collection of files stored in the designated location, as per the definitions on the referenced data product in the corresponding data mine."
session-independent_data: "Session independent data refers to data generated by sensors and indicators, not related to trading sessions."
session: "A session, or trading session, is a specific mode in which the trading bot instance may be run. The trading bot is prepared to run three different types of testing sessions—backtesting, paper trading, and forward testing—and the live trading session mode."
+2
View File
@@ -43,3 +43,5 @@ data_product: "A data product represents the collection of datasets generated by
dataset: "A data set is a collection of files stored in the designated location, as per the definitions on the referenced data product in the corresponding data mine."
session-independent_data: "Session independent data refers to data generated by sensors and indicators, not related to trading sessions."
session: "A session, or trading session, is a specific mode in which the trading bot instance may be run. The trading bot is prepared to run three different types of testing sessions—backtesting, paper trading, and forward testing—and the live trading session mode."
+1 -1
View File
@@ -109,7 +109,7 @@ entries:
url: /suite-references.html
output: web
- title: Trading System
- title: Trading Concepts
output: web
folderitems:
+237 -1
View File
@@ -793,6 +793,28 @@ toc: false
**{{site.data.data_mine.data_mine}}**
In a way, data mines are catalogs of bots that anyone may create. This hierarchy provides all the tools required to create sensors, indicators and plotters, with minimal coding required.
Because the hierachy provides definitions, it does not intervene in the execution of bots. Instead, the definitions in data mines are instanciated from the Network hierarchy by bot instances.
### Adding a Data Mine
To add a data mine, select *Add Data Mine* on the workspace node menu.
### Configuring a Data Mine
Select *Configure Data Mine* on the menu to access the configuration.
```json
{
"codeName": "Supermine"
}
```
* ```codeName``` is the name of the data mine as used within the code of the system.
@@ -803,6 +825,29 @@ toc: false
**{{site.data.data_mine.indicator_bot}}**
indicator_bot: "An indicator bot is an algorithm that processes information that other bots have generated, and produces elaborate datasets for others to consume. "
The indicator bot node holds all definitions required for an indicator bot to function. Definitions are split between the definition of processes and products. Processes are algorithms that go through an input dataset, perform certain calculations, and produce an output. Part of that output are data products.
### Adding an Inidicator Bot
To add an indicator bot, select *Add Indicator Bot* on the data mine node menu. An indicator bot is created along with the basic structure of nodes required to define it.
### Configuring an Indicator Bot
Select *Configure Bot* on the menu to access the configuration.
```json
{
"codeName": "Super",
"repo": "Super-Indicator-Bot"
}
```
* ```codeName``` is the name of the bot as used within the code of the system.
@@ -813,6 +858,109 @@ toc: false
**{{site.data.data_mine.process_definition}}**
process_definition: "The process definition node groups all definitions required for a process to function. The Multi Period Market Process deals with time frames of one hour and above. The Multi Period Daily Process deals with time frames of 45 minutes and below."
As hinted above, most bots, in particular indicators, have two different processes. The reason is that different data structures need be handled in different manners. The Multi Period Daily process handles daily files, while the Multi Period Market process handles market files.
### Adding a Process Definition
To add a process definition, select *Add Process Definition* on the bot's menu. A process definition node is created along with the basic structure of nodes comprissing the definition.
### Configuring a Process Definition
Select *Configure Process* on the menu to access the configuration.
**Multi-Period-Market:**
```json
{
"codeName": "Multi-Period-Market",
"description": "Brief description of what the bot does.",
"startMode": {
"allMonths": {
"run": "false",
"minYear": "",
"maxYear": ""
},
"oneMonth": {
"run": "false",
"year": "",
"month": ""
},
"noTime": {
"run": "true"
},
"fixedInterval": {
"run": "false",
"interval": 0
}
},
"deadWaitTime": 0,
"normalWaitTime": 0,
"retryWaitTime": 10000,
"sleepWaitTime": 3600000,
"comaWaitTime": 86400000,
"framework": {
"name": "Multi-Period-Market",
"startDate": {
},
"endDate": {
}
}
}
```
**Multi-Period-Daily:**
```json
{
"codeName": "Multi-Period-Daily",
"description": "Brief description of what the bot does.",
"startMode": {
"allMonths": {
"run": "false",
"minYear": "",
"maxYear": ""
},
"oneMonth": {
"run": "false",
"year": "",
"month": ""
},
"noTime": {
"run": "true"
},
"fixedInterval": {
"run": "false",
"interval": 0
}
},
"deadWaitTime": 0,
"normalWaitTime": 0,
"retryWaitTime": 10000,
"sleepWaitTime": 3600000,
"comaWaitTime": 86400000,
"framework": {
"name": "Multi-Period-Daily",
"startDate": {
"takeItFromStatusDependency": 0
},
"endDate": {
"takeItFromStatusDependency": 1
}
}
}
```
Both configurations deal with certain aspects of the internal workings of these kinds of processes. These are the ones you should adjust when creating a new bot:
* ```codeName``` is the name of the process as used within the code of the system; for consistency, always use ```Multi-Period-Market``` and ```Multi-Period-Daily``` accordingly.
* ```description``` is a short description of what the bot does.
@@ -823,6 +971,16 @@ toc: false
**{{site.data.data_mine.process_output}}**
process_output: "The process output groups the definitions of which datasets are impacted by the process, that is, which datasets the process builds or takes a part in building."
### Adding a Process Output
To add a process output node, select *Add Missing Items* on the process definition node menu. Items that may be missing are created along with the basic structure of nodes required to define them.
@@ -833,6 +991,24 @@ toc: false
**{{site.data.data_mine.output_dataset}}**
output_dataset: "The output dataset is a reference to a dataset definition. By establishing such a reference, the process acquires the definitions as of how the dataset is to be constructed."
There are other effects of establishing a reference from the output dataset to a product dataset definition. Upon execution, every time a process finishes a processing cycle, it triggers an event that may be consumed by other entities. This event indicates that the datasets impacted by the process have been updated.
An example of other entities that may be listening to such events is that of plotters. Plotters read datasets and create graphical representations of this data over the charts. You surely have noticed that the charts are constantly updating the information in form of candles and indicators in realtime, syncronized with the data being extracted from the exchange by the sensor bot. That kind of automatism is possible thanks to the events processes trigger every time an execution cycle is finished, signaling to everyone listening that new data is available on each of the impacted datasets.
### References
An output dataset must reference a dataset definition.
### Adding an Output Dataset
To add an output dataset, select *Add Output Dataset* on the process output node menu.
{% include note.html content="Remember to establish a reference to a valid dataset definition after creating the output dataset." %}
@@ -843,6 +1019,8 @@ toc: false
**{{site.data.data_mine.process_dependencies}}**
process_dependencies: "Process dependencies are references to various data structures on which the process depends to function."
@@ -853,6 +1031,8 @@ toc: false
**{{site.data.data_mine.status_dependency}}**
status_dependency: "Status dependencies define which other processes the process depends on."
@@ -863,6 +1043,8 @@ toc: false
**{{site.data.data_mine.data_dependency}}**
data_dependency: "Data dependencies define which datasets the processes uses as input."
@@ -873,6 +1055,8 @@ toc: false
**{{site.data.data_mine.status_report}}**
status_report: "Status reports serve as temporal annotations that bots read every time they run to know what was done in the previous cycle and what the state of affairs is at present. Status reports are dynamic, and they change constantly, with updates after every cycle of the associated process."
@@ -883,6 +1067,8 @@ toc: false
**{{site.data.data_mine.execution_started_event}}**
execution_started_event: "The execution started event is the event that triggers the execution of a process. It usually references the execution finished event of another process on which the process depends on."
@@ -893,6 +1079,8 @@ toc: false
**{{site.data.data_mine.execution_finished_event}}**
execution_finished_event: "The execution finished event is the event that processes trigger once they have finished an execution cylce. The event is broadcasted to whoever wants to listen, so that other bots may know when the process has finished its execution cycle."
@@ -903,6 +1091,8 @@ toc: false
**{{site.data.data_mine.product_definition}}**
product_definition: "The product definition node holds all definitions that make up a product, including how the product is calculated and the datasets the product is comprised of."
@@ -913,6 +1103,8 @@ toc: false
**{{site.data.data_mine.dataset_definition}}**
dataset_definition: "A dataset definition contains a configuration file that defines all aspects of the dataset. There are two types of dataset definitions: multi period market and multi period daily."
@@ -923,6 +1115,8 @@ toc: false
**{{site.data.data_mine.record_definition}}**
record_definition: "The record definition is where indicators define how many fields, what fields and in which order those fields will be stored in each record."
@@ -933,6 +1127,8 @@ toc: false
**{{site.data.data_mine.record_property}}**
record: "A record is the compendium of information stored for each period of time, including the start and end datetime of the period along with the properties that provide the information the indicator is meant to generate."
@@ -943,6 +1139,8 @@ toc: false
**{{site.data.data_mine.formula}}**
formula: "In the context of a record definition, formulas are used to assign a value to the property, usually in the form of a variable declared in the calculation or data building procedure."
@@ -953,6 +1151,8 @@ toc: false
**{{site.data.data_mine.calculations_procedure}}**
calculations_procedure: "The calculations procedure processes information to be fed to calculated properties. Calculated properties are not stored in the data set; instead, they are calculated in real-time as the data is consume by others."
@@ -963,6 +1163,8 @@ toc: false
**{{site.data.data_mine.procedure_loop}}**
procedure_loop: "The procedure loop holds the JavaScript code that cycles through an input processing information to generate an output."
@@ -973,6 +1175,8 @@ toc: false
**{{site.data.data_mine.javascript_code}}**
javascript_code: "The JavaScript code node may hold any snippet of valid JavaScript."
@@ -983,6 +1187,8 @@ toc: false
**{{site.data.data_mine.procedure_initialization}}**
procedure_initialization: "The procedure initialization is used to initialize variables used by the procedure loop."
@@ -994,6 +1200,8 @@ toc: false
**{{site.data.data_mine.data_building_procedure}}**
data_building_procedure: "The data building procedure processes information to be fed to properties that are stored in the output dataset."
@@ -1008,6 +1216,8 @@ toc: false
**{{site.data.data_mine.plotter}}**
plotter: "A plotter is a device used to produce a graphic representation of the data generated by indicators."
@@ -1018,6 +1228,8 @@ toc: false
**{{site.data.data_mine.plotter_module}}**
plotter_module: "A plotter module is a device within a plotter that contains the definitions to create the visualization, often times, for a particular product definition."
@@ -1028,6 +1240,8 @@ toc: false
**{{site.data.data_mine.plotter_panel}}**
plotter_panel: "A plotter panel usually shows numerical representations of data over the charts on a small window or panel. The plotter panel node features definitions to make that possible."
@@ -1038,6 +1252,8 @@ toc: false
**{{site.data.data_mine.panel_data}}**
panel_data: "A panel data node represents a data field to be plotted on the plotter panel."
@@ -1048,6 +1264,8 @@ toc: false
**{{site.data.data_mine.data_formula}}**
data_formula: "A data formula is the device used to assign a value to the panel data, usually in the form of the name of the corresponding record property."
@@ -1058,6 +1276,8 @@ toc: false
**{{site.data.data_mine.shapes}}**
shapes: "The shapes node contains definitions that define polygons and styles to be rendered on a per period basis over the charts."
@@ -1068,6 +1288,8 @@ toc: false
**{{site.data.data_mine.chart_points}}**
chart_points: "The chart points node groups definitions of points."
@@ -1078,6 +1300,8 @@ toc: false
**{{site.data.data_mine.point}}**
point: "A point is a definition of an [x,y] coordinate where x is usually a datetime and y is usually a value that refers to a price, as output by the indicator."
@@ -1088,6 +1312,8 @@ toc: false
**{{site.data.data_mine.point_formula}}**
point_formula: "A point formula is the node used to input the coordinates of a point."
@@ -1098,6 +1324,8 @@ toc: false
**{{site.data.data_mine.polygon}}**
polygon: "A polygon features definitions that make up shapes and their styles."
@@ -1108,6 +1336,8 @@ toc: false
**{{site.data.data_mine.polygon_vertex}}**
polygon_vertex: "A polygon vertex is a reference to a point defined in the chart points, constituting a specific coordinate to be used as vertex of a shape."
@@ -1118,6 +1348,8 @@ toc: false
**{{site.data.data_mine.polygon_border}}**
polygon_border: "A polygon border is a shape defined by at least two vertices used to draw lines or to apply a style to the stroke of a closed shape."
@@ -1128,6 +1360,8 @@ toc: false
**{{site.data.data_mine.style}}**
style: "A style is a definition of color and opacity in case of a polygon body, and of color, opacity and line thickness in case of polygon borders."
@@ -1138,6 +1372,8 @@ toc: false
**{{site.data.data_mine.style_condition}}**
style_condition: "A style condition is a rule that allows applying an alternative style in cases where the rule validates true."
@@ -1149,6 +1385,6 @@ toc: false
**{{site.data.data_mine.polygon_body}}**
polygon_body: "A polygon body is is the fill of a shape made out of at least 3 vertices."
+3 -3
View File
@@ -55,11 +55,11 @@ toc: false
You will use the network hierarchy for three main purposes:
* To control processes running sensor and indicator bots. These keep your data feeds up to date so that you may trade live.
* To control processes running sensor and indicator bots. These keep your data feeds and analysis up to date so that you may trade live with quality information.
* To control sessions.
* To control trading sessions including backtesting, paper trading, forward testing, and live trading sessios.
* To set up which data products will be made available to your strategies, and for visualization purposes on the charts.
* To administer the physical location in which the data products produced by bots reside.
@@ -5,6 +5,8 @@ sidebar: suite_sidebar
permalink: suite-trading-sessions.html
---
{{site.data.network.session}}
{% include /network/backtesting-session.md heading="##" icon="150-" adding="" configuring="" starting="" content="n" definition="regular" table="y" %}
{% include /network/paper-trading-session.md heading="##" icon="150-" adding="" configuring="" starting="" content="n" definition="regular" table="y" %}