Merge pull request #455 from julian-molina/develop

Contribution: Data Mines
This commit is contained in:
Julian Molina authored and GitHub committed 2021-01-24 11:23:31 +01:00
commit dd9fe3aed2
13 files changed
+1293 -4

No files matched your search

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 2.3 MiB

@@ -31,10 +31,6 @@
"text": "The market ending point is a status dependency existing both in Multi-Period-Market and Multi-Period-Daily processes so that the process establishing the reference knows the datetime of the end of the market.",
"updated": 1609787745299
},
{
"style": "Text",
"text": "The image above shows a case of a self-reference status dependency as well as a market ending point status dependency."
},
{
"style": "Block",
"text": "Configuring"
@@ -0,0 +1,106 @@
{
"topic": "Data Mines",
"pageNumber": "1",
"type": "Data Mine Hierarchy",
"definition": {
"text": "Data mines enable the processing of data by consuming datasets others have produced, performing calculations, and outputting a new value-added data product that others may consume.",
"updated": 1611407680590
},
"paragraphs": [
{
"style": "Text",
"text": "Superalgos puts a powerful data processing and visualization system in your hands. Use it to the fullest and you will see your intelligence increase by orders of magnitude.",
"updated": 1611407700153
},
{
"style": "Text",
"text": "Anyone may access raw trades and order book data from exchanges. Anyone may access information in the form of standard indicators and technical studies."
},
{
"style": "Callout",
"text": "However, Superalgos allows you to perform any sort of processing over existing information across the whole span of the market, store the results into new datasets, and make them available to trading strategies. Moreover, it allows you to produce elaborate graphical representations of data, which is instantly browsable over the charts."
},
{
"style": "Text",
"text": "Data mines’ functionality is highly flexible and works under the same paradigm used for building strategies. They provide a visual interface and a framework that leads you through the process of creating new bots, with minimal coding skills required."
},
{
"style": "Text",
"text": "The vast majority of the complexity of processing data is handled internally by the bots, and you don’t need to deal with it or even understand it. The system isolates complexity and only exposes you to the crucial elements that impact what you wish to accomplish."
},
{
"style": "Text",
"text": "This section of the documentation provides you with the technical understanding of how data is handled by the system. The core of the functionality this section unveils covers—in essence—the following points:"
},
{
"style": "Text",
"text": "The capacity to define any number of calculation processes that may feed on each other’s outputs, resulting in a wealth of information that may be consumed by trading strategies."
},
{
"style": "Text",
"text": "The properties and characteristics of datasets resulting from the execution of such processes—how and where they are stored, in what format, how they are consumed internally, and potentially, by other systems."
},
{
"style": "Text",
"text": "The capacity to visualize the data over the charts and browse it with all the features built in the charting system."
},
{
"style": "Title",
"text": "Data Mine",
"updated": 1611407778880
},
{
"style": "Include",
"text": "Superalgos->Node->Data Mine->Definition",
"updated": 1611407782909
},
{
"style": "Include",
"text": "Superalgos->Node->Data Mine->Content",
"updated": 1611407786577
},
{
"style": "Include",
"text": "Superalgos->Concept->Reusable Snippets->Note for Hierarchy Tables No Discussion"
},
{
"style": "Hierarchy",
"text": "Superalgos->Data Mine->2",
"updated": 1611407832651
},
{
"style": "Text",
"text": ""
},
{
"style": "Title",
"text": "Sensor Bot",
"updated": 1611407910652
},
{
"style": "Include",
"text": "Superalgos->Node->Sensor Bot->Definition",
"updated": 1611407914247
},
{
"style": "Title",
"text": "Indicator Bot",
"updated": 1611407935038
},
{
"style": "Include",
"text": "Superalgos->Node->Indicator Bot->Definition",
"updated": 1611407948292
},
{
"style": "Title",
"text": "Plotter",
"updated": 1611407956192
},
{
"style": "Include",
"text": "Superalgos->Node->Plotter->Definition",
"updated": 1611407972840
}
]
}
@@ -0,0 +1,30 @@
{
"topic": "Data Mines",
"pageNumber": "2",
"type": "Indicator Bots",
"definition": {
"text": "Indicator bots enable the processing of data by consuming datasets others have produced, performing calculations, and outputting a new value-added data product that others may consume.",
"updated": 1611408124910
},
"paragraphs": [
{
"style": "Include",
"text": "Superalgos->Node->Indicator Bot->Definition",
"updated": 1611408242603
},
{
"style": "Include",
"text": "Superalgos->Node->Indicator Bot->Content",
"updated": 1611408247595
},
{
"style": "Include",
"text": "Superalgos->Concept->Reusable Snippets->Note for Hierarchy Tables No Discussion"
},
{
"style": "Hierarchy",
"text": "Superalgos->Indicator Bot->2",
"updated": 1611408262027
}
]
}
@@ -0,0 +1,228 @@
{
"topic": "Data Mines",
"pageNumber": "3",
"type": "Indicator Bots - Process Definition",
"definition": {
"text": "The process definition is the compilation of configurations, dependencies, and references that make up a process.",
"updated": 1611410324543
},
"paragraphs": [
{
"style": "Include",
"text": "Superalgos->Node->Process Definition->Definition",
"updated": 1611408487552
},
{
"style": "Include",
"text": "Superalgos->Node->Process Definition->Content",
"updated": 1611408491273
},
{
"style": "Include",
"text": "Superalgos->Concept->Reusable Snippets->Note for Hierarchy Tables"
},
{
"style": "Hierarchy",
"text": "Superalgos->Process Definition->2",
"updated": 1611409733095
},
{
"style": "Title",
"text": "Process Output",
"updated": 1611408788144
},
{
"style": "Include",
"text": "Superalgos->Node->Process Output->Definition",
"updated": 1611408791768
},
{
"style": "Include",
"text": "Superalgos->Concept->Reusable Snippets->Note for Hierarchy Tables",
"updated": 1611409787758
},
{
"style": "Hierarchy",
"text": "Superalgos->Process Output->2"
},
{
"style": "Subtitle",
"text": "Output Dataset Folder",
"updated": 1611408893086
},
{
"style": "Include",
"text": "Superalgos->Node->Output Dataset Folder->Definition",
"updated": 1611408897031
},
{
"style": "Include",
"text": "Superalgos->Node->Output Dataset Folder->Content",
"updated": 1611408901554
},
{
"style": "Subtitle",
"text": "Output Dataset",
"updated": 1611408932688
},
{
"style": "Include",
"text": "Superalgos->Node->Output Dataset->Definition",
"updated": 1611408940761
},
{
"style": "Include",
"text": "Superalgos->Node->Output Dataset->Content",
"updated": 1611408945614
},
{
"style": "Title",
"text": "Process Dependencies",
"updated": 1611409011999
},
{
"style": "Include",
"text": "Superalgos->Node->Process Dependencies->Definition",
"updated": 1611409021282
},
{
"style": "Include",
"text": "Superalgos->Node->Process Dependencies->Content",
"updated": 1611409025810
},
{
"style": "Include",
"text": "Superalgos->Concept->Reusable Snippets->Note for Hierarchy Tables",
"updated": 1611409846972
},
{
"style": "Hierarchy",
"text": "Superalgos->Process Dependencies->4",
"updated": 1611409904800
},
{
"style": "Subtitle",
"text": "Status Dependency",
"updated": 1611409068335
},
{
"style": "Include",
"text": "Superalgos->Node->Status Dependency->Definition",
"updated": 1611409075880
},
{
"style": "Include",
"text": "Superalgos->Node->Status Dependency->Content",
"updated": 1611409079269
},
{
"style": "Subtitle",
"text": "Data Dependency",
"updated": 1611409226728
},
{
"style": "Include",
"text": "Superalgos->Node->Data Dependency->Definition",
"updated": 1611409243546
},
{
"style": "Include",
"text": "Superalgos->Node->Data Dependency->Content",
"updated": 1611409247489
},
{
"style": "Subtitle",
"text": "Data Mine Data Dependencies",
"updated": 1611409321781
},
{
"style": "Include",
"text": "Superalgos->Node->Data Mine Data Dependencies->Definition",
"updated": 1611409326033
},
{
"style": "Include",
"text": "Superalgos->Node->Data Mine Data Dependencies->Content",
"updated": 1611409329439
},
{
"style": "Subtitle",
"text": "Bot Data Dependencies",
"updated": 1611409418545
},
{
"style": "Include",
"text": "Superalgos->Node->Bot Data Dependencies->Definition",
"updated": 1611409422639
},
{
"style": "Include",
"text": "Superalgos->Node->Bot Data Dependencies->Content",
"updated": 1611409426189
},
{
"style": "Subtitle",
"text": "Data Dependency Folder",
"updated": 1611409491551
},
{
"style": "Include",
"text": "Superalgos->Node->Data Dependency Folder->Definition",
"updated": 1611409495392
},
{
"style": "Include",
"text": "Superalgos->Node->Data Dependency Folder->Content",
"updated": 1611409498720
},
{
"style": "Title",
"text": "Status Report",
"updated": 1611409535037
},
{
"style": "Title",
"text": ""
},
{
"style": "Include",
"text": "Superalgos->Node->Status Report->Definition",
"updated": 1611409555435
},
{
"style": "Include",
"text": "Superalgos->Node->Status Report->Content",
"updated": 1611409559374
},
{
"style": "Title",
"text": "Execution Started Event",
"updated": 1611409586862
},
{
"style": "Include",
"text": "Superalgos->Node->Execution Started Event->Definition",
"updated": 1611409590304
},
{
"style": "Include",
"text": "Superalgos->Node->Execution Started Event->Content",
"updated": 1611409593797
},
{
"style": "Title",
"text": "Execution Finished Event",
"updated": 1611409630398
},
{
"style": "Include",
"text": "Superalgos->Node->Execution Finished Event->Definition",
"updated": 1611409635844
},
{
"style": "Include",
"text": "Superalgos->Node->Execution Finished Event->Content",
"updated": 1611409640116
}
]
}
@@ -0,0 +1,181 @@
{
"topic": "Data Mines",
"pageNumber": "4",
"type": "Indicator Bots - Product Definition",
"definition": {
"text": "The product definition is the compilation of configurations, references, and JavaScript code that make up the data product.",
"updated": 1611425764441
},
"paragraphs": [
{
"style": "Include",
"text": "Superalgos->Node->Product Definition->Definition",
"updated": 1611425809992
},
{
"style": "Include",
"text": "Superalgos->Node->Product Definition->Content",
"updated": 1611425814195
},
{
"style": "Include",
"text": "Superalgos->Concept->Reusable Snippets->Note for Hierarchy Tables"
},
{
"style": "Hierarchy",
"text": "Superalgos->Product Definition->2",
"updated": 1611425866970
},
{
"style": "Title",
"text": "Dataset Definition",
"updated": 1611425958957
},
{
"style": "Include",
"text": "Superalgos->Node->Dataset Definition->Definition",
"updated": 1611425968181
},
{
"style": "Include",
"text": "Superalgos->Node->Dataset Definition->Content",
"updated": 1611425972488
},
{
"style": "Title",
"text": "Record Definition",
"updated": 1611426032529
},
{
"style": "Include",
"text": "Superalgos->Node->Record Definition->Definition",
"updated": 1611426038754
},
{
"style": "Include",
"text": "Superalgos->Node->Record Definition->Content",
"updated": 1611426042645
},
{
"style": "Include",
"text": "Superalgos->Concept->Reusable Snippets->Note for Hierarchy Tables",
"updated": 1611426075537
},
{
"style": "Hierarchy",
"text": "Superalgos->Record Definition->4",
"updated": 1611426088183
},
{
"style": "Subtitle",
"text": "Record Property",
"updated": 1611426123252
},
{
"style": "Include",
"text": "Superalgos->Node->Record Property->Definition",
"updated": 1611426129988
},
{
"style": "Include",
"text": "Superalgos->Node->Record Property->Content",
"updated": 1611426133406
},
{
"style": "Subtitle",
"text": "Record Formula",
"updated": 1611426162635
},
{
"style": "Text",
"text": ""
},
{
"style": "Include",
"text": "Superalgos->Node->Record Formula->Definition",
"updated": 1611426176713
},
{
"style": "Title",
"text": "Calculations Procedure",
"updated": 1611426226554
},
{
"style": "Include",
"text": "Superalgos->Node->Calculations Procedure->Definition",
"updated": 1611426233636
},
{
"style": "Include",
"text": "Superalgos->Node->Calculations Procedure->Content",
"updated": 1611426238550
},
{
"style": "Include",
"text": "Superalgos->Concept->Reusable Snippets->Note for Hierarchy Tables"
},
{
"style": "Hierarchy",
"text": "Superalgos->Calculations Procedure->5",
"updated": 1611426280589
},
{
"style": "Subtitle",
"text": "Procedure Initialization",
"updated": 1611426543003
},
{
"style": "Include",
"text": "Superalgos->Node->Procedure Initialization->Definition",
"updated": 1611426555977
},
{
"style": "Subtitle",
"text": "Procedure Loop",
"updated": 1611426364374
},
{
"style": "Include",
"text": "Superalgos->Node->Procedure Loop->Definition",
"updated": 1611426368953
},
{
"style": "Subtitle",
"text": "Procedure Javascript Code",
"updated": 1611426387898
},
{
"style": "Include",
"text": "Superalgos->Node->Procedure Javascript Code->Definition",
"updated": 1611426391763
},
{
"style": "Title",
"text": "Data Building Procedure",
"updated": 1611426735172
},
{
"style": "Include",
"text": "Superalgos->Node->Data Building Procedure->Definition",
"updated": 1611426741534
},
{
"style": "Include",
"text": "Superalgos->Node->Data Building Procedure->Content",
"updated": 1611426749058
},
{
"style": "Include",
"text": "Superalgos->Concept->Reusable Snippets->Note for Hierarchy Tables"
},
{
"style": "Hierarchy",
"text": "Superalgos->Data Building Procedure->3",
"updated": 1611426785067
},
{
"style": "Text",
"text": ""
}
]
}
@@ -0,0 +1,122 @@
{
"topic": "Data Mines",
"pageNumber": "6",
"type": "Indicator Bots - Data Mining Workflow",
"definition": {
"text": "When running on a data mining operation, indicators follow a cycle designed to fulfill their purpose: to process data others have produced and make the result available for others to consume.",
"updated": 1611427674749
},
"paragraphs": [
{
"style": "Text",
"text": "Indicators take datasets produced by sensors or by other indicators as input. These inputs are determined by the data dependencies defined. The calculations procedure and the data building procedure produce the intended transformations. The data building procedure stores the results in datasets that others may use as input, and the calculations procedure may make certain pieces of information available without ever writing it in the files.",
"updated": 1611427699110
},
{
"style": "Text",
"text": "The following is the breakdown of the cycle."
},
{
"style": "Title",
"text": "1. Loading of data dependencies"
},
{
"style": "Text",
"text": "Datasets produced by indicators are sets of text files with data stored in the form of arrays. Each value in a record is separated by colons, and each record is delimited by square brackets. Records are too separated by colons. The format in which data is stored in files aims to make files as lightweight as possible."
},
{
"style": "Text",
"text": "For example, the following fragment features three records in the 24-hours Multi-Period-Market Bollinger Bands Data.json file:"
},
{
"style": "Json",
"text": "... ,[1586044800000,1586131199999,6339.464,440.8934655492186,881.7869310984372],[1586131200000,1586217599999,6440.327,424.9041072418576,849.8082144837152],[1586217600000,1586303999999,6530.541,382.3824677453191,764.7649354906382], ..."
},
{
"style": "Text",
"text": "That is what the system encounters at the time of reading a typical file in a dataset, which is the first step in the runtime cycle."
},
{
"style": "Title",
"text": "2. Inflation of files into JSON objects with named properties"
},
{
"style": "Text",
"text": "Once a file is loaded in memory, it is inflated into a JSON object structure, with named properties. This is the second step in the cycle and aims to make accessing each record—from within the code—much easier."
},
{
"style": "Title",
"text": "3. Calculation of properties not stored in the dataset"
},
{
"style": "Text",
"text": "Because performance is inversely related to the weight of files, developers may choose not to store all indicator properties in files. They may choose to leave properties that are easy to calculate out of the dataset and make them available as calculated properties instead. Calculated properties are calculated by the calculations procedure and fed to the record definition formulas to be made available as an integral part of the object other indicators may access."
},
{
"style": "Text",
"text": "For example, picture an indicator that offers buy, sell, and total volumes. Because the total volume is so easy to calculate, the developer of such an indicator may choose to store the buy and sell volumes only and make the total volume available as a calculated property."
},
{
"style": "Success",
"text": "Make your indicators more efficient by limiting the amount of information you store in files.",
"updated": 1611428939223
},
{
"style": "Text",
"text": "The third step in the runtime cycle is, therefore, to run the calculations procedure of each data dependency and add the calculated properties to the JSON object structure in memory, so that all properties are available for the next steps in the cycle."
},
{
"style": "Title",
"text": "4. Products structure"
},
{
"style": "Text",
"text": "Steps 1, 2, and 3 are run for each one of the data dependencies, and the results are put inside an object named products. In this way, all data dependencies may be easily accessed with the following kind of syntax:"
},
{
"style": "Javascript",
"text": "products.pluralVariableName[index].property"
},
{
"style": "Text",
"text": "For example: ",
"updated": 1611428967240
},
{
"style": "Javascript",
"text": "products.bollingerBands[10].deviation"
},
{
"style": "Text",
"text": "Each product remains an array of JSON objects with named variables."
},
{
"style": "Title",
"text": "5. Data building"
},
{
"style": "Text",
"text": "The fifth step in the runtime cycle is running the indicator’s data building procedure. This is where the developer codes the logic to go over the input datasets and perform the calculations required to produce the indicator’s output."
},
{
"style": "Text",
"text": "Indicators may have multiple products, thus, the data building procedure corresponding to each product is run sequentially, in the order products have been defined around the indicator bot node."
},
{
"style": "Text",
"text": "As transformations for a given product are finished, the results are appended to the products object and made available immediately for the rest of the products in the queue to use as input, in addition to the original data dependencies. Data dependencies are the same for all products, as data dependencies are defined at the level of processes."
},
{
"style": "Title",
"text": "6. Writing the output"
},
{
"style": "Text",
"text": "The final step is turning the object-formatted records back into a minimized array format, and saving the corresponding files in the dataset."
},
{
"style": "Text",
"text": ""
}
]
}
@@ -0,0 +1,194 @@
{
"topic": "Data Mines",
"pageNumber": "7",
"type": "Indicator Bots - Coding the Indicator's Logic",
"definition": {
"text": "Learn how to use data dependencies, the variable object, and how to deal with indicators with irregular periods.",
"updated": 1611429211176
},
"paragraphs": [
{
"style": "Title",
"text": "Main Data Dependency",
"updated": 1611430812568
},
{
"style": "Text",
"text": "The data building procedure works under the premise that the algorithm loops around a data collection and that—for each loop—generates a record for the product being built. For example, the SMA indicator loops around the candles product of the Candles Volumes indicator, thus, candles is the main data dependency. In the case of the bollingerChannels product, the main data dependency is the bollingerBands product of the Bollinger Bands indicator."
},
{
"style": "Text",
"text": "The main data dependency is the first dependency declared at the level of the processes. This dependency is made available in the record object, which features three properties:"
},
{
"style": "List",
"text": "previous: ",
"updated": 1611429276716
},
{
"style": "Javascript",
"text": "mainDependency.records[index - 1]"
},
{
"style": "List",
"text": "current: ",
"updated": 1611429281291
},
{
"style": "Javascript",
"text": "mainDependency.records[index]",
"updated": 1611429297570
},
{
"style": "List",
"text": "next: ",
"updated": 1611429287433
},
{
"style": "Javascript",
"text": "mainDependency.records[index + 1]"
},
{
"style": "Text",
"text": "The variable index is the index in the array of the main dependency."
},
{
"style": "Text",
"text": "For example, you may use the statement record.propertyName to access the value of any of the properties in the input datasets for the current record, or record.previous.propertyName to access the value of the property in the previous record."
},
{
"style": "Note",
"text": "The order in which dependencies are defined determines which one is the main data dependency.",
"updated": 1611429327162
},
{
"style": "Title",
"text": "Multiple Data Dependencies",
"updated": 1611430826204
},
{
"style": "Text",
"text": "The products object features a property for each data dependency, and each property is named with the pluralVariableName defined in the product’s configuration. Each property features an array with the dependency’s data collection."
},
{
"style": "Text",
"text": "Data collections of different products may begin and end at different points in time. For example, it takes 20 candles to build the first bollingerBands object of the Bollinger Bands indicator, or 200 candles to build the first popularSMAs object of the SMA indicator. Therefore, the record in i position in the array of one product may not correspond to the same period of time of the record in the same position of a different product."
},
{
"style": "Text",
"text": "To overcome that hurdle, the getElement function, which takes two parameters, may be used:"
},
{
"style": "List",
"text": "The name of the data collection (pluralVariableName) you wish to consult."
},
{
"style": "List",
"text": "An object with begin and end properties, for example, the tenth candle."
},
{
"style": "Text",
"text": "The function returns the record that matches the begin and end datetime of the second parameter."
},
{
"style": "Text",
"text": "This is an example from the Candle Patterns indicator, which has the candles product of the Candles Volumes indicator as the main data dependency:"
},
{
"style": "Javascript",
"text": "let candle = record.current // Calling \"Candles\"\nlet basicCandle = getElement ('basicCandles', candle) // Calling \"Basic Candles\"",
"updated": 1611430980056
},
{
"style": "Title",
"text": "Variable Object",
"updated": 1611431014179
},
{
"style": "Text",
"text": "The output generated must be stored as new properties of the variable object, which is made available to record properties under the record definition node."
},
{
"style": "Text",
"text": "In the case of the data building procedure loop, the variable object may also be used to store and—later—retrieve information between different loop cycles, or even between different executions of the indicator process. Such a feature is not available in the calculations procedure loop, as the object does not persist beyond each loop in that context."
},
{
"style": "Title",
"text": "Multi-Period-Daily"
},
{
"style": "Text",
"text": "In time frames below one hour, the system allows to access data up to 48 hours in the past, relative to the current record. If more than 48-hours worth of data is required to produce the desired data building, then data must be stored into the variable object to be retrieved later on, as explained above."
},
{
"style": "Important",
"text": "Procedures may only access data up to 48 hours in the past counting from the datetime of the record being processed. For example, the calculation of a moving average spanning 100 periods requires temporarily storing data in an array, as the 45 minutes time frame needs 75-hours worth of data.",
"updated": 1611431032118
},
{
"style": "Text",
"text": "The following is an example of a procedure loop code, in particular, the code that calculates the Popular SMAs product of the Simple Moving Average (SMA) indicator. Notice how the output generated by the calculateSMA function is stored into the variable object, and how an array stored in the variable object is used to overcome the 48-hours data-access limitation of Multi-Priod-Daily time frames."
},
{
"style": "Javascript",
"text": "/* Loop Code. */\n\nlet candle = record.current // Our main dependency is candles \nvariable.last200.push(candle.close) // Add the current close value to the last 200 array.\n\nif (variable.last200.length > 200) { // Store data in the variable object to overcome the limitations in multi-period-daily.\n variable.last200.splice(0, 1) // Remove the first element of the array to keep it at a maximun of 200 elements.\n}\n\nvariable.sma20 = calculateSMA(20)\nvariable.sma50 = calculateSMA(50)\nvariable.sma100 = calculateSMA(100)\nvariable.sma200 = calculateSMA(200)\n\nfunction calculateSMA(periods) { // Having a function saves us from duplicating code.\n /* We check we have enough values to make the calculation */\n if (variable.last200.length < periods) { return 0 } // If we dont, we define the value is zero.\n\n let sum = 0 // Initialize sum variable. \n for (let i = variable.last200.length - periods; i < variable.last200.length; i++) { // Iterate through the last periods\n sum = sum + variable.last200[i]\n }\n let sma = sum / periods\n return sma\n}",
"updated": 1611431047895
},
{
"style": "Title",
"text": "Irregular Periods",
"updated": 1611431066723
},
{
"style": "Text",
"text": "Occasionally, a one-to-one mapping of the periods of the data dependency with the periods of the resulting product does not exist. For example, in the case of the bollingerChannels product, using the bollingerBands as the main dependency. In such a case, the algorithm may read the 1-hour bollingerBands as input but the resulting object may span several hours."
},
{
"style": "Text",
"text": "By default, the system pushes a record into the data collection for every loop. To build products that span several periods of the main dependency, you must make the push from within your own code."
},
{
"style": "Text",
"text": "When the system detects a push in the procedure loop code, it cancels the automatic push on each loop."
},
{
"style": "Text",
"text": "The following example taken from the data building procedure loop of the Bollinger Channels Object indicator features several instances of arbitrary use of pushes."
},
{
"style": "Javascript",
"text": "if (record.next !== undefined) {\n\n if (\n record.current.direction === record.next.direction) {\n\n if (variable.channel === undefined) {\n\n variable.channel = {\n begin: undefined,\n end: undefined,\n direction: undefined,\n period: 0,\n firstMovingAverage: 0,\n lastMovingAverage: 0,\n firstDeviation: 0,\n lastDeviation: 0\n }\n\n variable.channel.direction = record.current.direction\n variable.channel.period = 2\n\n variable.channel.begin = record.current.begin\n variable.channel.end = record.next.end\n\n variable.channel.firstMovingAverage = record.current.movingAverage\n variable.channel.lastMovingAverage = record.next.movingAverage\n\n variable.channel.firstDeviation = record.current.deviation\n variable.channel.lastDeviation = record.next.deviation\n\n } else {\n\n variable.channel.period++\n variable.channel.end = record.next.end\n variable.channel.lastMovingAverage = record.next.movingAverage\n variable.channel.lastDeviation = record.next.deviation\n\n }\n } else {\n\n if (variable.channel !== undefined) {\n results.push(variable.channel)\n variable.channel = undefined\n } else {\n /* The variable.channel has only one period */\n\n variable.channel = {}\n\n variable.channel.direction = record.current.direction\n variable.channel.period = 1\n\n variable.channel.begin = record.current.begin\n variable.channel.end = record.current.end\n\n variable.channel.firstMovingAverage = record.current.movingAverage\n variable.channel.lastMovingAverage = record.current.movingAverage\n\n variable.channel.firstDeviation = record.current.deviation\n variable.channel.lastDeviation = record.current.deviation\n\n results.push(variable.channel)\n variable.channel = undefined\n }\n }\n} else {\n if (variable.channel !== undefined) {\n variable.channel.period++\n variable.channel.end = record.current.end\n variable.channel.lastMovingAverage = record.current.movingAverage\n variable.channel.lastDeviation = record.current.deviation\n results.push(variable.channel)\n variable.channel = undefined\n } else {\n /* The variable.channel has only one period */\n\n variable.channel = {}\n\n variable.channel.direction = record.current.direction\n variable.channel.period = 1\n\n variable.channel.begin = record.current.begin\n variable.channel.end = record.current.end\n\n variable.channel.firstMovingAverage = record.current.movingAverage\n variable.channel.lastMovingAverage = record.current.movingAverage\n\n variable.channel.firstDeviation = record.current.deviation\n variable.channel.lastDeviation = record.current.deviation\n\n results.push(variable.channel)\n variable.channel = undefined\n }\n}",
"updated": 1611431088524
},
{
"style": "Title",
"text": "Managing Undefined Objects",
"updated": 1611431116629
},
{
"style": "Text",
"text": "In many instances, objects may be undefined. For example, the first 199 candles in the data collection corresponding to a 200-period moving average may be undefined, as the data building procedure may only build the object once the 200th candle is evaluated."
},
{
"style": "Text",
"text": "Storing undifined property values is highly undesirable, as that would likely produce errors downstream when others attempt to consume the products produced by the indicator. How to handle such circumstances is up to the developer."
},
{
"style": "Text",
"text": "The Bollinger Channels Object example above features deliberate handling of undefined objects."
},
{
"style": "Text",
"text": "In the case of a 200-period moving average discussed above, the developer may want to produce an arbitrary estimation of the moving average for the first 199 candles, or simply set the value to zero for the first 199 candles. The later was the choice for the popularSMAs product as may be observed in the code of the initialization procedure:"
},
{
"style": "Javascript",
"text": "/* Initialization Code*/\n\nvariable.previousEMA20 = 0\nvariable.previousEMA50 = 0\nvariable.previousEMA100 = 0\nvariable.previousEMA200 = 0",
"updated": 1611431131070
},
{
"style": "Text",
"text": ""
}
]
}
@@ -0,0 +1,49 @@
{
"topic": "Data Mines",
"pageNumber": "8",
"type": "Plotters",
"definition": {
"text": "Plotters read data products created by bots and map certain coordinate points to use them as vertices for polygons, which are assigned styles to create graphic representations of the underlying data.",
"updated": 1611431276108
},
"paragraphs": [
{
"style": "Include",
"text": "Superalgos->Node->Plotter->Definition",
"updated": 1611431307735
},
{
"style": "Include",
"text": "Superalgos->Node->Plotter->Content",
"updated": 1611431313998
},
{
"style": "Include",
"text": "Superalgos->Concept->Reusable Snippets->Note for Hierarchy Tables"
},
{
"style": "Hierarchy",
"text": "Superalgos->Plotter->2",
"updated": 1611431826589
},
{
"style": "Text",
"text": ""
},
{
"style": "Title",
"text": "Plotter Module",
"updated": 1611431881154
},
{
"style": "Include",
"text": "Superalgos->Node->Plotter Module->Definition",
"updated": 1611431891320
},
{
"style": "Include",
"text": "Superalgos->Node->Plotter Module->Content",
"updated": 1611431894467
}
]
}
@@ -0,0 +1,70 @@
{
"topic": "Data Mines",
"pageNumber": "9",
"type": "Plotters - Plotter Panel",
"definition": {
"text": "A plotter panel allows building panels with numerical representations of data.",
"updated": 1611432021846
},
"paragraphs": [
{
"style": "Include",
"text": "Superalgos->Node->Plotter Panel->Definition",
"updated": 1611432049796
},
{
"style": "Include",
"text": "Superalgos->Node->Plotter Panel->Content",
"updated": 1611432052734
},
{
"style": "Include",
"text": "Superalgos->Concept->Reusable Snippets->Note for Hierarchy Tables"
},
{
"style": "Hierarchy",
"text": "Superalgos->Plotter Panel->4",
"updated": 1611432061472
},
{
"style": "Title",
"text": "Panel Data",
"updated": 1611432322437
},
{
"style": "Include",
"text": "Superalgos->Node->Panel Data->Definition",
"updated": 1611432326601
},
{
"style": "Include",
"text": "Superalgos->Node->Panel Data->Content",
"updated": 1611432330023
},
{
"style": "Subtitle",
"text": "Data Formula",
"updated": 1611432338583
},
{
"style": "Include",
"text": "Superalgos->Node->Data Formula->Definition",
"updated": 1611432348376
},
{
"style": "Subtitle",
"text": "Text Style",
"updated": 1611432394440
},
{
"style": "Include",
"text": "Superalgos->Node->Text Style->Definition",
"updated": 1611432402298
},
{
"style": "Include",
"text": "Superalgos->Node->Text Style->Content",
"updated": 1611432405386
}
]
}
@@ -0,0 +1,202 @@
{
"topic": "Data Mines",
"pageNumber": "10",
"type": "Plotters - Shapes",
"definition": {
"text": "Shapes define coordinate points to use as vertex for polygons, which are assigned styles.",
"updated": 1611432676884
},
"paragraphs": [
{
"style": "Include",
"text": "Superalgos->Node->Shapes->Definition",
"updated": 1611432703073
},
{
"style": "Include",
"text": "Superalgos->Node->Shapes->Content",
"updated": 1611432706401
},
{
"style": "Include",
"text": "Superalgos->Concept->Reusable Snippets->Note for Hierarchy Tables"
},
{
"style": "Hierarchy",
"text": "Superalgos->Shapes->2",
"updated": 1611432712346
},
{
"style": "Title",
"text": "Chart Points",
"updated": 1611432774706
},
{
"style": "Include",
"text": "Superalgos->Node->Chart Points->Definition",
"updated": 1611432778350
},
{
"style": "Include",
"text": "Superalgos->Node->Chart Points->Content",
"updated": 1611432781813
},
{
"style": "Include",
"text": "Superalgos->Concept->Reusable Snippets->Note for Hierarchy Tables"
},
{
"style": "Hierarchy",
"text": "Superalgos->Chart Points->8",
"updated": 1611432807680
},
{
"style": "Title",
"text": "Point",
"updated": 1611432845898
},
{
"style": "Include",
"text": "Superalgos->Node->Point->Definition",
"updated": 1611432849816
},
{
"style": "Include",
"text": "Superalgos->Node->Point->Content",
"updated": 1611432853027
},
{
"style": "Subtitle",
"text": "Point Formula",
"updated": 1611432862646
},
{
"style": "Include",
"text": "Superalgos->Node->Point Formula->Definition",
"updated": 1611432866497
},
{
"style": "Include",
"text": "Superalgos->Node->Point Formula->Content",
"updated": 1611432870720
},
{
"style": "Title",
"text": "Polygon",
"updated": 1611433008071
},
{
"style": "Include",
"text": "Superalgos->Node->Polygon->Definition",
"updated": 1611433012029
},
{
"style": "Include",
"text": "Superalgos->Node->Polygon->Content",
"updated": 1611433015599
},
{
"style": "Include",
"text": "Superalgos->Concept->Reusable Snippets->Note for Hierarchy Tables"
},
{
"style": "Hierarchy",
"text": "Superalgos->Polygon->8",
"updated": 1611433074724
},
{
"style": "Text",
"text": ""
},
{
"style": "Title",
"text": "Polygon Condition",
"updated": 1611433130396
},
{
"style": "Include",
"text": "Superalgos->Node->Polygon Condition->Definition",
"updated": 1611433134852
},
{
"style": "Include",
"text": "Superalgos->Node->Polygon Condition->Content",
"updated": 1611433139614
},
{
"style": "Title",
"text": "Polygon Body",
"updated": 1611433166294
},
{
"style": "Include",
"text": "Superalgos->Node->Polygon Body->Definition",
"updated": 1611433175296
},
{
"style": "Include",
"text": "Superalgos->Node->Polygon Body->Content",
"updated": 1611433178676
},
{
"style": "Subtitle",
"text": "Style",
"updated": 1611433231745
},
{
"style": "Include",
"text": "Superalgos->Node->Style->Definition",
"updated": 1611433235523
},
{
"style": "Include",
"text": "Superalgos->Node->Style->Content",
"updated": 1611433238663
},
{
"style": "Subtitle",
"text": "Style Condition",
"updated": 1611433273738
},
{
"style": "Include",
"text": "Superalgos->Node->Style Condition->Definition",
"updated": 1611433277376
},
{
"style": "Include",
"text": "Superalgos->Node->Style Condition->Content",
"updated": 1611433280995
},
{
"style": "Title",
"text": "Polygon Border",
"updated": 1611433347131
},
{
"style": "Include",
"text": "Superalgos->Node->Polygon Border->Definition",
"updated": 1611433354769
},
{
"style": "Include",
"text": "Superalgos->Node->Polygon Border->Content",
"updated": 1611433358287
},
{
"style": "Title",
"text": "Polygon Vertex",
"updated": 1611433401279
},
{
"style": "Include",
"text": "Superalgos->Node->Polygon Vertex->Definition",
"updated": 1611433405348
},
{
"style": "Include",
"text": "Superalgos->Node->Polygon Vertex->Content",
"updated": 1611433408699
}
]
}
@@ -0,0 +1,111 @@
{
"topic": "Data Mines",
"pageNumber": "11",
"type": "How to Build an Indicator - Video Series",
"definition": {
"text": "This video series goes through the entire process of building an indicator, step by step.",
"updated": 1611434001380
},
"paragraphs": [
{
"style": "Text",
"text": "This video series focuses on the “how to” aspect of building an indicator. It does not go through definitions nor does it explain the underlying concepts. Therefore, it is highly recommended to read the data mines page as well as the pages about indicator bots before attempting to build an indicator.",
"updated": 1611434011723
},
{
"style": "Note",
"text": "Reading the whole documentation on indicators and plotters is certainly desirable and will save you time in the long run.",
"updated": 1611434037598
},
{
"style": "Important",
"text": "The video series on how to build an indicator was produced for the Beta 5 version of the system; however, the functionality still works similarly. That said, there may be minor differences that are yet to be documented. Please drop by the Develop group and ask any questions you may have, particularly if something is not working as expected, or as instructed in the videos.",
"updated": 1611434212027
},
{
"style": "Title",
"text": "Step 1 - Visual Definitions",
"updated": 1611434706444
},
{
"style": "Text",
"text": "Setting up the multi-period-market and multi-period-daily process definitions, status and data dependencies, execution events, and dataset definitions for the main product.",
"updated": 1611434720860
},
{
"style": "Youtube",
"text": "KXKKS_o8Zjw",
"updated": 1611434740393
},
{
"style": "Title",
"text": "Step 2 - Code & Record Definition",
"updated": 1611435476284
},
{
"style": "Text",
"text": "Reviewing common coding practice and configuring record properties and their formulas.",
"updated": 1611435495283
},
{
"style": "Youtube",
"text": "zx5N5D2tkwo",
"updated": 1611435834819
},
{
"style": "Title",
"text": "Step 3 - Testing & Troubleshooting",
"updated": 1611435879103
},
{
"style": "Text",
"text": "Setting up a data mining task to run the bot instance, checking the dataset files for errors, and troubleshooting common issues with the code and the process definitions.",
"updated": 1611435892931
},
{
"style": "Youtube",
"text": "gjv5lPmLVSI",
"updated": 1611435919312
},
{
"style": "Title",
"text": "Step 4 - Plotters",
"updated": 1611435948687
},
{
"style": "Text",
"text": "How to create the device that will interpret the dataset generated by the indicator and render a graphical representation over the charts.",
"updated": 1611435955116
},
{
"style": "Youtube",
"text": "UBtUtUfZPk4",
"updated": 1611436312247
},
{
"style": "Title",
"text": "Step 5 - Trading Bot Dependencies",
"updated": 1611436344230
},
{
"style": "Text",
"text": "Before you may use the new indicator on a trading system, you must create the corresponding dependencies on the Low frequency trading bot.",
"updated": 1611436373962
},
{
"style": "Text",
"text": "To call the properties of a new indicator from within the definitions of a trading system, you first need to let the Low Frequency trading bot that your data mine exists. To do this, you need to add the corresponding data dependencies at the trading bot’s Trading process definitions.",
"updated": 1611436404033
},
{
"style": "Gif",
"text": "GIFs/Superalgos/Docs/design-space/how-to-build-an-indicator-step-5.gif",
"updated": 1611436528224
},
{
"style": "Important",
"text": "Because the Low Frequency trading bot is a plugin, the modifications you make to the hierarchy do not persist after closing the frontend app. The workaround is to take a backup of the hierarchy after making the modifications, and replace the plugin file with the backup. Should you decide to contribute your data mine to the project, we will incorporate the dependencies in the official release.",
"updated": 1611436549743
}
]
}