diff --git a/README.md b/README.md index c7d86e362..117153c10 100644 --- a/README.md +++ b/README.md @@ -736,6 +736,73 @@ These Node.js components provide the infrastructure required to run different ki When you click the RESTART BOTS button, several bots are executed in a specific order, taking into account dependencies, as defined in the ```CloneExecutor\sequence.json``` file. These processes run in a loop, retrieving data from the exchange, producing indicators, running simulations and trading live—online. +## How Algorithms Work + +Bots mission is—in essence—creating _products_ that others can consume. To do this, they run _processes_ which produce and store _datasets_. + +Each bot may have several processes, and processes don't necessarily have a one-to-one relationship with products. That is, a product can be the result of the work of one or more processes. + +Bot processes run when called by the app and stop when they finish the task at hand, to wake up again only when the sequence is completed and a new round of executions starts. + +The datasets processes create are the actual _output_ of bots which are stored in the file system as described earlier. But processes also produce and store a second valuable piece of information: _status reports_. + +Status reports serve as temporal annotations that bots read every time they are called by the app to know what was done in the previous run and what the state of affairs is at present. Status reports are dynamic, and they change constantly, with updates after every single run of the associated process. + +### Dependencies + +We established that bots produce products for others to consume. This _others_ include other algorithms, meaning that bots usually depend on the datasets produced by other bots. We call these _data dependencies_, which are declared on each bot configuration file. + +Bots consume their own status report and they might as well consume status reports from other algorithms. We call these _status dependencies_, which are too declared in each bot configuration file. + +### Types of Data Sets + +At this point, there are five different types of datasets: market files, daily files, minutes files, single file, and file sequence. These types of datasets define the structure of the data and how it is stored. + +A _market file_ contains data spanning the whole existence of the market, that is, from the day the pair _(e.g. USDT-BTC)_ started trading up to the present time. The data is stored in one single file, which is appended every time the process runs generating new data. + +A _daily file_ contains data segmented by day. That is, the process generates one file per day and stores it in the deepest level of a folder tree structure of the following type: ```Year > Month > Day```. + +A _minutes file_ contains data corresponding to one single minute and is stored in the deepest level of a folder tree structure of the following type: ```Year > Month > Day > Hour > Minute```. + +A _file sequence_ consists of sequential information that is not necessarily structured on any particular timeframe. The process stores two types of files: the one ending in _.Sequence.json_ contains the number of files in the sequence, and the sequence is formed by multiple files ending in a sequential number _(e.g. 15.json)_. +A _single file_ is pretty much just that: a dataset that is stored in one file only. + +## Current Bots Sequence + +Let's put all this in perspective by analyzing the processes, products, and dependencies of a few existing bots. + +### Charly + +Charly is a _sensor_. As his README file explains, he gets trades data for all markets—both historic and live—assuring consistency using recursive processes and store it in a highly fragmented and usable dataset. + +Charly offers one product which is defined by the dataset scope and various characteristics. Charly has three different processes: Live-Trades, Historic-Trades, and Hole-Fixing. These three processes combined generate the one single dataset that constitutes Charly's single product. + +The dataset is stored under the _minutes_ file structure. + +### Bruce + +Now, let's see what Bruce, an indicator, does with Charly's product. As you can learn from Bruce's README file he produces two datasets: candles at 1-minute resolution and volumes at 1-minute resolution. The datasets are stored under the _daily file_ type of dataset. + +Now scroll down the README file and see what Bruce's dependencies are. That's right! Bruce depends on Charly's product. Bruce's processes take the trades data that Charly extracted from the exchange, performs calculations to build 1-minute candles and stores his dataset with more elaborate data. + +In other words, Bruce is adding value to Charly's product and offering a new value-added product of his own. But the value-adding chain does not stop there... + +### Olivia + +Let's take a look at another indicator, Olivia. According to her README file Olivia offers four different products: candles at sub-hour resolutions, candles in resolutions above one hour, volumes in sub-hour resolutions and volumes in resolutions above one hour. And guess what? Indeed, Olivia uses Bruce's 1-minute candles and 1-minute volumes to produce complementary candles and volumes at different resolutions. + +### Tom + +[Tom](https://github.com/AAMasters/AATom-Indicator-Bot) uses candles from Bruce and Olivia to produce the candle-stairs pattern data set described in the [Candle Stairs Patterns](#candle-stairs-patterns) layer. + +### Chris + +[Chris](https://github.com/AAMasters/AAChris-Indicator-Bot) uses Olivia's candles to produce the [Bollinger Bands](#bollinger-bands) indicator. + +### Paula + +[Paula]() uses Chris' Bollinger Bands data set to build the [Bollinger Channel](#bollinger-channels) indicator. + ## Outputs Each of these bots produces an output in the form of JSON files, which are stored under the ```\Data-Storage\aamasters\AAMasters``` folder, sorted by bot. @@ -783,7 +850,7 @@ Each bot keeps its own set of log files, stored under a similar folder structure ``` Log files contain detailed information about each execution of the bot. As such, a new folder is created for each execution, labeled with the exact DateTime. -Each folder may contain more than one file. Lighter files tend to include data about the initialization stage, while heavier files usually feature the date corresponding to the actual work the bot does. +Each folder may contain more than one file. Lighter files tend to include data about the initialization stage, while heavier files usually feature the data corresponding to the actual work the bot does. ![Technical-Logs](https://user-images.githubusercontent.com/13994516/63350228-4f38ad00-c35d-11e9-8074-bdd73ac68bd8.gif)