docs improved

2020-02-22 16:42:05 +01:00
1 parent 7fe0ba4927
commit 3228d3ac75
@@ -0,0 +1,11 @@
+ATOMIZE:
+
+Hierarchies tables, by segment.
+
+How to pages, create template.
+
+* Find each hierarchy. Create hierarchies map.
+* Set up and exchange account.
+* Set up an exchange account key.
+* Set up an exchange account key instance.
+* Expand and collapse data structures.
@@ -34,5 +34,9 @@ dataset: "A dataset is a collection of plain-text files storing information in t

 process: "A process is an algorithm defined in the process definition structure of nodes of the corresponding bot and data mine."

+masters_data_mine: "Masters is a data mine shipping with the system. It features the main trading bot and sensor bot, along with indicators offering the following data products: Candles, Volumes, Candle Stairs Patterns, Volume Stairs Patterns, Bollinger Bands, Percentage Bandwidth, Bollinger Channels, and Bollinger Sub-Channels."
+
+sparta_data_mine: "Sparta is a data mine shipping with the system. It features indicators offering the following data products: Relative Strength Index (RSI 14), Simple Moving Average (SMA), Exponential Moving Average (EMA), and Moving Average Convergence/Divergence (MACD)."
+
 pending: "balanceAssetA, balanceAssetB, overfitting, supported assets, supported exchanges, supported markets"

@@ -34,5 +34,9 @@ dataset: "A dataset is a collection of plain-text files storing information in t

 process: "A process is an algorithm defined in the process definition structure of nodes of the corresponding bot and data mine."

+masters_data_mine: "Masters is a data mine shipping with the system. It features the main trading bot and sensor bot, along with indicators offering the following data products: Candles, Volumes, Candle Stairs Patterns, Volume Stairs Patterns, Bollinger Bands, Percentage Bandwidth, Bollinger Channels, and Bollinger Sub-Channels."
+
+sparta_data_mine: "Sparta is a data mine shipping with the system. It features indicators offering the following data products: Relative Strength Index (RSI 14), Simple Moving Average (SMA), Exponential Moving Average (EMA), and Moving Average Convergence/Divergence (MACD)."
+
 pending: "balanceAssetA, balanceAssetB, overfitting, supported assets, supported exchanges, supported markets"

@@ -1,4 +1,4 @@
-data_mine: "A data mine is a hierarchy that contains definitions regarding bots, including the trading bot, sensors, indicators, and even plotters. These definitions make up the actual programs of such bots&mdash;the source code."
+data_mine: "A data mine is a hierarchy that contains definitions of bots, including the trading bot, sensors, indicators, and even plotters. These definitions make up the actual programs of such bots&mdash;the source code."

 sensor_bot: "A sensor bot is an algorithm that extracts raw data from external sources (i.e.: exchanges, Twitter, etc.) and stores it in a dataset that other bots may consume."

@@ -1,4 +1,4 @@
-data_mine: "A data mine is a hierarchy that contains definitions regarding bots, including the trading bot, sensors, indicators, and even plotters. These definitions make up the actual programs of such bots&mdash;the source code."
+data_mine: "A data mine is a hierarchy that contains definitions of bots, including the trading bot, sensors, indicators, and even plotters. These definitions make up the actual programs of such bots&mdash;the source code."

 sensor_bot: "A sensor bot is an algorithm that extracts raw data from external sources (i.e.: exchanges, Twitter, etc.) and stores it in a dataset that other bots may consume."

@@ -2,7 +2,7 @@ network: "The network hierarchy contains definitions regarding the physical loca

 network_node: "A network node represents a machine in the network on which processes run or data is stored."

-data_mining: "Data mining is the activity of processing data. You need to process data so that the trading bot may make decisions based on quality information. In the context of the network hierarchy, the data-mining node groups the task managers handling sensor and indicator bots instances."
+data_mining: "Data mining is the activity of processing data. You need to process data to feed charts, and so that the trading bot may make decisions based on quality information. In the context of the network hierarchy, the data-mining node groups the task managers handling sensor and indicator bots instances."

 testing_environment: "The testing environment node organizes trading sessions involving testing of trading systems."

@@ -2,7 +2,7 @@ network: "The network hierarchy contains definitions regarding the physical loca

 network_node: "A network node represents a machine in the network on which processes run or data is stored."

-data_mining: "Data mining is the activity of processing data. You need to process data so that the trading bot may make decisions based on quality information. In the context of the network hierarchy, the data-mining node groups the task managers handling sensor and indicator bots instances."
+data_mining: "Data mining is the activity of processing data. You need to process data to feed charts, and so that the trading bot may make decisions based on quality information. In the context of the network hierarchy, the data-mining node groups the task managers handling sensor and indicator bots instances."

 testing_environment: "The testing environment node organizes trading sessions involving testing of trading systems."

@@ -56,14 +56,18 @@ entries:
      url: /suite-step-3.html
      output: web

-    - title: "Step 4: Quick overview"
+    - title: "Step 4: Import a workspace"
      url: /suite-step-4.html
      output: web

-    - title: "Step 5: Import a workspace"
+    - title: "Step 5: Quick overview"
      url: /suite-step-5.html
      output: web

+    - title: "Step 6: Pull and process data"
+      url: /suite-step-6.html
+      output: web
+
 #    - title: Run your first backtest
 #      url: /suite-run-your-first-backtest.html
 #      output: web
@@ -0,0 +1,77 @@
+<!--------------------------------------------- TITLE AND DEFINITION starts -->
+
+{% assign title = "XXXXXXXXXXXXXXXX" %}
+{% assign definition = site.data.concepts.XXXXXXXXXXXXXXXX %}
+{% assign preposition = "XXXXXXXXXXXXXXXX" %}
+{% assign plural = "s" %}
+
+<!--------------------------------------------- TITLE AND DEFINITION ends -->
+
+{% if include.more == "yes" and include.heading == "more" %}
+<details class="detailsCollapsible"><summary class="nobr">Click to learn more about {{ title | downcase }}{{plural}}
+</summary>
+{% endif %}
+
+{% if include.heading != "" and include.heading != "more" %}
+{{include.heading}} {{title}}
+{% endif %}
+
+{% if include.icon != "no" %} 
+
+{% if include.table == "yes" and include.icon != "no" %}
+<table class="definitionTable"><tr><td>
+{% endif %}
+
+<img src='images/icons/{{include.icon}}{{ title | downcase | replace: " ", "-" }}.png' />
+
+{% if include.table == "yes" and include.icon != "no" %}
+</td><td>
+{% endif %}
+
+{% endif %}
+
+{% if include.definition == "bold" %}
+<strong>{{ definition }}</strong>
+{% else %}
+{% if include.definition != "no" %}
+{{ definition }}
+{% endif %}
+{% endif %}
+
+{% if include.table == "yes" and include.icon != "no" %}
+</td></tr></table>
+{% endif %}
+
+{% if include.more == "yes" and include.content == "more" and include.heading != "more" %}
+<details class="detailsCollapsible"><summary class="nobr">Click to learn more about {{ title | downcase }}{{plural}}
+</summary>
+{% endif %}
+
+{% if include.content != "no" %}
+
+<!--------------------------------------------- CONTENT starts -->
+
+XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
+
+<!--------------------------------------------- CONTENT ends -->
+
+{% endif %}
+
+{% if include.more == "yes" and include.extended == "more" and include.content != "more" and include.heading != "more" %}
+<details class="detailsCollapsible"><summary class="nobr">Click to learn more about {{ title | downcase }}{{plural}}
+</summary>
+{% endif %}
+
+{% if include.extended != "no" %}
+
+<!--------------------------------------------- EXTENDED starts -->
+
+XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
+
+<!--------------------------------------------- EXTENDED ends -->
+
+{% endif %}
+
+{% if include.more == "yes" %}
+</details>
+{% endif %}
@@ -90,7 +90,14 @@ Select *Configure Sensor Bot Instance* on the menu to access the configuration.
  }
 ```

-* ```startDate``` is the starting date of the data product the sensor bot instance builds. In other words, the sensor bot instance queries its data source for data starting on the configured ```startDate```. In cases when the sensor bot has built a data product starting on a certain date and the ```startDate``` is eventually changed for an earlier one, that is, a date further in the past, then the sensor bot discards the existing data product and starts over from the newly configured date.
+* ```startDate``` is the *desired starting date* of the data product the sensor bot instance builds, in the ```YYYY-MM-DD``` format. The sensor bot instance queries its data source for data starting on the configured ```startDate```.
+
+  * The actual date in which the dataset starts depends on external factors: A. The market may start at a later date. B. The exchange may limit how far in the past data may be retrieved. In both cases, the sensor bot automatically discovers the date closest to the *desired starting date* that is possible to start with, and proceeds accordingly.
+
+  * In the case the ```startDate``` is changed after the sensor bot has started building a data product, either for an earlier or later date, the sensor re-evaluates the feasibility of starting at the new date. The actual date may or may not change; regardless, the sensor bot discards the existing data product and starts over from the newly discovered date. In other words, if the ```startDate``` is changed, the sensor bot starts over.
+ 
+  * Notice that the above starts a chain reaction among all indicator bots that have a data dependency with the sensor bot's output dataset. Also, if the actual date ends up changing, all indicators that determine the starting date of the market by looking at the date discovered by the sensor bot have to discard their existing data products and start over from the new date.
+

 <!--------------------------------------------- CONFIGURING ends -->

@@ -1,6 +1,6 @@
 ---
 title:  "Step 0: Get ready"
-summary: Before you begin, please install Chrome. If you are on Mac OS or Linux, install Node JS too.
+summary: Before you begin, please install Chrome and Node JS .
 sidebar: suite_sidebar
 permalink: suite-step-0.html
 toc: false
@@ -8,13 +8,15 @@ toc: false

 This *Getting Started Guide* will quickly show you the very basics of Superalgos so that you may rapidly grasp the system's potential. The guide will take you through a sequence of instructions to download and install the system, run it for the first time, fetch some data from your preferred exchanges, browse markets on the charts, run a first backtest with an existing strategy, and even try a live trading session, for demonstration purposes.

-But, before you begin, it is highly reccommended that you <a href="https://www.google.com/chrome/" rel="nofollow" rel="noopener" target="_blank">install Chrome</a> and set it up as your default browser.
+## Chrome is the Only Tested Browser
+
+Before you begin, it is highly reccommended that you <a href="https://www.google.com/chrome/" rel="nofollow" rel="noopener" target="_blank">install Chrome</a> and set it up as your default browser.

 {% include note.html content="Use Chrome so that you have a similar environment as the dev team in case you need help. We are not testing on any other browsers, and it is a well-known fact that browsers behave differently." %}

-## If You Are NOT on Windows
+## Node JS is Required

-If you are running Mac OS, Linux or other operating systems, you will need to install Node JS, an open-source server environment required to run Superalgos. 
+Please install Node JS, an open-source server environment required to run Superalgos. 

 **1. Go to the <a href="https://nodejs.org/en/download/" rel="nofollow" rel="noopener" target="_blank">Node JS downloads page</a>**.

@@ -6,6 +6,8 @@ permalink: suite-step-1.html
 toc: false
 ---

+## Start Here
+
 **1. Go to the <a href="https://github.com/Superalgos/Superalgos" rel="nofollow" rel="noopener" target="_blank">Superalgos repository</a>** within the Superalgos organization in GitHub.

 **2. Click the green *Clone or download* button** on the right-hand side of the page. If you are a developer, feel free to clone the repository, otherwise, click the Download ZIP button.
@@ -6,6 +6,8 @@ permalink: suite-step-2.html
 toc: false
 ---

+## Start Here
+
 **1. Go to the ```Superalgos-master``` folder** you extracted from the ZIP file on the prior step.

 **2. On Windows systems, double click on ```Superalgos.exe```** and skip points 3 and 4.
@@ -1,25 +1,19 @@
 ---
-title:  Quick Overview
-summary: The charting space on top features the charts, and the design space at the bottom features the control room of the system.
+title:  Import a Workspace
+summary: Importing a workspace is a simple drag and drop operation.
 sidebar: suite_sidebar
 permalink: suite-step-4.html
+toc: false
 ---

-The first time you run the system, you see a screen split in half. The top half features the <a data-toggle="tooltip" data-original-title="{{site.data.charting_space.charting_space}}">charting space</a> and the bottom half features the <a data-toggle="tooltip" data-original-title="{{site.data.concepts.design_space}}">design space</a>.
+The <a data-toggle="tooltip" data-original-title="{{site.data.concepts.workspace}}">workspace</a> is an important concept within Superalgos. It is the entity that holds all the information handled by the system. In practical terms, it is a file that can be backed up, shared, and restored in the system.

-Use the control in the center of the turquoise bar to pull the slider up and down to make more room for either space.
+For your convenience, the system ships with a default workspace containing a ready-to-use set-up.

-You may also use the keyboard as follows:
+## Start Here

-1. <kbd>Ctrl or &#8984;</kbd> + <kbd>Shift</kbd> + <kbd>&#8593;</kbd> to close the charting space and open the design space.
-1. <kbd>Ctrl or &#8984;</kbd> + <kbd>Shift</kbd> + <kbd>&#8595;</kbd> to close the design space and open the charting space.
-1. <kbd>Ctrl or &#8984;</kbd> + <kbd>Shift</kbd> + <kbd>&#8594;</kbd> to incrementally lower the slider.
-1. <kbd>Ctrl or &#8984;</kbd> + <kbd>Shift</kbd> + <kbd>&#8592;</kbd> to incrementally raise the slider.
+**1. Locate the file ```Share - My Workspace.json```** in the root of the ```Superalgos-master``` folder.

-{% include tip.html content="Pulling the slider to the top of the screen causes the charting space to stop consuming resources and gives an ample view of the design space." %}
+**2. Left-click on the file and drag it over the browser** where the system is running. You will notice the system recognizes your intent and pulls de slider up, showing the <a data-toggle="tooltip" data-original-title="{{site.data.concepts.design_space}}">design space</a>.

-**The design space** is the place where you create. Think of the design space as the control room of the system. This is where you will control every aspect of your data, your trading strategies, your infrastructure, and the overall behavior of the system.
-
-{% include tip.html content="Pulling the slider to the bottom of the screen causes the design space to stop consuming resources and offers a full-screen view of the charting space." %}
-
-**The charting space** shows the charts, which feature market data, indicators, and&mdash;eventually&mdash;the actions of trading strategies. This is the section of the system in which you will monitor markets and analyze the progress and results of your strategies.
+**3. Drop the workspace file on the design space** and wait for a few seconds. You should see a ring around the workspace icon, indicating the progress of the import action. Once the ring dissappears, the import operation is over an you are free to start using the system.
@@ -1,16 +1,27 @@
 ---
-title:  Import a workspace
-summary: Importing a workspace is a simple drag and drop operation.
+title:  Quick Overview
+summary: The charting space on top features the charts, and the design space at the bottom features the control room of the system.
 sidebar: suite_sidebar
 permalink: suite-step-5.html
 ---

-The <a data-toggle="tooltip" data-original-title="{{site.data.concepts.design_space}}">workspace</a> is a substantial concept within Superalgos. It is the entity that holds all the information handled by the system. In practical terms, it is a file that can be backed up, shared, and restored in the system.
+The first time you run the system, you see a screen split in half. The top half features the <a data-toggle="tooltip" data-original-title="{{site.data.charting_space.charting_space}}">charting space</a> and the bottom half features the <a data-toggle="tooltip" data-original-title="{{site.data.concepts.design_space}}">design space</a>.

-For your convenience, the system ships with a default workspace containing a ready-to-use set-up.
+Use the control in the center of the turquoise bar to pull the slider up and down to make more room for either space.

-**1. Locate the file ```Share - Default Workspace.json```** in the root of the ```Superalgos-master``` folder.
+You may also use the keyboard as follows:

-**2. Left-click on the file and drag it over the browser** where the system is running. You will notice the system recognizes your intent and pulls de slider up, showing the designer.
+1. <kbd>Ctrl or &#8984;</kbd> + <kbd>Shift</kbd> + <kbd>&#8593;</kbd> to close the charting space and open the design space.
+1. <kbd>Ctrl or &#8984;</kbd> + <kbd>Shift</kbd> + <kbd>&#8595;</kbd> to close the design space and open the charting space.
+1. <kbd>Ctrl or &#8984;</kbd> + <kbd>Shift</kbd> + <kbd>&#8594;</kbd> to incrementally lower the slider.
+1. <kbd>Ctrl or &#8984;</kbd> + <kbd>Shift</kbd> + <kbd>&#8592;</kbd> to incrementally raise the slider.

-**3. Drop the workspace file on the designer** and wait for a few seconds. You should see a ring around the workspace icon, indicating the progress of the import action. Once the ring dissappears, the import operation is over an you are free to start using the system.
+{% include important.html content="In the begining, the charting space shows empty charts. This is because you still haven't pulled any data from exchanges. The next steps will take care of that!" %}
+
+{% include tip.html content="Pulling the slider to the top of the screen causes the charting space to stop consuming resources and gives an ample view of the design space." %}
+
+**The design space** is the place where you create. Think of the design space as the control room of the system. This is where you will control every aspect of your data, your trading strategies, your infrastructure, and the overall behavior of the system.
+
+{% include tip.html content="Pulling the slider to the bottom of the screen causes the design space to stop consuming resources and offers a full-screen view of the charting space." %}
+
+**The charting space** shows the charts, which feature market data, indicators, and&mdash;eventually&mdash;the actions of trading strategies. This is the section of the system in which you will monitor markets and analyze the progress and results of your strategies.
@@ -0,0 +1,67 @@
+---
+title:  Pull Data from Exchanges and Process Indicators
+summary: To pull data from exchanges, start the corresponding sensor bot tasks. To process indicators, start the associated indicator bot tasks.
+sidebar: suite_sidebar
+permalink: suite-step-6.html
+toc: false
+---
+
+Superalgos puts you in control of the market data you will work with. To do that, the system uses a <a data-toggle="tooltip" data-original-title="{{site.data.concepts.sensor_bot}}">sensor bot</a> to pull raw trades data from exchanges and store it in your machine. As the data starts flowing in, the system uses <a data-toggle="tooltip" data-original-title="{{site.data.concepts.indicator_bot}}">indicator bots</a> to add value to the raw data by producing more ellaborate <a data-toggle="tooltip" data-original-title="{{site.data.concepts.dataset}}">datasets</a> in the form of candles, and all sorts of indicators.
+
+The sensor bot fetches data starting from a configurable date. Once it reaches the present time, it continues connecting to the exchange once per minute, pulling data in a live stream. Indicators process data in a similar fashion, and keep all your data up to date.
+
+Depending on how you intend to use Superalgos, you may want to pull data from as far back as exchanges are willing to give. Such may be the case if you wish to backtest strategies.
+
+On the other hand, if your interest lies in monitoring markets starting from the present time, then you most likely do not need historical data stretching that far in the past.
+
+{% include tip.html content="It is up to you to decide how much historic data you wish to pull, and which exchanges you wish to work with. Downloading a month's worth of market data from high transaction volume exchanges such as Bitmex or Binance may take 2 to 3 hours. The same period worth of data in low transaction volume exchanges such as Bitfinex may take 10 to 20 minutes. What follows are the instructions to configure a date and start pulling data from you preferred exchanges. You may always go back and do things differently." %}
+
+{% include important.html content="Bear in mind that Binance requires an API key to retrieve data. If your exchange of choice is Binance, then go to the exchange, log in to your account, and generate an API key. Have the public key and secret at hand before you start." %}
+
+## Start Here
+
+**1. Open the design space** by pulling the slider to the top of the screen.
+
+**2. Go to the <a data-toggle="tooltip" data-original-title="{{site.data.network.network}}">network hierarchy</a>**. If you are on Windows, the <kbd>Ctrl</kbd> + <kbd>Alt</kbd> + <kbd>N</kbd> key combination should take you there. Otherwise, imitate the actions on the below video until you find it.
+
+[IMAGE]
+
+**3. Hover the mouse over the network <a data-toggle="tooltip" data-original-title="{{site.data.concepts.node}}">node</a>**. This action expands the <a data-toggle="tooltip" data-original-title="{{site.data.concepts.hierarchy}}">hierarchy</a>.and click the {% include inline_image.html file="icons/12-expand.png" %} button on the menu**.
+
+**4. Follow the <a data-toggle="tooltip" data-original-title="{{site.data.concepts.structure_of_nodes}}">structure of nodes</a> until you find the <a data-toggle="tooltip" data-original-title="{{site.data.network.data_mining}}">data mining</a> node**. This is where <a data-toggle="tooltip" data-original-title="{{site.data.network.sensor_bot_instance}}">sensor bot instances</a> and <a data-toggle="tooltip" data-original-title="{{site.data.network.indicator_bot_instance}}">indicator bot instances</a> live and function. 
+
+Notice how the hierarchy continues with several <a data-toggle="tooltip" data-original-title="{{site.data.network.task_manager}}">task managers</a>. 
+
+Each task manager has a label that starts with the words <a data-toggle="tooltip" data-original-title="{{site.data.concepts.masters_data_mine}}">Masters</a> or <a data-toggle="tooltip" data-original-title="{{site.data.concepts.sparta_data_mine}}">Sparta</a>. Those are two <a data-toggle="tooltip" data-original-title="{{site.data.data_mine.data_mine}}">data mines<a/> shipping with the system. The label continues with the name of the corresponding <a data-toggle="tooltip" data-original-title="{{site.data.crypto_ecosystem.crypto_exchange}}">exchange</a> and <a data-toggle="tooltip" data-original-title="{{site.data.crypto_ecosystem.market}}">market</a>.
+
+**5. Expand the task manager of your preferred exchange under the *Masters* label**. To do that, hover the mouse pointer over the select task manager and click the {% include inline_image.html file="icons/12-expand.png" %} button.
+
+Notice how the hierarchy continues with several <a data-toggle="tooltip" data-original-title="{{site.data.network.task}}">tasks</a>. There is one task per bot. The first one in a clock-wise direction corresponds to the sensor bot.
+
+**6. Select *Configure Bot instance* on the sensor bot menu**. A configuration bubble pops-up with the bot's configuration. 
+
+**7. Enter the date you would like your datasets to start on**. Once the date has been edited, remove the mouse pointer from the bubble so that it closes and saves the configuration. To learn more about the implications of your choice for a starting date, click the link below.
+
+{% include /network/sensor-bot-instance.md heading="" icon="no" adding="" configuring="######" starting="" content="no" definition="no" table="no" more="yes"%}
+
+**8. If you chose to pull data from Binance, this is the time to configure your <a data-toggle="tooltip" data-original-title="{{site.data.crypto_ecosystem.exchange_key}}">exchange key</a>**. Click the link below to learn how.
+
+<details class="detailsCollapsible"><summary class="nobr">Click to learn how to configure an exchange key
+</summary>
+
+**1. Go to the <a data-toggle="tooltip" data-original-title="{{site.data.crypto_ecosystem.crypto_ecosystem}}">crypto ecosystem hierarchy</a>** and expand it. If you are on Windows, the <kbd>Ctrl</kbd> + <kbd>Alt</kbd> + <kbd>E</kbd> key combination should take you there. Otherwise, imitate the actions on the below video until you find it.
+
+[IMAGE]
+
+**2. Look for the exchange of choice** around the <a data-toggle="tooltip" data-original-title="{{site.data.crypto_ecosystem.crypto_exchanges}}">crypto exchanges</a> node and expand it too.
+
+**3. Find the <a data-toggle="tooltip" data-original-title="{{site.data.crypto_ecosystem.user_keys}}">user keys</a> node** below <a data-toggle="tooltip" data-original-title="{{site.data.crypto_ecosystem.exchange_accounts}}">exchange accounts</a> &#8594; <a data-toggle="tooltip" data-original-title="{{site.data.crypto_ecosystem.user_account}}">user account</a>. 
+
+**4. Configure the <a data-toggle="tooltip" data-original-title="{{site.data.crypto_ecosystem.exchange_account_key}}">exchange account key</a>**. 
+
+{% include /crypto_ecosystem/exchange-account-key.md heading="" icon="no" adding="" configuring="######" starting="" content="no" definition="no" table="no" more="no"%}
+
+</details>
+
+**9. Go back to the task manager and click *Run All Tasks* on the menu**. This actions puts the sensor bot to run along with all Masters indicators.
+