Superalgos/docs/pages/suite/suite-process-definition.md

title, summary, sidebar, permalink

title	summary	sidebar	permalink
Process Definition		suite_sidebar	suite-process-definition.html

Introduction

Process Output	Process Dependencies	Status Report	Execution Started Event	Execution Finished Event

The two fundamental nodes of a process that need to be defined are the process output and the process dependencies. We will explore these two in detail.

You know from earlier explanations what a status report is. Status reports are handled internally by the bot and there isn't much more you need to know about them at this point. We will get back to status reports later on.

The execution started event and the execution finished event are too going to be described in detail.

Process Output

Output Dataset

The process output defines which datasets are impacted by the process. This definition is represented by the output dataset nodes under process output referencing the dataset definition of each of Paula's product.

For the sake of clarity, let's expand on the above statement.

We stated earlier that Paula offeres four different products. Each of this products stores two datasets, one in the market file type of structure and one in the daily file type. These products are created by two processes: one of them creates the market file for each of the products, and the other one creates the daily files.

The image above shows that relationship.

What we can see in the image above is the definition of the process output of the Multi-Period-Market process. Notice how the process output node has four offspring nodes named output dataset. One of the output dataset nodes is referencing the Market dataset definition of the Bollinger Channels product definition.

The rest of the output dataset nodes are linked to the corresponding Market dataset definition of the rest of Paula's products, which are out of the boundaries of the image.

Similarly, the output dataset nodes of the Multi-Period-Daily process are linked to the Daily dataset definition of each product. Feel free to go and check for yourself.

NOTE: Notice how this link is represented by a faint grey dotted line, instead of an orange or yellow dotted line. As you already know, yellow (or orange) lines denote a hierarchical relationship between nodes. On the other hand, a grey dotted line represents a reference].

The ultimate effect of establishing a reference from the output dataset to a product dataset definition is that—upon execution—every time the process finishes a processing cycle, it triggers an event that may be consumed by other entities. These events indicate that the corresponding datasets (the ones impacted by the process as per the process output definition) have been updated.

An example of other entities that may be listening to such events is that of plotters. Plotters read datasets and create graphical representations of this data over the charts. You surely have noticed that the charts are constantly updating the information in form of candles and indicators in realtime, syncronized with the data being extracted from the exchange by Charly, the sensor bot. That kind of automatism is possible thanks to the events processes trigger every time an execution cycle is finished, signaling to everyone listening that new data is available on each of the impacted datasets.

Process Dependencies

Status Dependency

The definition of status dependencies tell us which other processes the process depends on.

There are three kinds of status dependencies:

• Self Reference is mandatory, as a process will always need to read it's own status report every time it weaks up.

• Market Starting Point is a status dependency existing on Multi-Period-Daily processes (processes that store datasets in the daily file structure) so that the process knows the datetime of the start of the market. Usually, this will come from Charly's Hitoric-Trades process status report. Multi-Period-Market processes do not have this type of status dependency as the date of the start of the market is implied in their own dataset (a single file with all market data).

• Market Ending Point is a status dependency existing both in Multi-Period-Market and Multi-Period-Daily processes so that the process knows the datetime of the end of the market.

The image above shows Paula's Multi-Period-Market process status dependencies. As explained above, both the Self Reference and Market Ending Point are present.

Paula builds different types of Bollinger Channels based on the Bollinger Bands created by Chris. That is why the Market Ending Point status dependency links to Chris' Multi-Period-Market status report. Paula needs to know up to what date Chris has updated its datasets so that she knows up to which date to process her own.

Bear in mind that all of these bots need to be autonomous and resilient. Figure that Chris may be manually stopped, it's data may be erased, or something else may happen that makes Chris fall behind. In such cases, if Paula finds that her data is up to date with Chris' or maybe even further ahead in datetime, she will not run until Chris catches up.

This means Paula will preserve her dataset so that other's may relie on it and use it independently of the fate of Chris' dataset.

Data Dependency

The definition of data dependencies tell us which datasets the processes uses as input.

The concept is quite straightforward. Paula depends on Chris' datasets as input to build her own.

In the image below, you will see Paula's Multi-Period-Market process data dependecy linking to Chris' Multi-Period-Market dataset definition of the Bollinger Bands product.

Execution Started Event & Execution Finished Event

The execution started event is the event that triggers the execution of a process. The execution finished event is the event that processes trigger once they have finished an execution cylce.

You may think as the execution finished event as an event that is broadcasted to whoever wants to listen. It is the action of a process letting the world know it has finished its execution cycle.

This comes very handy for processes that need to wait for other processes to finish in order to start their own execution.

We have already established that Paula has a data dependency with Chris' datasets and that she reads Chris' status reports to know up to which datetime Chris has updated his datasets.

The one missing piece of information Paula requires is when she should run. By linking Paula's processes execution started event with Chris' processes execution finished events Paula knows exactly when to start her processes, right after Chris' have finished.

That link is what you can see in the below image.