Modules

Message

In order to effectively communicate between different parts of the application, all of the components must know what to expect when they get information. To enable this, messages should be created with the Message class.

This provides a consistent layout for handling messsages throughout the application. It is set up to be minimal and extendable. There are also various helper functions to automatically fill in some of these fields in common scenarios.

sender and receiver are a list of strings, which becomes useful when complex message routing is required.

command is simply required to ensure there is a straightforward way to identify different message types when handling messages.

package actually carries the data and may be of any type. From experience, dictionaries work well. Adding labels to the data does not add much overhead, but does make the message much easier to parse, allowing for changes in input order, new fields etc.

class pysimpleapp.message.Message(sender: Tuple[str], command: str, package: any)

Generic message class which will be used to move information around the system

The package can be of any type, dictionaries tend to be useful for decoding on the other end.

Parameters:

• sender – Address of the object which sent the message
• command – Headline of the message to allow simple handling
• package – Data transferred as part of the message

__init__(sender: Tuple[str], command: str, package: any)

Initialize self. See help(type(self)) for accurate signature.

Threads

class pysimpleapp.threads.simple_threads.SimpleThread(name: str, supervisor=None)

*SimpleThread* is an abstract class which will provide the basis for building out the threads provided in pysimpleapp.

Methods which must be specified by children:

• create_params - creates the parameters which should be used during main execution
• main - the body of the thread which performs the functionality
• _control_loop - describes how the class executes the main function

It executes its main function as part of a control loop which is specified by child classes. Before it starts, the parameters can be updated and it can even be ended and prevented from ever running.

Must be initialised with a name, owner, input and output queues.

The operation of the thread is implemented in main() This function should be overridden during implementation.

Parameters are created in create_params() which should be overridden when implemented. Parameters should not be created from anywhere else. Updates to the parameter will be collected after each run of the main function.

Messages are handled based on a lookup table called the address_book. There are some in-built commands which can be easily accessed, but the addresses can be changed easily. To add custom commands to the thread, simply create a function in the class which takes a pysimpleapp.message.Message object and give it a key in the address book.

Example

self.address_book["my_custom_command"] = self.my_custom_command_handler

# During message handling, this will be called as:
self.address_book["my_custom_command"](messsage)

__init__(name: str, supervisor=None)

Create the thread, set up control flags and call _create_params

Parameters:

• name – Identifier for the thread, could be list of strings depending on communication model
• owner – Address of object which created the thread (often a ThreadManager)
• message_queue – Input pipe for messages which the thread is expected to process

class Endpoints

An enumeration.

custom_handler(message: pysimpleapp.message.Message)

Override this function to handle custom user messages. Expect a pysimpleapp.Message object as an input, no return value is expected or handled by default

Good idea: Include a command as a key in a package dictionary to reliably handle custom messages. This will be much clearer than sorting by type, size or looking for a specific key or list value.

end()

Put THREAD_END message in queue

main()

Override this function to provide the thread with instructions.

main provides the functionality for the thread. It may contain heavy computations, IO processing and anything you want to do without having to worry about exactly how long it takes.

Good ideas:

• Read from parameters to decide on program execution
• Send pysimpleapp.Message to endpoints where they will be sent to subscribers
• Handle exceptions to prevent unnecessary failures of the thread

Bad ideas:

• Writing to parameters during program execution
• Writing things back to the input_queue
• Implementing operations which take a long time to complete and expecting fast reactions in changes to parameters or THREAD_STOP commands

publish(package, endpoint=<Endpoints.RESULT: 'result'>, command=<Commands.THREAD_HANDLE: 'THREAD_HANDLE'>)

Publishes a package to all the subscribers of an endpoint

By default, will publish to the RESULT endpoint with a command of THREAD_HANDLE. You may wish to provide a different endpoint or command depending on how the application is set up.

raise_exception(exception: Exception)

Notify the supervisor that an exception occurred during thread running

Passes same information as what is received by threading.excepthook

run()

This defines the programatic flow for the thread.

Threads wait for messages on the input_queue and then process them.

If an end command is given, the thread will return True and exit, therefore no longer being able to run.

If a start command is given and no end command is requested, the _control_loop function will be called. While running, the queue will not be processed.

If the control loop raises an exception, the thread will exit gracefully.

send_to(receiver: List[str], command: str, package: any)

Helper function for sending information from a thread correctly

setup()

Override this function to do whatever setup is necessary for the thread. Example:

self.parameters["param 1"] = 0
self.parameters["param 2"] = "some string"

start()

Put THREAD_START message in queue

stop()

Put THREAD_STOP message in queue

Single Run Thread

class pysimpleapp.threads.simple_threads.SingleRunThread(name: str, supervisor=None)

*SingleRunThread* is a child of *SimpleThread* which runs the main function once and then ends.

It will execute after a start command and then exit.

main and create_params are left as abstract methods for the user to implement.

Multi Run Thread

class pysimpleapp.threads.simple_threads.MultiRunThread(name: str, supervisor=None)

*MultiRunThread* is a child of *SimpleThread* which may run the main function repeatedly.

It will execute after a start command, and then await further instruction. This may be another start command, in which case the main function will repeat. Or it may be an update or end command, which will cause the parameters to update or the thread to end, respectively.

Other than that, it works exactly the same way as the *SimpleThread*.

main and _create_params are left as abstract methods for the user to implement.

Repeating Thread

class pysimpleapp.threads.simple_threads.RepeatingThread(name: str, interval: datetime.timedelta = datetime.timedelta(seconds=1))

*RepeatingThread* is a child of *SimpleThread* which runs its main function on a regular basis.

It differs from *MultiRunThread* in that once it has been started, it will trigger its own runs. It may be stopped with the “THREAD_STOP” command. Having received such a command, the thread will be waiting to start again and will not end without a THREAD_END command.

RepeatingThread has a parameter of “loop_timer” which specifies the time, in seconds, after program execution to wait before running main again. Default is 1s.

This is achieved by adding a start message to its own input queue after “loop_timer” amount of time. Therefore, it is susceptible to being flooded by other messages. It may also be triggered by sending another start command.

So, to use safely, set parameters before starting and do not flood with parameters while running. Issue a THREAD_STOP command before performing large changes to the parameters.

main and create_params are left as abstract methods for the user to implement.

repeating_start()

Command to set the main function going again

Precise Repeating Thread

ThreadManager

ThreadManager is a special thread which is designed make it easier to control many threads at the same time.

It is a subclass of the MultiRunThread and manages the flow of information to and from running thread instances.

Thread Manager commands:

• SET_THREAD_TYPES - send a dictionary of keys to thread classes which you will wish to instantiate through the Thread Manager
• NEW_THREAD - Create a new instance of a thread given a thread name and thread type
• DESTROY_THREAD - Destroy a running thread and prevent any delayed output from leaking
• ACTIVE_THREADS - Get a dictionary of all the threads which are active in the ThreadManager and what class of thread they are. Returns a message to the sender with ACTIVE_THREADS command

Threads which are running in the ThreadManager can be accessed through the address [“THREAD_MANAGER_NAME”, “THREAD_NAME”] with their usual commands.

Further explanation of the commands and how the ThreadManager works are given below:

The ThreadManager has a special dictionary of thread_types which may be set with the command SET_THREAD_TYPES. This maps human readable strings or identifiers to the more complex class names of thread types. It also means the operating the ThreadManager does not require importing the thread classes themselves locally, as this will have been done when the ThreadManager was setup, all that needs to be known is the identifier.

ThreadManagers are always addressed with a list, whether the message is for the thread manager or for a thread being managed. The ThreadManager checks the first value in the address to ensure it should be processing the message. If it finds its own name, it replaces the receiver attribute of the message with a new tuple with the first value sliced out. If it does not find its own name, the message will be sent away to the output queue with the name of the ThreadManager prepended to the front of the “from” message element. This will allow complex patterns to be easily built up, with layers of ThreadManagers, because threads do not need to know the whole hierarchy, only the name of their direct manager. They can also contact sibling threads through the same mechanism as an external message source, simplifying the ThreadManager code.

The top level ThreadManager (at the top of the hierarchy) will have None as its parent as it was not created by a ThreadManager. Therefore, when it sends a message outside the thread hierarchy, it will not prepend anything to the “from” address and therefore the message address will be a valid place to send a reply.

This constructing and destructing of messages as they travel up and down ensures that when a message is received by a thread, it can send a message back to the same location and be sure that it is a valid location (or was when the message was sent).

To create a new thread with the thread manager, send a NEW_THREAD command directly to the thread manager and provide a dictionary containing values for the thread type. The ThreadManager will look in its list of currently active threads to see if it has a thread with that name. If it does, the request will be rejected. If not, a new thread of that type and with that name will be created, with the owner being the name of the ThreadManager.

From then on, the thread will be addressed through the ThreadManager with an address of [thread_manager_name, thread_name].

Each thread being manager by a ThreadManager has its own internal input queue to receive messsages. This prevents name clashing which would inevitably occur by using a single queue for many threads.

Output queues from the threads are connected to the ThreadManager input queue, to be processed the same way as messages external to the ThreadManager. This allows the ThreadManager to wait for input on one queue only and to process all messages the same way. It also opens up the potential for threads to contact other threads, although this situation should be managed carefully.

Threads which are running happily are kept in the ThreadManager’s active thread list. The ThreadManager will preiodically review this list to see if any of the threads are stopped. Threads which are stopped are removed from the list.

Occasionally, it may be necessary to ask the ThreadManager to kill a thread and prevent its output from being sent. This might be the case if a thread has main function which takes a long time and sends a big message at the end, but is no longer needed. In this case, send a DESTROY_THREAD command directly to the ThreadManager with the name of the thread to be destroyed as the package. In this case, the thread will be moved to a list of threads to be destroyed and the output will be captured by the ThreadManager instead of being sent onwards. A new thread with the same name cannot be created until the thread is neither an active thread nor in the list of destroyed threads. This will require some garbage collection of the ThreadManager. Also a good function to return useful information about which names are currently being used