Data Collection

The Basics

In the Python API, Data Collection lets you collect a snapshot of data over a specified period of time from an Automation1 controller. Use Data Collection to gather a set of time-series data in a real-time deterministic way. You can collect only one set of data at a time on an Automation1 controller. The pieces of information that you collect are known as data signals. They are categorized into axis, task, and system categories. See The Three Controller Categories: Axis, Task, and System. All data signals return lists of floats in Python, which are double-precision, floating-point numbers. Some data signals represent enumerations or masks.

Data Collection is different from Status because status is not real time and it is not time-series data. Status also has variable latency when you retrieve it. You can get status at any time. But you can collect only one set of data at a time. See the Status page.

How to Use

In the Python API, Data Collection is part of the Controller.runtime property. For all the APIs that operate under this property, the Automation1 controller must be running before you can collect data. For more information about this property, see the Controller page.

To collect data by using the Python API, you must:

• Configure the data signals that you want to collect.
• Specify the axes or tasks from which you want to collect the data signals.
• Specify how many points of data you want to collect and the frequency at which to collect them.

First, create a new DataCollectionConfiguration object and specify the number of points to collect and the frequency at which to collect them. The number of points to collect and the frequency of data collection control the quantity of time that is necessary for data collection to complete.

The DataCollectionConfiguration class represents a configuration of data signals to collect from an Automation1 controller. It is one of the few classes in the Python API that you must construct. Refer to the example that follows.

data_collection_configuration = a1.DataCollectionConfiguration(number_of_points_to_collect, frequency)

The maximum number of points that you can collect is controlled by the DataCollectionPoints and DataCollectionItems controller parameters.

The typical data collection frequency is 1 kHz, which means there is 1 millisecond of time between each collected data point. You can collect data at frequencies that are faster than 1 kHz. They include 5 kHz, 10 kHz, 20 kHz, 100 kHz, and 200 kHz. But if you use a faster frequency, it will decrease the number of signals that you can collect.

After you create a configuration object, you can add the data signals that you want to collect. All the data signals are categorized into axis, task, and system categories:

• Axis data signals require you to specify the axis from which to collect the data signal. You can specify an axis by its axis name or axis index.
• Task data signals require you to specify the task from which to collect the data signal. You can specify a task by its task name or task index.
• The Automation1 controller is the only system. The system property does not require you to specify a system from which to collect the data signal.

Add the data signals that you want to collect to the configuration. To do this, use the axis, task, system, and industrial_ethernet properties with the add method.

data_collection_configuration.axis.add(axis_data_signal, axis)
data_collection_configuration.task.add(task_data_signal, task) 
data_collection_configuration.system.add(system_data_signal)
data_collection_configuration.industrial_ethernet.add(industrial_ethernet_mapping_name)

After you add the data signals, start data collection. You can collect one snapshot of data or collect data continuously:

• A snapshot of data - Collects the number of points that you specified in the configuration, blocking until the data collection completes or is stopped. Then the collected data is returned to you. Refer to the example that follows.
Copy

results = controller.runtime.data_collection.collect_snapshot(data_collection_configuration)

• Continuous data collection - Does not collect a set number of points. Instead, it will collect data indefinitely and will periodically give you results through a function when new data is available. For more information and examples, refer to the Continuous Data Collection section.

The DataCollectionResults object that is returned represents the time-series data that is collected from an Automation1 controller based on the specified DataCollectionConfiguration class.

To get the points for each data signal or Industrial Ethernet mapping, use the axis, task, and system or industrial_ethernet properties. You can use the get method on these properties by doing one of the options that follow:

• Specify the data signals and the corresponding axis or task, if applicable.
• Specify an Industrial Ethernet mapping.

axis_points = results.axis.get(axis_data_signal, axis).points
task_points = results.task.get(task_data_signal, task).points
system_points = results.system.get(system_data_signal).points
industrial_ethernet_points = results.industrial_ethernet.get(industrial_ethernet_mapping_name).points

The list of floats that is returned by the points property contains individual samples of the data signal from over the course of data collection. The time between each sample point was controlled by the frequency that you specified in the configuration.

Example Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# Configure data collection for 1 second worth of data.
data_collection_configuration = a1.DataCollectionConfiguration(1000, a1.DataCollectionFrequency.Frequency1kHz)

# Add the data signals that you want to collect.
data_collection_configuration.axis.add(a1.AxisDataSignal.PositionCommand, "X")
data_collection_configuration.task.add(a1.TaskDataSignal.TaskState, "Task 1")
data_collection_configuration.system.add(a1.SystemDataSignal.Timer)

# Start data collection, which will block until all the data is collected, or data collection is stopped.
data_collection_results = controller.runtime.data_collection.collect_snapshot(data_collection_configuration)

# Get the points for each collected data signal. We will get back 1000 points in each array, which
# represents 1 second worth of data collected at 1 kHz or 1 millisecond between each point.
position_command_axis_x = data_collection_results.axis.get(a1.AxisDataSignal.PositionCommand, "X").points
task_state_task_1 = data_collection_results.task.get(a1.TaskDataSignal.TaskState, "Task 1").points
system_timer = data_collection_results.system.get(a1.SystemDataSignal.Timer).points

Continuous Data Collection

Instead of collecting one snapshot of data, you can collect data continuously. Continuous data collection does not collect a set number of points. It collects data indefinitely and will periodically give you results through a function when new data is available.

The collect_continuous method will block until you explicitly stop data collection. You can stop data collection from a different thread or from within the handle_new_results function. Refer to the example that follows.

controller.runtime.data_collection.collect_continuous(data_collection_configuration, handle_new_results)

The number of points to collect that you specified in the DataCollectionConfiguration object controls how often the handle_new_results function is invoked. This function will be invoked on the same thread that called the method. When you specify the number of points to collect, you must balance the quantity of time necessary to process previous results with the speed at which new results are returned. Thus make sure that you have a sufficient quantity of time between each set of results so that you can always process the previous results.

If you specify a number of points to collect that is too small, your custom application will get results too frequently. Thus, it might not be able to keep up with continuous data collection. As a result, an overflow error will occur and data collection will stop.

When you write your function to process results, make sure that the code runs fast. If it does not, your custom application might not be able to keep up with continuous data collection. As a result, an overflow error will occur and data collection will stop.

Example Code

import automation1 as a1

controller = a1.Controller.connect()
controller.start()

def results_handler(data_collection_results):
    # Get the points from the last 250 milliseconds of continuous data collection. You will get back
    # 250 points in each list because you are collecting points at 1 kHz or 1 millisecond between each point.

    position_command_axis_x = data_collection_results.axis.get(a1.AxisDataSignal.PositionCommand, "X").points
    task_state_task_1 = data_collection_results.task.get(a1.TaskDataSignal.TaskState, "Task 1").points
    system_timer = data_collection_results.system.get(a1.SystemDataSignal.Timer).points

    # Here you might append the above points to an existing list of points, or you might update your UI.
    # Remember that this function must be fast, otherwise your custom application might not be able to
    # keep up with continuous data collection. Thus, data collection will stop with an overflow error.

    # In this example, you want continuous data collection because your process can take any quantity of time.
    # Stop continuous data collection after the program on Task 1 completes, which means that your process is done.
    for point in task_state_task_1:
        if point == a1.TaskState.ProgramComplete or point == a1.TaskState.Error:
            controller.runtime.data_collection.stop()

data_collection_configuration = a1.DataCollectionConfiguration(1000, a1.DataCollectionFrequency.Frequency1kHz)

data_collection_configuration.axis.add(a1.AxisDataSignal.PositionCommand, "X")
data_collection_configuration.task.add(a1.TaskDataSignal.TaskState, "Task 1")
data_collection_configuration.system.add(a1.SystemDataSignal.Timer)

controller.runtime.data_collection.collect_continuous(data_collection_configuration, results_handler)

Collect Industrial Ethernet Mappings

In the Python API, you can use Data Collection to collect the values of Industrial Ethernet mappings. To do this, use the DataCollectionConfiguration.industrial_ethernet property.

configuration = DataCollectionConfiguration(1000, DataCollectionFrequency.Frequency1kHz)
configuration.industrial_ethernet.add(“myMapping”); // Configured to collect the value of “$myMapping”.
configuration.industrial_ethernet.add(“myMappingArray”, 2); // Configured to collect the value of “$myMappingArray[2]”.

After you use the DataCollection.collect_snapshot() or DataCollection.collect_continuous() function to collect points, you can use the DataCollectionResults.industrial_ethernet property to access the points that were collected for your configured Industrial Ethernet mappings.

results = controller.runtime.data_collection.collect_snapshot(configuration);

firstResults = results.industrial_ethernet.get(“myMapping”).points;
secondResults = results.industrial_ethernet.get(“myMappingArray”, 2).points;

Thread Safety

The threading library is the only multithreading library that is officially supported by the Automation1 Python API.

The DataCollectionConfiguration class is not thread safe. Only one thread at a time can access an instance of this class. If it is necessary for two or more threads to access or modify an instance of this class, you must use locks to limit the access.

The Controller.runtime.data_collection property is not thread safe. Only one set of data can be collected at a time on an Automation1 controller. Thus, only one thread at a time can access data collection. If it is necessary for two or more threads to access or modify data collection, you must use locks to limit the access.

The DataCollectionResults class is thread safe. You can access an instance of this class and its properties from two or more threads without interference.

The axis, task, system, and industrial_ethernet properties on the DataCollectionResults class are collections of DataCollectionResults objects that can be iterated over.

Full Reference

For more information about the properties and methods that are available for Data Collection, refer to the lists that follow.

Controller.runtime.data_collection Properties

> Controller.runtime.data_collection.status | get

Gets the current status of any data collection (both DataCollectionMode.Snapshot and DataCollectionMode.Continuous enum values).

Controller.runtime.data_collection Methods

> Controller.runtime.data_collection.collect_continuous(data_collection_configuration, handle_new_results)

Runs continuous data collection on the Automation1 controller, which will collect data indefinitely and will periodically give you results through the handle_new_results function. This method does not return anything. Instead, individual chunks of continuously collected data are given to you through the handle_new_results function. The number of points that you get in each chunk is defined by the DataCollectionConfiguration.number_of_points_to_collect property. This property will block until you explicitly stop data collection by calling the Controller.runtime.data_collection.stop() method. You can call the stop method from a different thread or from within the handle_new_results function.

data_collection_configuration: What data to collect and at what period.

handle_new_results: The function that is called when a new chunk of continuously collected data is available.

> Controller.runtime.data_collection.collect_snapshot(data_collection_configuration, handle_progress_changed)

Runs data collection on the Automation1 controller for a single snapshot of data and then returns the results, blocking until all the points are collected or until data collection is stopped. If data collection completes normally, this method returns the DataCollectionConfiguration.number_of_points_to_collect property. If data collection was stopped prematurely, this method returns the points that were collected before it was stopped.

> Controller.runtime.data_collection.start(data_collection_mode, data_collection_configuration)

Starts data collection on the Automation1 controller with the specified mode and uses the specified DataCollectionConfiguration class. This method waits for data collection to start. But it does not wait for data collection to complete. You can stop data collection by calling the Controller.runtime.data_collection.stop() method. You can get the collected data by calling the get_results method. This method gives you the most control. But the collect_snapshot(DataCollectionConfiguration) and collect_continuous methods are easier to use.

data_collection_mode: The data collection mode to start.

data_collection_configuration: What data to collect and at what period.

> Controller.runtime.data_collection.stop()

Stops all data collection on the Automation1 controller (DataCollectionMode.Snapshot and DataCollectionMode.Continuous enum values), regardless of who started it.

> Controller.runtime.data_collection.get_results(data_collection_configuration, number_of_points_to_get)

Returns collected results from data collection. This method can return data from a completed data collection or from data collection that is still occurring. This method waits for data collection to collect the specified number of points, blocking until they are available, and then returns them. This method throws an exception if the specified number of points will never be available (e.g., if data collection was stopped before the requested number of points was collected).

data_collection_configuration: The DataCollectionConfiguration class that was used to start data collection.

number_of_points_to_get: The number of points to get and return.

Returns: A DataCollectionResults object, which contains the collected data up to the number_of_points_to_get.