Archiver Appliance Enabled Time Plots

Time plots can be augmented with the ability to automatically request archived data from an instance of the EPICS archiver appliance if such an instance is available to the user.

In order to use this functionality, the environment variable PYDM_ARCHIVER_URL must be set to point to the archiver appliance instance. For example:

export PYDM_ARCHIVER_URL=http://lcls-archapp.slac.stanford.edu

These plots can then be created with python code, or using designer.

• Creating in Designer

After setting the above environment variable, archiver time plots will then be able to be created in designer through the usual drag and drop flow. Upon opening designer, there will now be a plotting widget called PyDMArchiverTimePlot available for use. The channel connections can be created in the same way as a regular time plot, no need to preface anything with “archiver://”, the requests to archiver will happen automatically upon running the plot and panning the x-axis to the left, or zooming out.

One property that will be particularly useful to set is the timeSpan. This value (in seconds) will determine how much data to request from archiver upon plot initialization. For example, setting it to 600 will cause the plot to backfill with the last 10 minutes worth of data for the plotted values anytime the plot is opened. If more data is needed after starting up the plot, it can be requested as described below.

• Plot Usage

Upon opening the plot, it will plot the requested amount of archived data. It will then behave like a regular time plot with live data being added to the plot. If more archived data would be useful, requests can be made by either panning the x-axis to the left, or zooming out on the entire plot. This will automatically generate a call to the archiver appliance that will add data to the plot once it is received.

Note that there is a property on the plot called bufferSize which determines how many data points can be displayed on the plot at once. If the request for archived data would return an amount greater than that buffer, it will convert to a request for optimized data which includes the average, min, and max of each data point returned. These will then be plotted as bars to show the full range of data represented by each point. As an example - with a buffer size of 365, a request for a year of data for a PV that updates every second would return roughly 365 points each of which will contain the min and max of that day’s data to plot the full range represented.

PyDMArchiverTimePlot

class pydm.widgets.archiver_time_plot.PyDMArchiverTimePlot(parent: Optional[QObject] = None, init_y_channels: List[str] = [], background: str = 'default', optimized_data_bins: int = 2000)

Bases: PyDMTimePlot

PyDMArchiverTimePlot is a PyDMTimePlot with support for receiving data from the archiver appliance.

Parameters:

• parent (QObject, optional) – The parent of this widget.

• init_y_channels (list) – A list of scalar channels to plot vs time.

• background (str) – The background color for the plot. Accepts any arguments that pyqtgraph.mkColor will accept.

• optimized_data_bins (int) – The number of bins of data returned from the archiver when using optimized requests

addFormulaChannel(yAxisName: str, **kwargs) → FormulaCurveItem

Creates a FormulaCurveItem and links it to the given y axis

addYChannel(y_channel=None, plot_style=None, name=None, color=None, lineStyle=None, lineWidth=None, symbol=None, symbolSize=None, barWidth=None, upperThreshold=None, lowerThreshold=None, thresholdColor=None, yAxisName=None, useArchiveData=False, liveData=True) → ArchivePlotCurveItem

Overrides timeplot addYChannel method to be able to pass the liveData flag.

archive_data_received()

Take any action needed when this plot receives new data from archiver appliance

clearCurves() → None

Clear all curves from the plot

createCurveItem(*args, **kwargs) → ArchivePlotCurveItem

Create and return a curve item to be plotted

curves

Dump and return the current list of curves and each curve’s settings into a list of JSON-formatted strings.

getArchiveBufferSize() → int

Returns the size of the data buffer used to store archived data

getCurves() → List[str]

Dump and return the current list of curves and each curve’s settings into a list of JSON-formatted strings.

requestDataFromArchiver(min_x: Optional[float] = None, max_x: Optional[float] = None) → None

Make the request to the archiver appliance data plugin for archived data.

Parameters:

• min_x (float, optional) – Timestamp representing the start of the time period to fetch archive data from. Defaults to the minimum value visible on the plot when omitted.

• max_x (float, optional) – Timestamp representing the end of the time period to fetch archive data from. Defaults to the timestamp of the oldest live data point in the buffer if available. If no live points are recorded yet, then defaults to the timestamp at which the plot was first rendered.

setAutoScroll(enable: bool = False, timespan: float = 60, padding: float = 0.1, refresh_rate: int = 5000)

Enable/Disable autoscrolling along the x-axis. This will (un)pause the autoscrolling QTimer, which calls the auto_scroll slot when time is up.

Parameters:

• enable (bool, optional) – Whether or not to start the autoscroll QTimer, by default False

• timespan (float, optional) – The timespan to set for autoscrolling along the x-axis in seconds, by default 60

• padding (float, optional) – The size of the empty space between the data and the sides of the plot, by default 0.1

• refresh_rate (int, optional) – How often the scroll should occur in milliseconds, by default 5000

setCurves(new_list: List[str]) → None

Add a list of curves into the graph.

Parameters:

new_list (list) – A list of JSON-formatted strings, each contains a curve and its settings

setTimeSpan(value)

Set the value of the plot’s timespan

updateXAxis(update_immediately: bool = False) → None

Manages the requests to archiver appliance. When the user pans or zooms the x axis to the left, a request will be made for backfill data

ArchivePlotCurveItem

class pydm.widgets.archiver_time_plot.ArchivePlotCurveItem(channel_address: Optional[str] = None, use_archive_data: bool = True, liveData: bool = True, **kws)

Bases: TimePlotCurveItem

ArchivePlotCurveItem is a TimePlotCurveItem with support for receiving data from the archiver appliance.

Parameters:

• channel_address (str) – The address to of the scalar data to plot. Will also be used to retrieve data from archiver appliance if requested.

• use_archive_data (bool) – If True, requests will be made to archiver appliance for archived data when the plot is zoomed or scrolled to the left.

• liveData (bool) – If True, the curve will gather data in real time.

• **kws (dict[str: any]) – Additional parameters supported by pyqtgraph.PlotDataItem.

archiveConnectionStateChanged(connected: bool) → None

Capture the archive channel connection status and emit changes

Parameters:

connected (bool) – The new connection status of the archive channel

channels() → List[PyDMChannel]

Return the list of channels this curve is connected to

getArchiveBufferSize() → int

Return the length of the archive buffer

initializeArchiveBuffer() → None

Initialize the archive data buffer used for this curve.

insert_archive_data(data: ndarray) → None

Inserts data directly into the archive buffer.

An example use case would be zooming into optimized mean-value data and replacing it with the raw data.

Parameters:

data (np.ndarray) – A numpy array of shape (2, length_of_data). Index 0 contains timestamps and index 1 contains the data observations.

max_archiver_x()

Provide the the most recent timestamp from the archiver data buffer. This is useful for scaling the x-axis.

Returns:

The timestamp of the most recent data point in the archiver data buffer.

Return type:

float

min_archiver_x()

Provide the the oldest valid timestamp from the archiver data buffer.

Returns:

The timestamp of the oldest data point in the archiver data buffer.

Return type:

float

receiveArchiveData(data: ndarray) → None

Receive data from archiver appliance and place it into the archive data buffer. Will overwrite any previously existing data at the indices written to.

Parameters:

data (np.ndarray) – A numpy array of varying shape consisting of archived data for display. At a minimum, index 0 will contain the timestamps and index 1 the actual data observations. Additional indices may be used as well based on the type of request made to the archiver appliance. For example optimized data will include standard deviations, minimums, and maximums

redrawCurve(min_x=None, max_x=None) → None

Redraw the curve with any new data added since the last draw call.

resetArchiveBufferSize() → None

Reset the length of the archive buffer back to the default and zero it out

setArchiveBufferSize(value: int) → None

Set the length of the archive data buffer and zero it out

to_dict() → OrderedDict

Returns an OrderedDict representation with values for all properties needed to recreate this curve.

FormulaCurveItem

class pydm.widgets.archiver_time_plot.FormulaCurveItem(formula: Optional[str] = None, pvs: Optional[dict] = None, use_archive_data: Optional[bool] = True, liveData: Optional[bool] = True, color: Optional[str] = 'green', plot_style: str = 'Line', **kws)

Bases: BasePlotCurveItem

FormulaCurveItem is a BasePlotCurve that takes in a formula of curves and evaluates to graph a function.

To use, instead of typing in a PV channel, this takes in the prefix ‘f://’ to indicate a function, then uses curly braces ‘{<PV row header>}’ to find which curves to use as inputs. Other than that, FormulaCurveItems have the capacity to handle basic arithmetic functions and also special functions like log() and trigonometry.

Finally, when populating its data buffers, it uses the union of the timesteps for each of its input curves, and uses last seen data to fill in the gaps when calculating.

Parameters:

• formula (str) – The formula that we are graphing

• use_archive_data (bool) – If True, requests will be made to archiver appliance for archived data when the plot is zoomed or scrolled to the left.

• pvs (dict[str: BasePlotCurveItem]) – Has all the information for our FormulaCurveItem to evaluate the value at every timestep

• **kws (dict[str: any]) – Additional parameters supported by pyqtgraph.PlotDataItem.

arch_conn_change(status: bool) → None

Capture the archive channel connection status of a given curve and check connection status

Parameters:

status (bool) – Archive connection status for a given curve

checkFormula() → bool

Make sure that our formula is still valid. Namely, all of the input curves need to still exist in the viewer

compute_evaluation(formula: str, pvData: dict, pvValues: dict, pvIndices: dict, archive: bool) → ndarray

This is where the actual computation takes place. We are going to go through the data step by step and calculate our formula at each timestamp available

Parameters:

• formula (str) – The formula to compute

• pvData (dict) – A dictionary containing all of the Archive or Live data for each curve

• pvValues (dict) – The value of each curve at the current timestep. At the start of this function, each is set to their respective last seen values when the time is equal to the latest start time of all of the curves.

• pvIndices (dict) – A dictionary storing where in each curve’s data buffer we are currently at while calculating

• archive (bool) – Whether or not this is computing for the Archive or for Live

connection_status_check()

Check the connection status of all live and archive curves. Save and emit any changes.

createTrueFormula() → str

Convert our human-readable formula to something easier to use for the computer, in the background only

evaluate() → None

Use our formula and input curves to calculate our value at each timestep. If one curve updates at a certain timestep and another does not, it uses the previously seen data of the second curve, and assumes it is accurate at the current timestep.

getArchiveBufferSize() → int

Return the length of the archive buffer

initializeArchiveBuffer() → None

Initialize the archive data buffer used for this curve.

live_conn_change(status: bool) → None

Capture the live channel connection status of a given curve and check connection status

Parameters:

status (bool) – Live connection status for a given curve

max_archiver_x()

Provide the the most recent timestamp from the archiver data buffer. This is useful for scaling the x-axis.

Returns:

The timestamp of the most recent data point in the archiver data buffer.

Return type:

float

min_archiver_x()

Provide the the oldest valid timestamp from the archiver data buffer.

Returns:

The timestamp of the oldest data point in the archiver data buffer.

Return type:

float

redrawCurve(min_x=None, max_x=None) → None

Redraw the curve with any new data added since the last draw call.

resetArchiveBufferSize() → None

Reset the length of the archive buffer back to the default and zero it out

setArchiveBufferSize(value: int) → None

Set the length of the archive data buffer and zero it out

set_up_eval(archive: bool) → dict

Because we are doing very similar evaluations for Archive and Live Data, we are going to set up our data structures such that we can compute our evaluation more easily. This function will (generally) be called twice, once with archive = True, once with False

Parameters:

archive (bool) – Whether this is setting up for Archive Data or Live Data

to_dict() → OrderedDict

Returns an OrderedDict representation with values for all properties needed to recreate this curve.