Skip to main content
All CollectionsNeuron AppResources
Using formula expressions
Using formula expressions

Resources

Alvaro Sánchez avatar
Written by Alvaro Sánchez
Updated over a year ago

Formula expressions are a key feature when defining models. With them, you can define the mathematical functions to transform and aggregate your raw industrial data to gain insights about your operation.

You can apply formula expressions on transforms and metrics to define the value and format you want Neuron to show the results in.


Formula expression elements


Variables

Variables represent asset properties in formula expressions. Use variables to input values from other asset properties in your expressions, so that you can process data from constant properties (attributes), raw data streams (measurements), and other formula properties.

Variables can represent asset properties from the same asset model or from associated child asset models. Only metric formulas can input variables from child asset models. Variable names are case sensitive.


Literals

You can define number and string literals in formula expressions.

  • Numbers: use numbers and scientific notation to define integers and doubles. You can use E notation to express numbers with scientific notation. Examples: 1, 2.0, .9, -23.1, 7.89e3, 3.4E-5

  • Strings: Use the ' (quote) and " (double quote) characters to define strings. The quote type for the start and end must match. To escape a quote that matches the one that you use to declare a string, include that quote character twice. This is the only escape character in strings. Examples: 'active' , "inactive" , '{"temp": 52}' , "{""temp"": ""high""}" .


Operators

You can use the following common operators in formula expressions:

Operator

Description

+

If both operands are numbers, this operator adds the left and right operands.

If either operand is a string, this operator concatenates the left and right operands as strings.

For example:

The expression 1 + 2 + " is three" evaluates to "3 is three" .

The concatenated string can have up to 1024 characters. If the string exceeds 1024 characters, then there is no data point for that computation.

-

Subtracts the right operand from the left operand.


You can only use this operator with numeric operands.

/

Divides the left operand by the right operand.

You can only use this operator with numeric operands.

*

Multiplies the left and right operands.

You can only use this operator with numeric operands.

^

Raises the left operand to the power of the right operand (exponentiation).

You can only use this operator with numeric operands.

%

Returns the remainder from dividing the left operand by the right operand.

The result has the same sign as the left operand. This behavior differs from the modulo operation.

You can only use this operator with numeric operands.

[]

s[index]

Returns the character at an index index of the string s .

Examples:

  • "Hello!"[1] returns e .

  • "Hello!"[-2] returns o .

[]

s[start:end:step]

Returns a slice of the string s .

This operator has the following arguments:

  • start - (optional): The inclusive start index of the slice. Defaults to 0 .

  • end - (optional): The exclusive end index of the slice. Defaults to the length of the string.

  • step - (optional): The number to increment for each step in the slice. For example, you can specify 2 to return a slice with every other character, or specify -1 to reverse the slice. Defaults to 1 .

You can omit the step argument to use its default value. For example, s[1:4:1] is equivalent to s[1:4] .

The arguments must be integers or the none constant.

If you specify none , it uses the default value for that argument.

Examples:

  • "Hello!"[1:4] returns "ell".

  • "Hello!"[:2] returns "He".

  • "Hello!"[3:] returns "lo!".

  • "Hello!"[:-4] returns "He".

  • "Hello!"[::2] returns "Hlo".

  • "Hello!"[::-1] returns "!olleH".


Constants

You can use the following common mathematical constants in your expressions. All constants are case insensitive.

⛔ If you define a variable with the same name as a constant, the variable overrides the constant.

Constant

Description

pi

The number pi (π): 3.141592653589793.

e

The number e: 2.718281828459045.

true

Equivalent to the number 1.

Booleans convert to their number equivalents.

false

Equivalent to the number 0.

Booleans convert to their number equivalents.

none

Equivalent to no value. You can use this constant to output nothing as the result of a conditional expression.


Functions

You can use the following functions to operate on data in your formula expressions.

Transforms and metrics support different functions. The following table indicates which types of functions are compatible with each type of formula property.

Function type

Transforms

Metrics

Common functions

Yes

Yes

Comparison functions

Yes

Yes

Conditional functions

Yes

Yes

String functions

Yes

Yes

Aggregation functions

No

Yes

Temporal functions

No

Yes

Date and time functions

Yes

Yes


Function syntax

You can use the following syntax to create functions:

  • Regular syntax: With the regular syntax, the function name is followed by parentheses with zero or more arguments. function_name(argument1, argument2, argument3, ...) . For example, functions with the regular syntax might look like log(x) and contains(s, substring) .

  • Uniform function call syntax (UFCS): UFCS enables you to call functions using the syntax for method calls in object-oriented programming. With UFCS, the first argument is followed by dot (.), then the function name and the remaining arguments (if any) inside parentheses. argument1.function_name(argument2, argument3, ...) . For example, functions with UFCS might look like x.log() y s.contains(substring) . You can also use UFCS to chain subsequent functions. Neuron uses the evaluation result of the current function as the first argument for the next function. For example, you can use message.jp('$.status').lower().contains('fail') instead of contains(lower(jp(message, '$.status')),'fail') .


Common functions

In transforms and metrics, you can use the following functions to calculate common mathematical functions in transforms and metrics:

Function

Description

abs(x)

Returns the absolute value of x.

acos(x)

Returns the arccosine of x.

asin(x)

Returns the arcsine of x.

atan(x)

Returns the arctangent of x.

cbrt(x)

Returns the cubic root of x.

ceil(x)

Returns the nearest integer greater than x.

cos(x)

Returns the cosine of x.

cosh(x)

Returns the hyperbolic cosine of x.

cot(x)

Returns the cotangent of x.

exp(x)

Returns e to the power of x.

expm1(x)

Returns exp(x) - 1. Use this function to more accurately calculate exp(x) - 1 for small values of x.

floor(x)

Returns the nearest integer less than x.

log(x)

Returns the loge (base e) of x.

log10(x)

Returns the log10 (base 10) of x.

log1p(x)

Returns log(1 + x). Use this function to more accurately calculate log(1 + x) for small values of x.

log2(x)

Returns the log2 (base 2) of x.

pow(x, y)

Returns x to the power of y. This is equivalent to x ^ y.

signum(x)

Returns the sign of x (-1 for negative inputs, 0 for zero inputs, +1 for positive inputs).

sin(x)

Returns the sine of x.

sinh(x)

Returns the hyperbolic sine of x.

sqrt(x)

Returns the square root of x.

tan(x)

Returns the tangent of x.

tanh(x)

Returns the hyperbolic tangent of x.


Comparison functions

In transforms and metrics, you can use the following comparison functions to compare two values and output 1 (true) or 0 (false).

Neuron compares strings by lexicographic order.

Function

Description

gt(x, y)

Returns 1 if x is greater than y, otherwise 0 (x > y).

This function doesn't return a value if x and y are incompatible types, such as a number and a string.

gte(x, y)

Returns 1 if x is greater than or equal to y, otherwise 0 (x ≥ y).

Neuron considers the arguments equal if they are within a relative tolerance of 1E-9. This behaves similar to the isclose function in Python.

This function doesn't return a value if x and y are incompatible types, such as a number and a string.

eq(x, y)

Returns 1 if x is equal to y, otherwise 0 (x == y).

Neuron considers the arguments equal if they are within a relative tolerance of 1E-9. This behaves similar to the isclose function in Python.

lt(x, y)

Returns 1 if x is less than y, otherwise 0 (x < y).

This function doesn't return a value if x and y are incompatible types, such as a number and a string.

lte(x, y)

Returns 1 if x is less than or equal to y, otherwise 0 (x ≤ y).

Neuron considers the arguments equal if they are within a relative tolerance of 1E-9. This behaves similar to the isclose function in Python.

This function doesn't return a value if x and y are incompatible types, such as a number and a string.

isnan(x)

Returns 1 if x is equal to NaN, otherwise 0.

This function doesn't return a value if x is a string.


Conditional functions

In transforms and metrics, you can use the following function to check a condition and return different results whether the condition evaluates to true or false:

Function

Description

if(condition, result_if_true, result_if_false)

Evaluates the condition and returns result_if_true if the condition evaluates to true or result_if_false if the condition evaluates to false. condition must be a number.

This function considers 0 as false and everything else (including NaN) as true. Booleans convert to 0 (false) and 1 (true).

You can return the none constant from this function to discard the output for a particular condition. This means you can filter out data points that don't meet a condition. For more information, see Filtering data points.

Examples:

  • if(0, x, y) returns the variable y.

  • if(5, x, y) returns the variable x.

  • if(gt(temp, 300), x, y) returns the variable x if the variable temp is greater than 300.

  • if(gt(temp, 300), temp, none) returns the variable temp if it's greater than or equal to 300 or none (no value) if temp is less than 300.

We recommend that you use UFCS for nested conditional functions where one or more arguments are conditional functions. You can use if(condition, result_if_true) to evaluate a condition and elif(condition, result_if_true, result_if_false) to evaluate additional conditions.

For example, you can use if(condition1, result1_if_true).elif(condition2, result2_if_true, result2_if_false) instead of if(condition1, result1_if_true, if(condition2, result2_if_true, result2_if_false)).

Important:

You must use elif(condition, result_if_true, result_if_false) with UFCS.


String functions

In transforms and metrics, you can use the following functions to operate on strings:

⚠️ Formula expressions can only output double values. Nested expressions can output other data types, such as strings, but the formula as a whole must evaluate to a number.

You can use the jp function to convert a string to a number. If you define a formula that computes a non-numeric value, Neuron doesn't output a data point for that computation.

Function

Description

len(s)

Returns the length of the string s.

find(s, substring)

Returns the index of the string substring in the string s.

contains(s, substring)

Returns 1 if the string s contains the string substring, otherwise 0.

upper(s)

Returns the string s in uppercase form.

lower(s)

Returns the string s in lowercase form.

jp(s, json_path)

Evaluates the string s with the JsonPath expression json_path and returns the result.

Use this function to do the following:

  • Extract a value, array, or object from a serialized JSON structure.

  • Convert a string to a number. For example, the formula jp('111', '$') returns 111 as a number.

To extract a string value from a JSON structure and return it as a number, you must use multiple nested jp functions. The outer jp function extracts the string from the JSON structure, and the inner jp function converts the string to a number.

The string json_path must contain a string literal. This means that json_path can't be an expression that evaluates to a string.

Examples:

  • jp('{"status":"active","value":15}', '$.value') returns 15.

  • jp('{"measurement":{"reading":25,"confidence":0.95}}', '$.measurement.reading') returns 25.

  • jp('[2,8,23]', '$[2]') returns 23.

  • jp('{"values":[3,6,7]}', '$.values[1]') returns 6.

  • jp('111', '$') returns 111.

  • jp(jp('{"measurement":{"reading":25,"confidence":"0.95"}}', '$.measurement.confidence'), '$') returns 0.95.

For more information, see Undefined, infinite, and overflow values.


Aggregation functions

In metrics only, you can use the following functions that aggregate input values over each time interval and calculate a single output value. Aggregation functions can aggregate data from associated assets.

Aggregation function arguments can be variables, number literals, temporal functions, or aggregation functions. This means that you can't provide nested expressions as arguments to aggregation functions. For example, the formula avg(x + 1) isn't valid. By contrast, the formula max(latest(x), latest(y), latest(z)) is valid and returns the largest current value of the x, y, and z properties.

⛔ Neuron also automatically computes aggregates over certain time intervals for all properties.

Function

Description

avg(x0, ..., xn)

Returns the mean of the given variables' values over the current time interval.

This function outputs a data point only if the given variables have at least one data point over the current time interval.

sum(x0, ..., xn)

Returns the sum of the given variables' values over the current time interval.

This function outputs a data point only if the given variables have at least one data point over the current time interval.

min(x0, ..., xn)

Returns the minimum of the given variables' values over the current time interval.

This function outputs a data point only if the given variables have at least one data point over the current time interval.

max(x0, ..., xn)

Returns the maximum of the given variables' values over the current time interval.

This function outputs a data point only if the given variables have at least one data point over the current time interval.

count(x0, ..., xn)

Returns the total number of data points for the given variables over the current time interval. This function computes a data point for every time interval.


Temporal functions

In metrics only, you can use the following functions that return values based on timestamps of data points.

Temporal function arguments must be properties from the local asset model. This means that you can't use properties from child asset models in temporal functions. You also can't use expressions as arguments to temporal functions.

Function

Description

first(x)

Returns the given variable's value with the earliest timestamp over the current time interval.

last(x)

Returns the given variable's value with the latest timestamp over the current time interval.

earliest(x)

Returns the given variable's value with the latest timestamp before the current time interval.

This function computes a data point for every time interval, if the input property has at least one data point in its history.

latest(x)

Returns the given variable's value with the latest timestamp before the end of the current time interval.

This function computes a data point for every time interval, if the input property has at least one data point in its history.

statetime(x)

Returns the amount of time in seconds that the given variables are positive over the current time interval. You can use the comparison functions to create a transform property for the statetime function to consume.

For example, if you have an Idle property that is 0 or 1, you can calculate idle time per time interval with this expression: IdleTime = statetime(Idle).

This function doesn't support metric properties as input variables. This function computes a data point for every time interval, if the input property has at least one data point in its history.

The following diagram shows how Neuron computes the temporal functions first, last, earliest, and latest, relative to the current time interval.

AWS IoT SiteWiseLas funciones temporales de  devuelven puntos de datos en función de su marca temporal.

Example statetime scenario:

Consider an example where you have an asset with the following properties:

  • Idle – A measurement that is 0 or 1. When the value is 1, the machine is idle.

  • Idle Time – A metric that uses the formula statetime(Idle) to calculate the amount of time in seconds where the machine is idle, per 1 minute interval.

The Idle property has the following data points.

Timestamp

2:00:00 PM

2:00:30 PM

2:01:15 PM

2:02:45 PM

2:04:00 PM

Idle

0

1

1

0

0

Neuron calculates the Idle Time property every minute from the values of Idle. After this calculation completes, the Idle Time property has the following data points.

Timestamp

2:00:00 PM

2:01:00 PM

2:02:00 PM

2:03:00 PM

2:04:00 PM

Idle Time

N/D

30

60

45

0

Neuron performs the following calculations for Idle Time at the end of each minute.

  • At 2:00 PM (for 1:59 PM to 2:00 PM)

  • At 2:01 PM (for 2:00 PM to 2:01 PM)

  • At 2:02 PM (for 2:01 PM to 2:02 PM)

  • At 2:03 PM (for 2:02 PM to 2:03 PM)

  • At 2:04 PM (for 2:03 PM to 2:04 PM)


Date and time functions

In transforms and metrics, you can use the date and time functions in the following ways:

  • Retrieve the current timestamp of a data point in UTC or in the local time zone.

  • Construct timestamps with arguments, such as year, month, and day_of_month.

  • Extract a time period such as a year or month with the unix_time argument.

Function

Description

now()

Returns the current date and time, in seconds, in the Unix epoch format.

timestamp()

In transforms, the function returns the timestamp of the input message, in seconds, in the Unix epoch format.

In metrics, the function returns the timestamp retrieved at the end of the current window, in seconds, in the Unix epoch format.

mktime(time_zone, year, month, day_of_month, hour, minute, second)

Returns the input time in seconds, in the Unix epoch format.

The following requirements apply, when you use the function:

  • The time zone argument must be a quoted string ('UTC'). If not specified, the default time zone is UTC.

  • The year, month, day of month, hour, minute, and second arguments must be in order.

  • The year, month, and date arguments are required.

The following limits apply, when you use the function:

  • year: Valid values are between 1970 and 2250.

  • month: Valid values are between 1 and 12.

  • day-of-month: Valid values are between 1 - 31.

  • hour: Valid values are between 0 and 23.

  • minute: Valid values are between 0 and 59.

  • second: Valid values are between 0 and 59.

Examples:

  • mktime(2020, 2, 29)

  • mktime('UTC+3', 2021, 12, 31, 22)

  • mktime(2022, 10, 13, 2, 55, 13.68, 'PST')

localtime(unix_time, time_zone)

Returns the year, the day of the month, the day of the week, the day of the year, the hour, the minute, or the second in the specified time zone from the unix time.

The following requirements apply, when you use the function:

  • The time zone argument must be a quoted string ('UTC'). If not specified, the default time zone is UTC.

  • The unix time argument is the time in seconds, in the Unix epoch format. The valid range is between 1-31556889864403199. It can be a floating point number.

Example response: 2007-12-03T10:15:30+01:00[Europe/Paris]

localtime(unix_time, time_zone) isn't a standalone function. The year(), mon(), mday, wday(), yday(), hour(), munite(), and sec() functions take localtime(unix_time, time_zone) as an argument.

Examples:

  • year(localtime('GMT', 1605898608.8113723))

  • now().localtime().year()

  • timestamp().localtime(‘PST’).year()

  • localtime(1605289736, ‘Europe/London’).year()

year(localtime(localtime(unix_time, time_zone))

Returns the year fromlocaltime(unix_time, time_zone).

mon(localtime(unix_time, time_zone))

Returns the month from localtime(unix_time, time_zone).

mday(localtime(unix_time, time_zone))

Returns the day of the month from localtime(unix_time, time_zone).

wday(localtime(unix_time, time_zone))

Returns the day of the week from localtime(unix_time, time_zone).

yday(localtime(unix_time, time_zone))

Returns the day of the year from localtime(unix_time, time_zone).

hour(localtime(unix_time, time_zone))

Returns the hour from localtime(unix_time, time_zone).

minute(localtime(unix_time, time_zone))

Returns the minute from localtime(unix_time, time_zone).

sec(localtime(unix_time, time_zone))

Returns the second from localtime(unix_time, time_zone).


Supported time zone formats

You can specify the time zone argument in the following ways:

  • Time zone offset - Specify 'Z' for UTC or an offset ('+2' or '-5').

  • Offset IDs - Combine a time zone abbreviation and an offset. For example, 'GMT+2' and 'UTC-01:00'.

  • Region based IDs - For example, 'Etc/GMT+12' and 'Pacific/Pago_Pago'.

The date and time functions support three-letter time zone abbreviations. Check this list of supported time zone abbreviations and region based IDs.


Using strings in formulas

You can operate on strings in your formula expressions. You also can input strings from variables that reference attribute and measurement properties.

⚠️ Formula expressions can only output double values. Nested expressions can output other data types, such as strings, but the formula as a whole must evaluate to a number.

You can use the jp function to convert a string to a number. If you define a formula that computes a non-numeric value, Neuron doesn't output a data point for that computation.

Neuron provides the following formula expression features that you can use to operate on strings:

  • String literals

  • The index operator (s[index])

  • The slice operator (s[start:end:step])

  • Comparison functions, which you can use to compare strings by lexicographic order

  • String functions, which includes the jp function that can parse serialized JSON objects and convert strings to numbers


Filtering data points

You can use the if function to filter out data points that don't meet a condition. The if function evaluates a condition and returns different values for true and false results. You can use the none constant as an output for one case of an if function to discard the data point for that case.

To filter out data points that match a condition, create a transform that uses the if function to define a condition that checks if a condition is met, and returns none as either the result_if_true or result_if_false value.

Example of filtering out data points where water isn't boiling:

Consider a scenario where you have a measurement, temp_c, that provides the temperature (in Celsius) of water in a machine. You can define the following transform to filter out data points where the water isn't boiling:

Transform: boiling_temps = if(gte(temp_c, 100), temp_c, none) – Returns the temperature if it's greater than or equal to 100 degrees Celsius, otherwise returns no data point.


Counting data points that match a condition

You can use comparison functions and sum() to count the number of data points for which a condition is true.

To count data points that match a condition:

  1. Create a transform that uses a comparison function to define a filter condition on another property.

  2. Create a metric that sums the data points where that condition is met.

Example of counting the number of data points where water is boiling:

Consider a scenario where you have a measurement, temp_c, that provides the temperature (in Celsius) of water in a machine. You can define the following transform and metric properties to count the number of data points where the water is boiling:

  • Transform: is_boiling = gte(temp_c, 100) – Returns 1 if the temperature is greater than or equal to 100 degrees Celsius, otherwise returns 0.

  • Metric: boiling_count = sum(is_boiling) – Returns the number of data points where water is boiling.


Late data in formulas

Neuron supports late data ingestion of data that is up to 7 days old. When Neuron receives late data, it recalculates existing values for any metric that inputs the late data in a past window. These recalculations result in data processing charges.

⛔ When Neuron computes properties that input late data, it uses each property's current formula expression. After Neuron recalculates a past window for a metric, it replaces the previous value for that window. If you enabled notifications for that metric, Neuron also emits a property value notification.

This means that you can receive a new property value update notification for the same property and timestamp for which you previously received a notification. If your applications or data lakes consume property value notifications, you must update the previous value with the new value so that their data is accurate.


Data quality in formulas

In Neuron each data point has a quality code, which can be one of the following:

  • GOOD: The data isn't affected by any issues.

  • BAD: The data is affected by an issue such as sensor failure.

  • UNCERTAIN: The data is affected by an issue such as sensor inaccuracy.

Neuron consumes only GOOD quality data when it computes transforms and metrics. Neuron outputs only GOOD quality data for successful computations. If a computation is unsuccessful, then Neuron doesn't output a data point for that computation. This can occur if a computation results in an undefined, infinite, or overflow value.


Undefined, infinite, and overflow values

Some formula expressions (such as x / 0, sqrt(-1), or log(0)) calculate values that are undefined in a real number system, infinite, or outside the range supported by Neuron. When an asset property's expression computes an undefined, infinite, or overflow value, Neuron doesn't output a data point for that computation.

Neuron also doesn't output a data point if it computes a non-numeric value as the result of a formula expression. This means that if you define a formula that computes a string, array, or the none constant, then Neuron doesn't output a data point for that computation.

Examples:

Cada una de las siguientes expresiones de fórmula da como resultado un valor que Neuron no puede representar como un número. Neuron no genera un punto de datos cuando calcula estas expresiones de fórmula.

Each of the following formula expressions results in a value that Neuron can't represent as a number. Neuron doesn't output a data point when it computes these formula expressions.

  • x / 0 is undefined.

  • log(0) is undefined.

  • sqrt(-1) is undefined in a real number system.

  • "hello" + " world" is a string.

  • jp('{"values":[3,6,7]}', '$.values') is an array.

  • if(gte(temp, 300), temp, none) is none when temp is less than 300.


If you're having trouble using formula expressions on asset models or you're interested in learning more details, get in touch with our technical support team by sending an email to neuron@plexo.cloud.

Did this answer your question?