pmRegisterDerivedMetric(3)
NAME
pmRegisterDerived, pmRegisterDerivedMetric - register a derived metric name and definition
C SYNOPSIS
#include <pcp/pmapi.h> char *pmRegisterDerived(char *name, char *expr); int pmRegisterDerivedMetric(char *name, char *expr, char **errmsg); cc ... -lpcp
DESCRIPTION
Derived metrics provide a way of extending the Performance Metrics Name
Space (PMNS) with new metrics defined at the PCP client-side using
arithmetic expressions over the existing performance metrics.
Typical uses would be to aggregate a number of similar metrics to
provide a higher-level summary metric or to support the ``delta V over
delta V'' class of metrics that are not possible in the base data
semantics of PCP. An example of the latter class would be the average
I/O size, defined as
delta(disk.dev.total_bytes) / delta(disk.dev.total)
where both of the disk.dev metrics are counters, and what is required
is to to sample both metrics, compute the difference between the
current and previous values and then calculate the ratio of these
differences.
The arguments to pmRegisterDerived are the name of the new derived
metric and expr is an arithmetic expression defining how the values of
name should be computed.
pmRegisterDerivedMetric is the exact functional equivalent to
pmRegisterDerived except that it provides a simplified model of error
handling, where a formatted message is returned via the errmsg
parameter.
Syntactic checking is performed at the time pmRegisterDerived is
called, but semantic checking is deferred until each new context is
created with pmNewContext(3) or re-established with
pmReconnectContext(3), at which time the PMNS and metadata is available
to allow semantic checking and the metadata of the derived metrics to
be established. This means pmRegisterDerived does not apply
retrospectively to any open contexts, so the normal use would be to
make all calls to pmRegisterDerived (possibly via
pmLoadDerivedConfig(3)) or pmRegisterDerivedMetric and then call
pmNewContext(3).
name should follow the syntactic rules for the names of performance
metrics, namely one or more components separated with a dot (``.''),
and each component must begin with an alphabetic followed by zero or
more characters drawn from the alphabetics, numerics and underscore
(``_''). For more details, refer to PCPIntro(1) and pmns(5).
name must be unique across all derived metrics and should not match the
name of any regular metric in the PMNS. It is acceptable for name to
share some part of its prefix with an existing subtree of the PMNS,
e.g. the average I/O size metric above could be named disk.dev.avgsz
which would place it amongst the other disk.dev metrics in the PMNS.
Alternatively, derived metrics could populate their own subtree of the
PMNS, e.g. the average I/O size metric above could be named
my.summary.disk.avgsz.
The expression expr follows these syntactic rules:
* Terminal elements are either names of existing metrics or integer
constants. Recursive definitions are not allowed, so only the names
of regular metrics (not other derived metrics) may be used. Integer
constants are constrained to the precision of 32-bit unsigned
integers.
* The usual binary arithmetic operators are supported, namely -
addition (``+''), subtraction (``-''), multiplication (``*'') and
division (``/'') with the normal precedence rules where
multiplication and division have higher precedence than addition and
subtraction, so a+b*c is evaluated as a+(b*c)
* Parenthesis may be used for grouping.
* The following unary functions operate on a single performance metric
and return one or more values. For all functions (except count() and
instant()), the type of the operand metric must be arithmetic
(integer of various sizes and signedness, float or double).
┌───────────┬────────────────────────────────────────────────────┐
│ Function │ Value │
├───────────┼────────────────────────────────────────────────────┤
│avg(x) │ A singular instance being the average value across │
│ │ all instances for the metric x. │
├───────────┼────────────────────────────────────────────────────┤
│count(x) │ A singular instance being the count of the number │
│ │ of instances for the metric x. │
├───────────┼────────────────────────────────────────────────────┤
│delta(x) │ Returns the difference in values for the metric x │
│ │ between one call to pmFetch(3) and the next. There │
│ │ is one value in the result for each instance that │
│ │ appears in both the current and the previous │
│ │ sample. │
├───────────┼────────────────────────────────────────────────────┤
│rate(x) │ Returns the difference in values for the metric x │
│ │ between one call to pmFetch(3) and the next │
│ │ divided by the elapsed time between the calls to │
│ │ pmFetch(3). The semantics of the derived metric │
│ │ are based on the semantics of the operand (x) with │
│ │ the dimension in the time domain decreased by one │
│ │ and scaling if required in the time utilization │
│ │ case where the operand is in units of time, and │
│ │ the derived metric is unitless. This mimics the │
│ │ rate conversion applied to counter metrics by │
│ │ tools such as pmval(1), pmie(1) and pmchart(1). │
│ │ There is one value in the result for each instance │
│ │ that appears in both the current and the previous │
│ │ sample. │
├───────────┼────────────────────────────────────────────────────┤
│instant(x) │ Returns the current value of the metric x, even it │
│ │ has the semantics of a counter, i.e. │
│ │ PM_SEM_COUNTER. The semantics of the derived │
│ │ metric are based on the semantics of the operand │
│ │ (x); if x has semantics PM_SEM_COUNTER, the │
│ │ semantics of instant(x) is PM_SEM_INSTANT, │
│ │ otherwise the semantics of the derived metric is │
│ │ the same as the semantics of the metric x. │
├───────────┼────────────────────────────────────────────────────┤
│max(x) │ A singular instance being the maximum value across │
│ │ all instances for the metric x. │
├───────────┼────────────────────────────────────────────────────┤
│min(x) │ A singular instance being the minimum value across │
│ │ all instances for the metric x. │
├───────────┼────────────────────────────────────────────────────┤
│sum(x) │ A singular instance being the sum of the values │
│ │ across all instances for the metric x. │
└───────────┴────────────────────────────────────────────────────┘
* White space is ignored.
SEMANTIC CHECKS AND RULES
There are a number of conversions required to determine the metadata
for a derived metric and to ensure the semantics of the expressions are
sound.
In a binary expression, if the semantics of both operands is not a
counter (i.e. PM_SEM_INSTANT or PM_SEM_DISCRETE) then the result will
have semantics PM_SEM_INSTANT unless both operands are PM_SEM_DISCRETE
in which case the result is also PM_SEM_DISCRETE.
The mapping of the pmUnits of the metadata uses the following rules:
* If both operands have a dimension of COUNT and the scales are not the
same, use the larger scale and convert the values of the operand with
the smaller scale.
* If both operands have a dimension of TIME and the scales are not the
same, use the larger scale and convert the values of the operand with
the smaller scale.
* If both operands have a dimension of SPACE and the scales are not the
same, use the larger scale and convert the values of the operand with
the smaller scale.
* For addition and subtraction all dimensions for each of the operands
and result are identical.
* For multiplication, the dimensions of the result are the sum of the
dimensions of the operands.
* For division, the dimensions of the result are the difference of the
dimensions of the operands.
Scale conversion involves division if the dimension is positive else
multiplication if the dimension is negative. If scale conversion is
applied to either of the operands, the result is promoted to type
PM_TYPE_DOUBLE.
Putting all of this together in an example, consider the derived metric
defined as follows:
x = network.interface.speed - delta(network.interface.in.bytes) /
delta(sample.milliseconds)
The type, dimension and scale settings would propagate up the
expression tree as follows.
┌────────────────────────┬────────┬───────────────┬─────────────────┐
│ Expression │ Type │ Dimension & │ Scale Factor(s) │
│ │ │ Scale │ │
├────────────────────────┼────────┼───────────────┼─────────────────┤
│sample.milliseconds │ DOUBLE │ millisec │ │
│delta(...) │ DOUBLE │ millisec │ │
│network...bytes │ U64 │ byte │ │
│delta(...) │ U64 │ byte │ │
│delta(...) / delta(...) │ DOUBLE │ byte/millisec │ /1048576 and │
│ │ │ │ *1000 │
│network...speed │ FLOAT │ Mbyte/sec │ │
│x │ DOUBLE │ Mbyte/sec │ │
└────────────────────────┴────────┴───────────────┴─────────────────┘
Because semantic checking cannot be done at the time pmRegisterDerived
is called, errors found during semantic checking are reported using
pmprintf(3). These include:
Error: derived metric <name1>: operand: <name2>: <reason>
There was a problem calling pmLookupName(3) to identify the
operand metric <name2> used in the definition of the derived
metric <name1>.
Error: derived metric <name1>: operand (<name2> [<pmid2>]): <reason>
There was a problem calling pmLookupDesc(3) to identify the
operand metric <name2> with PMID <pmid2> used in the definition
of the derived metric <name1>.
Semantic error: derived metric <name>: <operand> <op> <operand>:
Illegal operator for counters
If both operands have the semantics of counter, only addition or
subtraction make sense, so multiplication and division are not
allowed.
Semantic error: derived metric <name>: <operand> <op> <operand>:
Illegal operator for counter and non-counter
Only multiplication or division are allowed if the left operand
has the semantics of a counter and the right operand is not a
counter.
Semantic error: derived metric <name>: <operand> <op> <operand>:
Illegal operator for non-counter and counter
Only multiplication is allowed if the right operand has the
semantics of a counter and the left operand is not a counter.
Semantic error: derived metric <name>: <operand> <op> <operand>: Non-
arithmetic type for <left-or-right> operand
The binary arithmetic operators are only allowed with operands
with an arithmetic type (integer of various sizes and
signedness, float or double).
Semantic error: derived metric <name>: <function>(<operand>): Non-
arithmetic operand for function
The unary functions are only defined if the operand has
arithmetic type.
Semantic error: derived metric <name>: Incorrect time dimension for
operand
Rate conversion using the rate() function is only possible for
operand metrics with a Time dimension of 0 or 1 (see
pmLookupDesc(3)). If the operand metric's Time dimension is 0,
then the derived metrics has a value "per second" (Time
dimension of -1). If the operand metric's Time dimension is 1,
then the derived metrics has a value of time utilization (Time
dimension of 0).
EXPRESSION EVALUATION
For the binary arithmetic operators, if either operand must be scaled
(e.g. convert bytes to Kbytes) then the result is promoted to
PM_TYPE_DOUBLE. Otherwise the type of the result is determined by the
types of the operands, as per the following table which is evaluated
from top to bottom until a match is found.
┌─────────────────────────┬──────────┬────────────────┐
│ Operand Types │ Operator │ Result Type │
├─────────────────────────┼──────────┼────────────────┤
│either is PM_TYPE_DOUBLE │ any │ PM_TYPE_DOUBLE │
├─────────────────────────┼──────────┼────────────────┤
│any │ division │ PM_TYPE_DOUBLE │
├─────────────────────────┼──────────┼────────────────┤
│either is PM_TYPE_FLOAT │ any │ PM_TYPE_FLOAT │
├─────────────────────────┼──────────┼────────────────┤
│either is PM_TYPE_U64 │ any │ PM_TYPE_U64 │
├─────────────────────────┼──────────┼────────────────┤
│either is PM_TYPE_64 │ any │ PM_TYPE_64 │
├─────────────────────────┼──────────┼────────────────┤
│either is PM_TYPE_U32 │ any │ PM_TYPE_U32 │
├─────────────────────────┼──────────┼────────────────┤
│otherwise (both are │ any │ PM_TYPE_32 │
│PM_TYPE_32) │ │ │
└─────────────────────────┴──────────┴────────────────┘
CAVEATS
Unary negation is not supported, so the following expressions would be syntactically incorrect, -3*abc and -this.number Derived metrics are not available when using pmFetchArchive(3) as this routine does not use a target list of PMIDs that could be remapped (as is done for pmFetch(3)). There is no pmUnregisterDerived method, so once registered a derived metric persists for the life of the application.
DIAGNOSTICS
On success, pmRegisterDerived returns NULL. If a syntactic error is found at the time of registration, the value returned by pmRegisterDerived is a pointer into expr indicating where the error was found. To identify what the error was, the application should call pmDerivedErrStr(3) to retrieve the corresponding parser error message. pmRegisterDerivedMetric returns 0 and errmsg is undefined if the parsing is successful. If the given expr does not conform to the required syntax pmRegisterDerivedMetric returns -1 and a dynamically allocated error message string in errmsg. The error message is terminated with a newline and includes both the input name and expr, along with an indicator of the position at which the error was detected. e.g. Error: pmRegisterDerivedMetric("my.disk.rates", ...) syntax error The position indicator line may be followed by an additional diagnostic line describing the nature of the error, when available. In the case of an error, the caller is responsible for calling free(3) to release the space allocated for errmsg.
SEE ALSO
PCPIntro(1), free(3), PMAPI(3), pmDerivedErrStr(3), pmFetch(3), pmLoadDerivedConfig(3), pmNewContext(3) and pmReconnectContext(3).
Opportunity
Personal Opportunity - Free software gives you access to billions of dollars of software at no cost. Use this software for your business, personal use or to develop a profitable skill. Access to source code provides access to a level of capabilities/information that companies protect though copyrights. Open source is a core component of the Internet and it is available to you. Leverage the billions of dollars in resources and capabilities to build a career, establish a business or change the world. The potential is endless for those who understand the opportunity.
Business Opportunity - Goldman Sachs, IBM and countless large corporations are leveraging open source to reduce costs, develop products and increase their bottom lines. Learn what these companies know about open source and how open source can give you the advantage.
Free Software
Free Software provides computer programs and capabilities at no cost but more importantly, it provides the freedom to run, edit, contribute to, and share the software. The importance of free software is a matter of access, not price. Software at no cost is a benefit but ownership rights to the software and source code is far more significant.
Free Office Software - The Libre Office suite provides top desktop productivity tools for free. This includes, a word processor, spreadsheet, presentation engine, drawing and flowcharting, database and math applications. Libre Office is available for Linux or Windows.
Free Books
The Free Books Library is a collection of thousands of the most popular public domain books in an online readable format. The collection includes great classical literature and more recent works where the U.S. copyright has expired. These books are yours to read and use without restrictions.
Source Code - Want to change a program or know how it works? Open Source provides the source code for its programs so that anyone can use, modify or learn how to write those programs themselves. Visit the GNU source code repositories to download the source.
Education
Study at Harvard, Stanford or MIT - Open edX provides free online courses from Harvard, MIT, Columbia, UC Berkeley and other top Universities. Hundreds of courses for almost all major subjects and course levels. Open edx also offers some paid courses and selected certifications.
Linux Manual Pages - A man or manual page is a form of software documentation found on Linux/Unix operating systems. Topics covered include computer programs (including library and system calls), formal standards and conventions, and even abstract concepts.
