SCAN WITH ROLLUP

Purpose
For every input record, Scan with Rollup sends an output record to its out port that
includes a running, cumulative summary for the input group that input record belongs to.
In addition, after reading all input records for a particular input group, Scan with Rollup
sends a summary record to its rollup port for that input group. For example, suppose
transaction records are keyed on the stores in which they occur. Each record sent to the
out port might include the year-to-date transaction total for the store in which the
transaction occurred. Each record sent to the rollup port would include the year's total for
transactions at one store, and there would be one record for each store.
The Scan with Rollup component is the Scan component with the addition of a rollup
output port. To use the rollup output port, you must define the rollup_finalize function in
the Scan with Rollup transform. If you do not use the rollup output port, the behavior is
identical to the behavior of the Scan component.
The documentation in this section includes information about Scan with Rollup that is
different from Scan. This section does not duplicate information that is the same for Scan
and Scan with Rollup. If you are not already familiar with Scan, you should read the
documentation for Scan before you try to use Scan with Rollup.
Parameters
See Parameter Descriptions for Scan.
Specifying a Transform for Scan with Rollup
In the file specified in the transform parameter, you can define any of the functions that
you can define for Scan. you can define the following two optional functions:
Transform
Description
Arguments
Return Value
Function
rollup_finalize Scan with Rollup calls this function for
Temporary
Output record that
each input group after reading all input
record,
goes to rollup port
records for that input group. You must
input record
define this function to use the rollup
feature of the Scan with Rollup
component.
rollup_select
If defined, Scan with Rollup calls this
Output record Integer(4)
function to filter the output records
0 is false, non-0 is
generated by rollup_finalize. You must
true
have defined the rollup_finalize
function.
Runtime Behavior of Scan with Rollup
The runtime behavior of the Scan with Rollup component is the same as the runtime
behavior of the Scan component with the following addition. The additional behavior
assumes that the rollup_finalize function has been defined.

Like the runtime behavior of Scan, the runtime behavior of Scan with Rollup varies
according to the value of the sorted-input parameter:
Input must be sorted or grouped
When the key value in the input records changes, Scan with Rollup calls
rollup_finalize for the input group it just finished reading. The rollup_finalize
function generates a summary output record for that group. If you defined the
rollup_select function, Scan with Rollup calls it to determine whether to send the
summary record to the rollup port. If you did not define the rollup_select function,
Scan with Rollup sends the summary record to the rollup port.
In memory: Input need not be sorted
After all input records have been processed, Scan with Rollup calls rollup_finalize
for each input group and generates a summary output record for each group. If you
defined the rollup_select function, Scan with Rollup calls it for each summary output
record to determine whether to send the summary record to the rollup port. Scan with
Rollup then sends the filtered output summary records to the rollup port. If you did
not define the rollup_select function, Scan with Rollup sends all summary output
records to the rollup port.
Using Scan with Rollup in Continuous Flows Applications
In a continuous flows application, the behavior of Scan with Rollup is the same as the
behavior of Scan with the following addition.
Assume that you have defined the rollup_finalize function, and that Scan with Rollup
has just received a checkpoint or computepoint. Again, Scan with Rollup behavior varies
according to the value of the sorted-input parameter:

Input must be sorted or grouped

Scan with Rollup calls rollup_finalize for the input group it was reading when
the checkpoint or computepoint arrived. The rollup_finalize function generates a
summary output record for that group. If you defined the rollup_select function,
Scan with Rollup calls it to determine whether to send the summary record to the
rollup port. If you did not define the rollup_select function, Scan with Rollup
sends the summary record to the rollup port.
Bear in mind that the next input record that Scan with Rollup reads might belong
to the same input group for which Scan with Rollup sent an output record to the
rollup port. If it is, Scan with Rollup treats it as if it is the first input record it has
read from this group. In other words, Scan with Rollup does not maintain state
across computepoints and checkpoints.

In memory: Input need not be sorted

Scan with Rollup calls rollup_finalize for each input group for which it read
records since the last computepoint or checkpoint. Scan with Rollup generates a
summary output record for each of these groups. If you defined the rollup_select
function, Scan with Rollup calls it for each summary output record to determine

whether to send the summary record to the rollup port. Scan with Rollup then
sends the filtered output summary records to the rollup port. If you did not define
the rollup_select function, Scan with Rollup sends all summary output records to
the rollup port.
Bear in mind that the records that Scan with Rollup reads before the next
computepoint or checkpoint might belong to the same groups for which Scan with
Rollup generated summary output records for the rollup port.