filterBaselineTable

Filter the baseline table by batches relative to the most recent batch

Filter the baseline table by specified batches relative to the most recent batch. Compare with filterBaselineTableByTime, which filters batches by time intervals.

This operator is not a generic operator that works on any type of a data. It uses specific data (table) that is generated by the baseline playbook.

Operator Usage in Easy Mode

  1. Click + on the parent node.
  2. Enter Filter Baseline Table operator in the search field and select the operator from the Results to open the operator form.
  3. In the Select Baseline drop-down, enter or select the baseline to pull data from.
  • Optional fields
    • Start Time: Enter an offset time or a specific start time to pull the data from. Offset time allows
      you to set time in seconds, minutes, hours, days, or weeks; Specific time allows you to select the
      date and time.
    • End Time: Enter an offset time or a specific end time to pull the data from. Offset time allows you
      to set time in seconds, minutes, hours, days, or weeks; Specific time allows you to select the date
      and time.
  1. Click Run to view the result.
  2. Click Save to add the operator to the playbook.
  3. Click Cancel to discard the operator form.

Usage Details

filterBaselineTable(baselineTable, n) -- read last n batches
filterBaselineTable(baselineTabel, a, b) -- read batches that fall between a, b. Both a and b batches inclusive

Example

Input
'table': github_logs

id batchId
12017-10-10 18:00:00
22017-10-10 19:00:00
32017-10-10 20:00:00
-- read last 2 batches (-1) is last batch and inclusive
filterBaselineTable(baselineTable, -2, -1)
 
-- read last 2 batches
filterBaselineTable(baselineTable, -2)
 
-- read 2 batches from the end of a stream except the last one
filterBaselineTable(baselineTable, -3, -2)

Output
Table containing union of all batches that were read.


© Devo Technology Inc. All Rights Reserved.