Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In Datameer, you can create a workbook to get to new insights with your data. Inside the workbook, you can add additional data sources, change the column and sheet names, collapse columns, apply formulas, view workbook details, and more.

...

  1. Choose Workbooks from the + (plus) button or right-click in the navigation bar on the left side of the screen and select Create New > Workbook.
  2. From the File menu, select Add Data.
  3. Navigate to the folder where the data sources are located and select the name of the data source you want to use.
  4. Click Add Data.

When using data from a sheet in another workbook or an import job, the data from that source sheet is copied to the new sheet if the new sheet is marked as kept. In previous versions, the source sheet was referenced instead, which led to workbook data not being deleted by housekeeping.

The workbook is populated with a sample of data from that data source. To use data in Datameer, that data must be part of a data source. If you have a source of data you want to use for analysis, such as an Excel spreadsheet, you need to create a data source using that data, or have a system administrator create one for you.

...

Status notifications have been added to the bottom of workbook worksheets to show if the worksheet is dispaying displaying information based on sample or full data and if the worksheet is filtered.

...

If the worksheet is displaying no information, this could be that none of the origial original sample data fits the criteria of the caculationscalculations. Running the workbook takes into account all current worksheet caculations calculations and might bring in new sample data to populate the worksheet if data from the data source meet the caculation calculation criteria.

Full data messages

If the worksheet's caculations calculations are based on the full data, the notification is green. It tells you how many sample rows are being displayed in the worksheet out of how many total records from the data source. If all the records of the source are being used, the status bar displays the total record count.


Opening Existing Workbooks

To open an existing workbook:

  1. On the the File Browser tab tab, find the workbook you want to open.
  2. Highlight the workbook by clicking on it.
  3. Double-click to open or right-click on the highlighted workbook and select select Open.

Opening a workbook using a URL

...

Workbook ID: http(s)://<server>:<port>/workbook/<wbkID>

Example: https://localhost:8080/workbook/25

Workbook path: http(s)://<server>:<port>/workbook?path=<path/to/workbook.wbk>

Example: https://localhost:8080/workbook?path=Users/Matthew/Seasonal_Earnings

Open a workbook to a specific worksheet: http(s)://<server>:<port>/workbook?path=<path/to/workbook.wbk>&sheet=<sheetName>

Example:   https://localhost:8080/workbook?path=/Users/Matthew /Seasonal_Earnings.wbk&sheet=products

Opening a prior version of a workbook

Datameer gives you the option of saving previous versions of a workbook when it is run more than once. Learn more about about optimizing your workbooks and data retention

To open a prior version of a workbook:

  1. On the the File Browser tab tab, find the workbook you want to open.
  2. Highlight the workbook by clicking on it.
  3. Right-click on the highlighted workbook and select select Show Details.
    Image Modified 
  4. Find the previously run workbook iteration in the the Current Data section section. Click the Show Data icon.
    Image Modified 
  5. From the the Full Results page page, you can open the workbook or download the data. 
    For convenience, other previous ran workbook iterations have a link.
    Image Modified 

Anchor
Add_additional
Add_additional
Adding Additional Data Sources

To add additional data sources to an existing workbook:

  1. From the the File menu menu, choose choose Add Data or  or click the the Add Data icon  icon on the toolbar.
  2. Navigate to the folder where the data sources are located and select the name of the data source you want to use.
  3. Click Click Add Data.
  4. Repeat the previous steps for each data source you want to use.

For each data source you select, a tab is added to the workbook, containing a sample of data from that data source. You can add add multiple data sources at the same time by holding the SHIFT button.  

Exchanging Data Sources

 You can exchange data sources to run a workbook on a different set of similar data, which is faster than recreating the workbook.

For example, you can run a workbook on data for each month by exchanging the data source to point to the data for the next month. Each datasource must have the same structure. Further analysis applies only to the new data after the exchange. This option is only available when the current data sheet uses a data source imported by adding data. The option isn't available for sheets created using filters, joins, sorts, external datasheetsdata sheets, or formulas. You must also have write permissions for this workbook to use this option.

To exchange data sources in an existing workbook:

  1. From the the File menu menu, choose choose Exchange Data or  or click the the Exchange Data icon  icon on the toolbar.
  2. Navigate to the folder where the datasources are located and select the name of the data source you want to use.
  3. Click Click Exchange Data.
Note

Errors might occur when exchanging data if column names have changed. On a joined sheet, original references to columns remain even if that column name doesn't exist in the new exchanged data. If this happens, replace the joined sheet reference with the updated column name from the exchanged data.

...

Anchor
sheet_navigate
sheet_navigate
 
Viewing sheets within a workbook

...

  • Click the sheet tab you want to view a the bottom of the page.
  • Select the the Sheets menu  menu and click the sheet you want to view.
  • Click on the the Sheet Dependencies button  button in the toolbar and switch between sheets by clicking on sheets that are dependent with each other.

...

Setting up sheet names 

...

Renaming sheets

To change the name of a sheet:

  1. Right-click the sheet name and click click Rename.
  2. Enter the new name and press Enter.
  3. To undo the change, right-click the name again and click click Undo.

Deleting sheets in a workbook

You can To delete sheets in a workbook that you don't want to use or keep.To delete sheetsa sheet:

  1. Click the tab in the workbook that you want to delete.
  2. Right-click the tab and choose choose Delete.
  3. Click Click Delete.

Duplicating sheets in a workbook

You can duplicate workbook sheets - so you can create variations on a theme. The new sheet is . New sheets are named with the next available number . If there are five sheets and you duplicated Sheet3, the new sheet is named Sheet6appended to the sheet name.

 To duplicate sheets in a workbook:

  1. Click the tab in the workbook.
  2. From the the Workbook menu menu Sheets, select Duplicate Sheet, or click the Duplicate Worksheet icon on the toolbar.
    You can also rightDuplicate  Sheet or right-click the sheet tab and select Duplicate.
  3. Select In the Duplicate Worksheet, select the columns you want to copy to the new sheet and click Create Sheet Copy.

...

  1. click Create  Sheet  Copy.
  2. If a sheet has actions applied to it, you have the option to copy both the logic and the data, or you can select the Copy data onlycheckbox to copy just the data.
Note
titleAs of Datameer v6.3

 In Datameer v6.3, you cannot only copy In addition to the raw data of a sheet with formulas, joins, sorts, or clustering, but you can also copy the underlying logic and actions to the of a sheet into a new sheet. If a sheet in a workbook has actions applied to it, you can copy both the logic and the data or select the Copy data only checkbox to copy over only the data in the sheet without the logic.

...

You can reorder sheet locations inside of a workbook.

...

Anchor
reorder
reorder
Reordering sheets in a workbook

To reorder sheets within a workbook:

  1. Right-click the sheet tab.
  2. Select Select Move.
  3. Click the space between the current sheet tabs where you want the selected sheet to be placed.

Anchor
reorder_column
reorder_column
Reordering columns in a workbook

...

a

...

workbook

...

To reorder columns on in a sheet:

  1. Left-click and drag the column name to the new desired position.

Anchor
sheet_dependency
sheet_dependency

...

Displaying sheet dependency (graph

...

overlay)

The graph overlay shows the current selected sheet with in feature provides a graphic display of the currently selected sheet and any ingoing and outgoing sheets in a visual manner.

  1. View the sheet dependency overlay by clicking the the Sheet Dependencies button  button in the toolbar. The current sheet in which you have selected with currently selected sheet will be highlighted in the middle of the graph.
  2. To navigate between dependent sheets, click the sheet name located on the graphic overlay.
  3. To close the sheet dependencies overlay, click the toolbar button or click click X on  on the current sheet.

Anchor
import_sheet
import_sheet
Importing sheets

Importing sheets allows you to chain workbooks together. 

To import sheets from one workbook into another workbook:

  1. Open the workbook you want to import the worksheet into.
  2. From the the Workbook menu menu, select select Add Data, or click the Add Data icon on the toolbar.
  3. Select First, select the workbook you want to use and click click Add Data.
  4. Select Then select the sheets you want to import and click click Add Data.
  5. The new worksheet tabs are added to the right of the existing worksheets.

Suggested column names

...

Anchor
set_up_col
set_up_col
Column names

When a user creates new columns using the Formula Builder

...

, Datameer creates default column names based on the column name

...

in the formula, and the formula type

...

.

If you want to change these names refer to setting up column names.

...

Column names can only contain letters (only standard, Column names in workbook sheets that are editable can be renamed. The status line just above the list of tabs indicates whether a particular sheet is editable.

Column names can only contain capital or lower-case characters), numbers, and /or underscores. Column names must A column name cannot begin with a letter or underscore.Column number. Note that column names are case sensitive. Currently there isn't a parameter available that can disable the case-sensitive nature of these names.

You can rename the column names on workbook sheets that are editable. The status line just above the list of tabs tells you if a particular sheet is writable.

...

! For example, column names foo, FOO, FoO and fOO will be unique columns within the same worksheet. 

Note
titleAs of Datameer 7.5

All column names, in all workbook sheet types, are case sensitive. This is a behavior change with version 7.5.

Note
titleAs of Datameer 7.2

Column names have a 255 character limit.

 To edit the a column name:

  1. Right-click on the column name and click click Rename.
  2. Enter the new name and press Enter.
  3. To undo the change, right-click the name again and click click Undo.

Resizing columns

To resize columns:

  1. Place the mouse between two column headings . The so the icon changes to a double-ended arrow.
  2. Drag the column marker to the desired width.
  3. Click to lock the width.

You can also double-click a column name to fully expand the column.

Anchor
showhide
showhide
Hiding and expanding columns

To hide (collapse) a column, right-click the column name and select select Hide Column. 

Double-click the hidden column to bring it back into view.

 

A toggle columns toolbar button displays all columns of the current worksheet. You can use this menu to show or hide columns as a batch process.

 

Anchor
insert_column
insert_column
Inserting columns

...

  1. Right-click a column name.
  2. Select Insert Column.
    A new column is added to the left of the chosen column chosen. 

    Note

    Each workbook can contain a maximum of 702 columns.

...

  1. Right-click a column name.
  2. Select Select Remove Column.

Anchor
cell_context
cell_context
Cell context menu options

...

  • Copy - Copy the cell value to the clipboard.
  • Filter by - A shortcut to open the filter feature and auto completes the fields using the equals expression to filter for matchs matches the cell's value.
  • Exclude - A shortcut to open the filter feature and auto completes the fileds fields using the does not equal expression to filter for matches that don't match the cell's value.

Anchor

...

splitting_

...

columns

...

splitting_

...

columns

...

To format the contents of a column:

  1. Right-click a column name.
  2. Select Format Cells.
    • General
      • Column text color
      • Column background color
      • Column text alignment (right, left, center)
    • Numbers
      • Thousands separator (comma to separate numbers by thousand)
      • Select the amount of decimal places to display in the workbook
    • Date
      • Input a parse pattern to display the date. (The default doesn't display milliseconds)

Image Removed

Shading rows in a workbook

...

Copy cells

Double-click a cell to select it, and then right-click it and select Copy Cell Content to copy its contents to the browser's clipboard. This functionality doesn't work with the Safari browser.

Image Removed

...

The inspector gives you important information about the data of each column in your workbook. Use this feature for quick insights into the data that each of your columns contain.

Section
Column

Image Removed

Column
width80%

Information displayed in the column inspector:

  • The name of the column being inspected.
  • The number and percentage of valid and empty rows.
  • A vertical bar chart displays all the different values and their frequency.
    • Mousing over a bar in the chart displays the value(s) and count.
    • If there are over 32 unique number type values, the bars group the values in bins.
    • If there are over 32 unique non-number type values, the vertical bar chart is not displayed.
  • The middle row of statisitcs displays the column's count of unique, minimum, mean, and maximum value.
    • Mousing over a value displays the full value.
    • Number and date field types are display all values. Othe field types display only the unique count.
  • The bottom horizontal bar chart displays the top unique values for the column and their respective counts.
    • Mousing over a bar in the chart displays the value and count.
    • Click the See more link below the bar chart to view additional column values in decending order.

...

Note

As of Datameer 7.2

There are two tabs at the top of the workbook inspector. The inspector is separated at the worksheet level and column level.

The worksheet level tab has a description box where text can be entered about each worksheet in the workbook.

Image Removed

The column level tab has a description box where text can be entered about each column of a worksheet. Below the description box, the column information is as described above.

Image Removed

...

  • Click View in the menu bar and select/unselect Inspector.
  • Click the Inspector icon in the tool bar.

Image Removed

Revert Actions

If you would like to revert an action that you performed in a workbook, you can  use the Undo and Redo buttons available in the toolbar. By default, you can undo or redo the last two actions performed on a workbook that modify the workbook state in the database. These actions don't include changes such as font style or column size, as the workbook's state remains the same. You can also access these options through the Edit menu or using keyboard shortcuts (Cmd+Shift+Z for undo, Cmd+Shift+Z for redo). If the cursor is in a text field, using the keyboard shortcut reverts only the most recent changes in the text field, not the whole workbook.

Image Removed

Administrators can adjust the settings for these buttons in default.properties and system.properties. There, admins can edit the amount of time the history of a workbook is kept, the maximum number of actions a a user can revert, and how many workbooks can retain a history at the same time.

To make sure this functionality isn't causing performance issues, you can check various statistics about the memory usage and workbook states in <Datameer>/logs/workbook_undo.log. The log is set to record the following values separated by comma:

 absoluteMemoryConsumption,absoluteMemoryUsageInBytes,generalMemoryUsage,historiesInUse,historyStatesInMemory,maxMemory,relativeUsageInPercent,usedMemory,usersWorking,workbooksInUse

All undo related actions are collected, but for memory performance, the system prints the values to the log once per minute. 

...

Formulas provide the ability to analyze your data in powerful ways.

Using the formula bar

The formula bar is located at the top of every worksheet within a workbook. Select a column on a worksheet and apply a formula with supported Datameer functions and operators.

Formulas can be added manually or using Datameer's Formula Builder.

Image Removed

...

Note

As of Datameer 7.2

The formula bar has multi-line functionality. Add more than one line when writing a formula so that it is easier to both read and write long and/or nested formulas.

Image Removed

The multi-line editor can be expanded or collapsed by clicking and holding the edge of the formula bar and dragging your mouse.

Referencing worksheets and columns in the formula editor

A formula can only reference columns from it's own worksheet and one other worksheet in the workbook. Referencing columns from different worksheets is possible after a join sheet has first been created. 

...

Writing a function in the formula editor

Datameer's supported functions.

Enter a function name into the formula editor (case insensitive) with variables separated by a semicolon within parentheses.

...

Example:

...

The Greater Than or Equal To function is being used comparing the "status" column from the "apache_log" worksheet
to see if the values are greater than or equal to the constant integer "400".

Writing nested functions in the formula editor

From the formula editor, you can enter functions and operators within functions.

...

You can create formulas on worksheets that are editable. When you click the data area of a column that has a formula associated with it, the formula displays above the workbook in the Fx bar. You can also create a new sheet and create formulas that reference fields from other sheets. Click the Fx button on the formula line to display the formula builder. (As of Datameer 7.2the formula builder is located in the worksheet inspector.)

To create a formula using the Formula Builder:

  1. Click the Fx button on the formula line to access the formula builder.
  2. Select the categories in the left column to choose the type of function.
  3. Choose a function from the list on the right.
  4. Enter the argument or arguments shown for that function. You can click a sheet and select a column.
  5. Click the Plus icon to enter additional arguments (if supported).
  6. Click OK.

The resulting function displays next to the fx icon. To learn more, see using the Formula Builder.

To use operators in a formula:

  1. Click the data area in a column and the current formula if one exists displays above the workbook.
  2. Create or edit the formula using operators such as < or > or <= and press Enter.

To use regular expressions in a formula:

  1. Click the data area in a column and the current formula if one exists displays above the workbook.
  2. Create or edit the formula using regular expressions press Enter.

See Using Regular Expressions to see some examples. Datameer offers an API to extend the built-in formulas. See the Developer's Guide to learn more.

Editing formulas

To edit formulas:

  1. Click the data area in a column and the current formula displays above the workbook.
  2. Edit the formula and press Enter.

...

Splitting columns

Datameer 7.5 includes the option to split workbook string and list type column content by a selected delimiter. Several optional additional controls are available. Resulting column names are the original column name with a sequential number appended. For list columns, commas are the only delimiter and there will be one column per comma up to the column limit.

The split column operation can be updated as many times as you like until you have desired split configuration.

To split a column:

  1. Right-click on the column header of the column you want to split to open the context menu and select Split Column, or use the icon in the toolbar.
    Image Added

    Image Added
  2. In the Split to Columns dialog, the Input column will be selected automatically. 
    Image Added

    Image Added
  3. For string columns, you must select the Type of column contents, either Delimited String or JSON Array String.
  4. For string columns, enter a Delimiter character.

The following optional controls are available:

  • (Strings only) Use Skip Leading to enter the number of characters that will be ignored before the delimiter used to split the column.
  • (Strings only) Use Skip Trailing to enter the number of characters that will be ignored after the specified number of columns are split.
  • Use Skip first to ignore the first x number of elements, where elements are the characters between delimiters.
  • Use Split into to specify a fixed number of columns into which data will be split.
  • (JSON array only) When Catch additional data in overflow column is checked, if the number of original elements exceeds the number of specified output columns, the remaining elements will be kept and grouped into one column, in brackets.
  • Use Drop additional elements to ignore all elements past the specified output columns. 
  • (Strings only) Use Trim Whitespace to delete any leading or trailing spaces and line breaks.
  • (Strings only) Use Define Empty Value to specify what is substituted in if no existing data is split into the last columns (i.e. when the amount of data is smaller than the column limit.)

      5. Optionally, enter a Column prefix that be prepended to the new column names. 
      6. Once you have entered your desired parameters, click Create Sheet. 
      7. A new SplitString or SplitList sheet opens containing the split columns.

If desired, you can use the Split tab in the Inspector to enter new split criteria. Click Update Sheet to apply changes.

Encoding Columns

Info
titleINFO

Column encoding performs ordinal, one-hot or binned encoding on column data which assigns a unique numeric value to each categorical or continuous value. Once applied, column encoding can be updated as needed until the desired results are achieved.

Columns with a high cardinality are not suited for ordinal/ binned encoding.


Tip

Column encoding provides a consistent view of prepared data which is especially helpful for teams working together on model building and testing activities.

Ordinal Encoding

For ordinal encoding:

  1. Right-click on a column header and select "Encoding" or click on the "Encode Column" icon from the toolbar. The 'Encode Column' dialog is displayed on the right. 
    Image Added 
    or Image Added
  2. If needed, change the column by entering the required column name in 'Column'. 
    Image Added 
  3. Select the encoding type "Ordinal Encoding" from the drop-down. Further selection options adapt to the needs.
    Image Added 
  4. Decide how to deal with unknown values by clicking the required statement. 
    INFO: 'Drop Value' ignores values beyond the first 100 most frequent. 'Default Value' shows values, which can not be encoded. 
    Image Added 
  5. View the top 32 values (by count).
    Image Added 
  6. If needed, add a new value in the blank field, change the order of the top values or delete single values.
    Image Added 
  7. Confirm with "Encode"The encoding result is displayed in a new encoding sheet within the workbook. Ordinal Encoding is finished. 
    Image Added 

One-hot Encoding

For one-hot encoding:

  1. Right-click on a column header and select "Encoding" or click on the "Encode Column" icon from the toolbar. The 'Encode Column' dialog is displayed on the right. 
    Image Added 
    or Image Added
  2. If needed, change the column by entering the required column name in 'Column'.
    Image Added 
  3. Select the encoding type "1-Hot Encoding" from the drop-down. Further selection options adapt to the needs.
    Image Added 
  4. Decide how to deal with unknown values by clicking the required statement. 
    INFO: 'Drop Value' ignores values beyond the first 100 most frequent. 'Include at last column' adds a new element to the list that encodes together all values beyond the 100 most frequent. 
    Image Added
  5. Select the output format from the dropdown 'Output'. 
    INFO: 'As List' keeps all binary pairs together in a single column. 'As Column' creates binary pairs each in their own column.
    Image Added
  6. View the top 32 values (by count).
    Image Added 
  7. If needed, add a new value in the blank field, change the order of the top values or delete single values.
    Image Added 
  8. Confirm with "Encode"The encoding result is displayed in a new encoding sheet within the workbook. One-hot encoding is finished.
    Image Added 

Anchor
DM_WB_Encoding_Binned
DM_WB_Encoding_Binned
Binned Encoding

For binned encoding:

  1. Right-click on a column header and select "Encoding" or click on the "Encode Column" icon from the toolbar. The 'Encode Column' dialog is displayed on the right. 
    Image Added 
    or Image Added
  2. If needed, change the column by entering the required column name in 'Column'.
    Image Added 
  3. Select the encoding type "Binned Encoding" from the drop-down. Further selection options adapt to the needs.
    Image Added 
  4. Select the output format from the dropdown 'Output'.
    INFO: 'As List' keeps all binary pairs together in a single column. 'Ordinal' encodes as ordinal numbers in a single column.
    Image Added 
  5. View the default value distributions. 
    INFO: The graph changes according to the amount of dividers. 
    Image Added 
  6. Enter the required bin dividers to change the percentile size of the divider. 
    INFO: There are 3 dividers set as default, e.g. 'Divider 1' contains the 25% of the selected column values. 
    INFO: To delete a divider, click on "x" next to the required divider.
    Image Added 
  7. If needed, click on "Add new Divider" to add a new divider. 
    INFO: Clinking the button adds an additional bucket, recalculates the percentile size and the corresponding absolute values. 
    Image Added 
  8. Confirm with "Encode"The encoding result is displayed in a new encoding sheet within the workbook. Binned encoding is finished. 
    Image Added

Anchor
format_column
format_column
Formatting columns

To format the contents of a column:

  1. Right-click a column name.
  2. Select Format Cells.
    • General
      • Column text color
      • Column background color
      • Column text alignment (right, left, center)
    • Numbers
      • Thousands separator (comma to separate numbers by thousand)
      • Select the amount of decimal places to display in the workbook
    • Date
      • Input a  parse pattern  to display the date. (The default doesn't display milliseconds)

Image Added

Shading rows in a workbook

The worksheet highlights rows by each group series when a  GROUPBY  function is used. 

Image Added

Copy cells

Double-click a cell to select it, and then right-click it and select Copy Cell Content to copy its contents to the browser's clipboard. This functionality doesn't work with the Safari browser.

Image Added

Anchor
inspector
inspector
Workbook inspector

The inspector gives you important information about the data of each column in your workbook. Use this feature for quick insights into the data that each of your columns contain.

Section
Column

Image Added

Column
width80%

Information displayed in the column inspector:

  • The name of the column being inspected.
  • The number and percentage of valid and empty rows.
  • A vertical bar chart displays all the different values and their frequency.
    • Mousing over a bar in the chart displays the value(s) and count.
    • If there are over 32 unique number type values, the bars group the values in bins.
    • If there are over 32 unique non-number type values, the vertical bar chart is not displayed.
  • The middle row of statistics displays the column's count of unique, minimum, mean, and maximum value.
    • Mousing over a value displays the full value.
    • Number and date field types are display all values. Other field types display only the unique count.
  • The bottom horizontal bar chart displays the top unique values for the column and their respective counts.
    • Mousing over a bar in the chart displays the value and count.
    • Click the See more link below the bar chart to view additional column values in descending order.

Anchor
inspector_tabs
inspector_tabs

Note

As of Datameer 7.2

There are two tabs at the top of the workbook inspector. The inspector is separated at the worksheet level and column level.

The worksheet level tab has a description box where text can be entered about each worksheet in the workbook.

Image Added

The column level tab has a description box where text can be entered about each column of a worksheet. Below the description box, the column information is as described above.

Image Added


To open or close the workbook column inspector:

  • Click View in the menu bar and select/unselect Inspector.
  • Click the Inspector icon in the tool bar.

Image Added

Anchor
dedupe
dedupe
Deduplication

The deduplication feature eliminates duplicate/redundant data from your worksheet columns. This procedure can be performed across all columns of a worksheet or for specifically selected columns. 

To deduplicate a column(s) in a worksheet:

  1. From the Edit menu, click Deduplicate OR click the Deduplicate icon on the toolbar.
    Image Added or Image Added
  2. The worksheet inspector changes to offer options on which columns you want to perform deduplication on. 
    Select the radio button for your choice between performing deduplication across all columns or select columns of the worksheet.
    1. If you selected the all columns option, click the Create Sheet button at the bottom of the inspector to apply the deduplication process.
      You workbook creates a new sheet titled TransformSheet. This sheet has purged any records that are a duplicate across all columns. 
      A new tab titled Deduplicate is located in your worksheet inspector.
      Run the workbook to view the full results.
    2. If you selected the select columns option, additional criteria fields appear.
      Enter one or multiple columns to be used to deduplicate your data. 
      Optionally, you can select an additional column and choose to keep the first or last record of the deduplicated data.
      Click the Create Sheet button at the bottom of the inspector to apply the deduplication process.
    Image Added

A deduplicated sheet can be updated by clicking on the Deduplicate tab on the page inspector. Make a change to any option(s) and click Update Sheet.

Image Added

Revert Actions

If you would like to revert an action that you performed in a workbook, you can  use the Undo and Redo buttons available in the toolbar. By default, you can undo or redo the last two actions performed on a workbook that modify the workbook state in the database. These actions don't include changes such as font style or column size, as the workbook's state remains the same. You can also access these options through the Edit menu or using keyboard shortcuts (Cmd+Shift+Z for undo, Cmd+Shift+Z for redo). If the cursor is in a text field, using the keyboard shortcut reverts only the most recent changes in the text field, not the whole workbook.

Image Added

Administrators can adjust the settings for these buttons in default.properties and system.properties. There, admins can edit the amount of time the history of a workbook is kept, the maximum number of actions a a user can revert, and how many workbooks can retain a history at the same time.

To make sure this functionality isn't causing performance issues, you can check various statistics about the memory usage and workbook states in <Datameer>/logs/workbook_undo.log. The log is set to record the following values separated by comma:

absoluteMemoryConsumption,absoluteMemoryUsageInBytes,generalMemoryUsage,historiesInUse,historyStatesInMemory,maxMemory,relativeUsageInPercent,usedMemory,usersWorking,workbooksInUse

All undo related actions are collected, but for memory performance, the system prints the values to the log once per minute. 

Anchor
formulas
formulas
Using Formulas

Formulas provide the ability to analyze your data in powerful ways.

Using the formula editor

The formula editor is located at the top of a worksheet within a workbook. Select a column on a worksheet and apply a formula with supported Datameer functions and operators.

Formulas can be added manually or using Datameer's Formula Builder. An auto-complete feature gives you the ability to visualize and complete formulas efficiently.

Image Added

Anchor
multi-line
multi-line

Note

As of Datameer 7.2

The formula editor has multi-line functionality. Add more than one line when writing a formula so that it is easier to both read and write long and/or nested formulas.

Image Added

Add additional lines by pressing Enter on your keyboard while in the editor. Submit your formulas by clicking the checkbox on the right of the editor or using the keyboard shortcut ⌘-Enter (mac) and Ctrl-Enter (win).

The multi-line editor can be expanded or collapsed by clicking and holding the edge of the formula bar and dragging your mouse.

Referencing worksheets and columns in the formula editor

A formula can only reference columns from it's own worksheet and one other worksheet in the workbook. Referencing columns from different worksheets is possible after a join sheet has first been created. 

Syntax for referencing a worksheet and column

SyntaxExample
Worksheet##apache_log
Column!!remoteUser
Worksheet and column#apache_log!remoteUser

Writing a function in the formula editor

Datameer's supported functions.

Enter a function name into the formula editor (case insensitive) with variables separated by a semicolon within parentheses.

Example:

Result explained
GE(#apache_log!status;400)

The Greater Than or Equal To function is being used comparing the "status" column from the "apache_log" worksheet
to see if the values are greater than or equal to the constant integer "400".

Writing nested functions in the formula editor

From the formula editor, you can enter functions and operators within functions.

ExampleResults explained
IF(EQUALS(#apache_log!status;400);"Bad Request";"other")The IF function is being used with the Equals function as one of the variables.

Anchor
formula_builder
formula_builder
Creating formulas using the formula builder

You can create formulas on worksheets that are editable. When you click the data area of a column that has a formula associated with it, the formula displays above the workbook in the Fx bar. You can also create a new sheet and create formulas that reference fields from other sheets. Click the  Fx  button on the formula line to display the formula builder. (As of Datameer 7.2the formula builder is located in the worksheet inspector.)

To create a formula using the Formula Builder:

  1. Click the Fx button on the formula line to access the formula builder.
  2. Select the categories in the left column to choose the type of function.
  3. Choose a function from the list on the right.
  4. Enter the argument or arguments shown for that function. You can click a sheet and select a column.
  5. Click the Plus icon to enter additional arguments (if supported).
  6. Click OK.

The resulting function displays next to the fx icon. To learn more, see using the Formula Builder.

To use operators in a formula:

  1. Click the data area in a column and the current formula if one exists displays above the workbook.
  2. Create or edit the formula using operators such as < or > or <= and press Enter.

To use regular expressions in a formula:

  1. Click the data area in a column and the current formula if one exists displays above the workbook.
  2. Create or edit the formula using regular expressions press Enter.

See Using Regular Expressions to see some examples. Datameer offers an API to extend the built-in formulas. See the Developer's Guide to learn more.

Editing formulas

To edit formulas:

  1. Click the data area in a column and the current formula displays above the workbook.
  2. Edit the formula and press Enter.

Anchor
variables
variables
Using Variables Within Workbooks

Variable values can be used within the worksheet formula bar as well as other areas such as the filter worksheet feature.

Call a variable value using the following syntax:

Panel

${<variable name>}

Image Added

Variable values can be set by an administrator.

Anchor
book_prod_mode
book_prod_mode
Workbook Production Mode

Production mode can be applied to a workbook to accelerate performance by skipping the calculation of metrics used for columnmetric, flip sheet displays and generation of data samples. 

To launch production mode for a workbook:

  1. Within the file browser, select a workbook.
  2. Open the Config tab of the Inspector.
  3. Check Enable Production Mode.

Production mode is effective the next time the workbook is executed.

Image Added Image Added

Completing Workbook Level Tasks

You can save workbooks, view or change workbook settings, link a workbook to multiple data sources, calculate a workbook, and more.

...

When you save a workbook, you also specify the workbook settings. If you don't know which settings to use, you can keep the default settings and you can change them later or the system administrator can set them. See See Configuring Workbook Settings to  to learn more.

To save workbooks:

  1. From the the File menu menu, select Save, or click the Save Workbook icon on the toolbar.
  2. Navigate to the folder where you want to save this workbook or choose to save the workbook in a new folder.
  3. Enter a name in the the Save  as field field.
  4. Click Click Save.

If workbook settings need to be updated after saving, right-click on the workbook in the file browser and select select Configure .

From the the File menu menu, select Save As, or click the Save Workbook As icon on the toolbar.To save a copy of a workbook:

  1. Navigate to the folder where you want to save the workbook, enter a name, and click click Save As.

Creating multiple workbooks

You can create multiple workbooks that reference the same data set:

  1. From the the File menu menu, select select New. Save your changes if needed.
  2. In the new workbook, from the the File menu menu, select select Add Data.
  3. Navigate to the data source you want to use and select the data source.
  4. Click Add Data.

Calculating a workbook

From the workbook you can calculate the workbook using the entire data set. Depending on the volume of data involved, this process might take awhile.

  1. From the the Workbook menu menu, select select Calculate or  or click the Calculate Full Workbook button on the toolbar. While the data is being calculated, you can click click Abort if  if you want to calculate at a later time.
  2. Once the calculation is complete, the the Full  Data page  page opens.
  3. You can click click Next button  button to view additional pages of data or click Go  To  Line and enter a line number to view a specific record.
  4. Click Click Edit to  to return to the workbook.

After making changes to an existing workbook, the next time the workbook is run, the changes are applied in the workbook calculation. Historical data before the change to the workbook isn't updated.

Viewing full results in a workbook

 To view full results:

  1. From the the Workbook menu menu, select select View Full Results, or click the View Full Results icon on the toolbar.
  2. Click Click Open to  to view the data set in the worksheet view or click click Next at  at the bottom of the table to view more records.

...

To go to a specific record while in the workbook:

  1. From the the Workbook menu menu, select select Go to Line or click the Go to Line button on the toolbar.
  2. Enter a line number to view a specific record and click click Go.

Viewing workbook details

...


To view workbook details:

  1. From the the Workbook menu menu, select select Workbook Info or  or click the Workbook Info button on the toolbar.
  2. The workbook information displays in a new window.

...

You can set up the schedule of when a job runs when you create a workbook or you can change the schedule settings later. See See Configuring Workbook Settings to  to learn more about scheduling.

...

  1. Click Copy Integration Link in the the Download dialog  dialog for full results page of a workbook. If you are using a browser that doesn't support copying to the clipboard, copy the provided URL instead.
  2. Go to Power BI Desktop and in the Get Data workflow, find Other Web.
  3. Paste the URL in dialog and click OK.
  4. Select Basic authentication and enter your Datameer username and password for the environment linked previously and click Connect.
  5. On the next screen review and confirm the data to be loaded.
  6. Data is loaded and you can take full advantage of Power BI capabilities. You can also publish your Powe rBI dashboards and reports to Power BI Cloud maintaining reference to this Datameer connection.

...

Workbook

...

Sharing Permissions and Security Settings

Only users with admin rights can set or change permissions.

To view sharing permissions and security settings for a workbook:

  1. Click the the File Browser tab  tab at the top of the page.
  2. Select the Workbooks folder folder.
  3. Click to highlight the workbook for which you want to edit permissions.
  4. Right-click the workbook and select select Information or  or click the Information button on the tool bar.
  5. The sharing and full results sharing permissions can be found by clicking the Lock icon in the info box.

...