Precision#
GraphX separates numerical storage, numerical presentation, and visual placement aids. Calibration and point geometry are retained as numbers; dataset Precision chooses how coordinates are displayed and exported; the Precision magnifier helps you see the source image more closely. Changing one does not silently change the others.
Significant figures#
Significant figures describe the number of meaningful digits, regardless of where the decimal point falls. Open Precision from Data points and choose Significant figures for either axis when a consistent measurement precision is more useful than a fixed decimal place.
For example:
- 60.0028 at 3 significant figures displays as
60.0. - 1.2 at 4 significant figures displays as
1.200. - 0.006 at 3 significant figures displays as
0.00600.
The trailing zeroes are intentional: they communicate the selected precision rather than implying that GraphX discarded them. Significant figures are useful when values span several orders of magnitude, such as concentration or response data.
Decimal places#
Decimal places fixes the number of digits after the decimal point. It is useful where an axis has a fixed reporting resolution. For example, 12.3456 at 2 decimal places displays as 12.35, while 1.2 at 3 decimal places displays as 1.200.
The two modes answer different questions: significant figures preserve a count of meaningful digits; decimal places preserve a fixed decimal resolution. Choose the one that matches the way the resulting data will be read or reported.
X and Y independently#
X and Y use independent precision settings for every dataset. A time axis might use 2 decimal places while a pressure axis uses 4 significant figures. The controls in the Precision dialog are therefore grouped into X values and Y values rather than applying one global format to both columns.
New datasets begin with 4 significant figures for X and Y. Significant figures allow 1–15; decimal places allow 0–15.
Apply to all datasets#
Apply updates the active dataset. Apply to all datasets copies the current X/Y choice to every dataset in the project. This is a one-time copy, not a link: later changes to one dataset’s precision do not alter the others.
Where precision is used#
Dataset precision formats graph coordinates in the Data points table, CSV export, copied tab-separated values, and PDF data tables. It is applied independently to the X and Y coordinates of each dataset, so adjacent datasets in a multi-dataset CSV can legitimately use different formatting.
What it does not change#
Dataset precision is presentation and export formatting only. It does not round or alter calibration values, stored point geometry, direct coordinate edits, trace calculations, point sorting, trend fitting, Axis lock values, or Crosshair calculations. This is especially important when a table displays fewer digits than GraphX retains internally.
If you enter a point coordinate directly, GraphX uses the entered numerical value rather than the abbreviated display string. If calibration is later changed, point coordinates are recalculated from their source-image positions under that calibration; that behaviour is independent of dataset precision.
Scientific notation#
For extremely small or large significant-figure values, GraphX uses scientific notation so a fixed-format decimal does not become unwieldy. The current formatter switches below 1e-9 or at/above 1e21. Decimal-place formatting remains a fixed decimal representation. These thresholds are display choices; they do not change the underlying values.
Precision magnifier#
The Precision magnifier is enabled from the graph toolbar’s Display menu. It enlarges the original source pixels around the cursor with a centred crosshair, helping you place calibration references or points. Its Magnifier zoom range is 100%–400%, with 200% as the default.
The magnifier is visual only. It does not change graph zoom, image data, calibration, stored point positions, or dataset number formatting. See Interface overview for the surrounding Display controls.
For the distinction between image, graph, and viewport coordinates, see Data and coordinate model.