The Ice Format is designed to support import/export of data from Opticks that preserves the Opticks specific data structures. The Ice Format is actually written as an HDF5 file. Information on HDF5 can be found at http://www.hdfgroup.org/HDF5/. This document will describe how the data is stored within the HDF5 file for various versions of the Ice Format.
Versions of Ice File:
| Major Ice Format Version # | Minor Ice Format Version # | Versions of Application That Will Export | Versions of Application That Will Import
|
| 0 | 0 | COMET MPR01.03.3686 to COMET MPR03.08.6674 | COMET MPR01.03.3686 to Current Version |
| 0 | 70 | COMET MPR03.09.7404 only | COMET MPR03.09.7404 to Current Version |
| 0 | 90 | COMET MPR03.10.8013 only | COMET MPR03.10.8013 to Current Version |
| 1 | 0 | COMET MPR03.11.738 to Opticks 4.0.0 | COMET MPR03.11.738 to Current Version |
| 1 | 10 | Opticks 4.0.1 to Current Version for the "RasterElement" and "PseudocolorLayer" file types | Opticks 4.0.1 to Current Version |
| 1 | 20 | Opticks 4.7.1 to Current Version for the "ThresholdLayer" file type | Opticks 4.7.1 to Current Version |
The most basic part of the Ice file is the IceFormatDescriptor which identifies an Ice file from any other HDF5 file. The IceFormatDescriptor is an HDF5 group with the name of "IceFormatDescriptor" written at the root node of the HDF5 file. Its contents are described below.
| Attribute Name | HDF5 Type | HDF5 DataSpace | Description |
| FormatVersion | H5T_NATIVE_UINT | Scalar | The format version is stored in the following manner: MajorFormatVersion * 100 + MinorFormatVersion. For example, 0.90 would be calculated as 0 * 100 + 90, so a value of 90 would be stored in this field. Another example, 2.32 would be calculated as 2 * 100 + 32, so a value of 232 would be stored in this field. |
| FileType | H5T_C_S1 | Scalar | The contents of this file. Valid values are "RasterElement", "PseudocolorLayer", and "ThresholdLayer". This field was introduced in version 1.10. The "ThresholdLayer" value was introduced in version 1.20. If the file version is prior to 1.10, readers can parse this file as though this field contains "RasterElement". |
| Creator | H5T_C_S1 | Scalar | Name of the application that wrote the ice.h5 file. |
| CreatorVersion | H5T_C_S1 | Scalar | Version # of the application that wrote the ice.h5 file. |
| CreatorOS | H5T_C_S1 | Scalar | Operating system name that application was running on when the .ice.h5 file was written. This value should only be used for informational purposes and should NOT be relied upon by readers or writers of this format. |
| CreatorArch | H5T_C_S1 | Scalar | Architecture of the system that the application was running on when the .ice.h5 file was written. This should only be used for informational purposes and should NOT be relied upon by readers or writers of this format. |
| Interleave | Dimension 0 | Dimension 1 | Dimension 2 |
| BIP | Row | Column | Band |
| BSQ | Band | Row | Column |
| BIL | Row | Band | Column |
In order to determine the interleave, there will be an HDF5 Attribute present on the HDF5 Dataset called "InterleaveFormat" which will be of type H5T_C_S1 that has a scalar dataspace. The value of the string will be one of the following: "BIP", "BSQ" or "BIL". The HDF5 type of the HDF5 Dataset can be one of the following:
| HDF5 Type | EnumValue used by application in EncodingType enum |
| H5T_NATIVE_CHAR | INT1SBYTE |
| H5T_NATIVE_UCHAR | INT1UBYTE |
| H5T_NATIVE_SHORT | INT2SBYTES |
| H5T_NATIVE_USHORT | INT2UBYTES |
| H5T_NATIVE_INT | INT4SBYTES |
| H5T_NATIVE_UINT | INT4UBYTES |
| H5T_NATIVE_FLOAT | FLT4BYTES |
| H5T_NATIVE_DOUBLE | FLT8BYTES |
A HDF5 Compound Datatype with two members:
| INT4SCOMPLEX |
A HDF5 Compound Datatype with two members:
| FLT8COMPLEX |
The following only applies to version 0.0 of the Ice format:
The original numbers are stored as HDF5 Attributes of this HDF5 Dataset. There are 3 attributes with the following names: "Original Cube Row Numbers", "Original Cube Column Numbers", and "Original Cube Band Numbers". All three of the attributes are of type H5T_NATIVE_UINT and have a dataspace with 1 dimension. As of version 0.70 these values are now being stored inside an HDF5 Group, please see OriginalNumbers for more information.
| Dataset Name | HDF5 Type | HDF5 DataSpace |
| Row | H5T_NATIVE_UINT | 1 dimension |
| Column | H5T_NATIVE_UINT | 1 dimension |
| Band | H5T_NATIVE_UINT | 1 dimension |
The size of the first and only dimension for each Dataset must be the same size as the corresponding dimension in RawData . For example, if the size of the three dimensions for RawData are {120, 250, 10} and the InterleaveFormat attribute for the RawData is "BIP". The "Row" dataset must then have a size of 120 for its first dimension. The "Column" dataset must then have a size of 250 for its first dimension. The "Band" dataset must then have a size of 10 for its first dimension. As another example, if the size of the three dimensions for RawData are {8, 500, 384} and the InterleaveFormat attribute for the Raw Data is "BSQ". The "Row" dataset must then have a size of 500 for its first dimension. The "Column" dataset must then have a size of 384 for its first dimension. The "Band" dataset must then have a size of 8 for its first dimension.
| Dataset Name | HDF5 Type | HDF5 DataSpace | Description |
| Start | H5T_NATIVE_DOUBLE | 1 dimension | This dataset will contain the micron value of the starting wavelength value detected by the sensor that recorded the information for the given band in RawData. This dataset is optional and may not be present. |
| Center | H5T_NATIVE_DOUBLE | 1 dimension | This dataset will contain the micron value of the center wavelength value detected by the sensor that recorded the information for the given band in RawData. This dataset is optional and may not be present. |
| End | H5T_NATIVE_DOUBLE | 1 dimension | This dataset will contain the micron value of the ending wavelength value detected by the sensor that recorded the information for the given band in RawData. This dataset is optional and may not be present. |
For all of the HDF5 Datasets shown above, the size of their dimension will be the same size as the Band dimension in RawData. An example of how data is stored in this group is shown below. Imagine that a sensor detected the following wavelengths as shown below:
| Band | Start Wavelength Detected (in Microns) | Center Wavelength Detected (in Microns) | End Wavelength Detected (in Microns) |
| 0 | 0.34 | 0.44 | 0.54 |
| 1 | 0.52 | 0.62 | 0.72 |
| 2 | 0.70 | 0.80 | 0.90 |
Then if data from this sensor was stored in an Ice formatted file, the following would be true for the HDF5 datasets stored as children of this group. The "Start" HDF5 Dataset would contain (0.34, 0.52, 0.70). The "Center" HDF5 Dataset would contain (0.44, 0.62, 0.80). The "End" HDF5 Dataset would contain (0.54, 0.72, 0.90). If data from only band 0 and band 2 of this sensor was stored in an Ice formatted file, the following would be true for the HDF5 datasets stored as children of this group. The "Start" HDF5 Dataset would contain (0.34, 0.70). The "Center" HDF5 Dataset would contain (0.44, 0.80). The "End" HDF5 Dataset would contain (0.54, 0.90).
| Name | HDF5 Container | HDF5 Type | HDF5 DataSpace | Description |
| ClassificationText | HDF5 Attribute | H5T_C_S1 | Scalar | Contains the text of the security marking that should be placed on the top and bottom of the image rendered from the RawData. This attribute is derived from the contents of the other attributes present on the Classification group. Changing the value of this attribute is not supported by external readers or writers of Ice formatted files. |
| Level | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| System | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| Codewords | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| Control | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| Releasing | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| Reason | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| DeclassificationType | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| DeclassificationDate | HDF5 Attribute | FILLIN | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| DeclassificationExemption | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| DowngradeLevel | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| DowngradeDate | HDF5 Attribute | FILIN | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| CountryCode | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| Description | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| Authority | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| AuthorityType | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| SourceDate | HDF5 Attribute | FILLIN | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| SecurityControlNumber | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| CopyNumber | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| NumberOfCopies | HDF5 Attribute | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| Metadata | HDF5 Dataset | H5T_C_S1 | Scalar | INTERNAL ONLY. NOT SUPPORTED FOR EXTERNAL READERS/WRITERS. |
| Member Name | HDF5 Type | HDF5 DataSpace | Description |
| pixelX | H5T_NATIVE_DOUBLE | Scalar | This value will be the on-disk column number that this point refers to. See Discussion of Original, On-Disk and Active Numbers for details on the on-disk number |
| pixelY | H5T_NATIVE_DOUBLE | Scalar | This value will be the on-disk row number that this point refers to. See Discussion of Original, On-Disk and Active Numbers for details on the on-disk number |
| latitude | H5T_NATIVE_DOUBLE | Scalar | This value will be the latitude that corresponds to the given pixelX. |
| longitude | H5T_NATIVE_DOUBLE | Scalar | This value will be the longitude that corresponds to the given pixelY. |
The latitude and longitude values are defined to use the WGS84 (World Geodetic System 84) coordinate system. The accuracy of the latitude and longitude values for the given pixelX and pixelY values is not defined by the Ice format. The pixelX and pixelY correspond to the on-disk numbers of the RawData. The pixelX and pixelY are stored as floating point numbers, but the on-disk numbers of the RawData are integral. This allows the latitude and longitude to be stored for intermediate points on the RawData. For example, the latitude and longitude values for pixelX 20.5 and pixelY 50.891 could be stored in this HDF5 dataset. This HDF5 Dataset will usually contain latitude and longitude information for the 4 corners of the RawData, but that is not required. This HDF5 Dataset will only be written out if the geocoordinate information was available to the application.
| Name | HDF5 Container | HDF5 Type | HDF5 DataSpace | Description |
| RangeMax | HDF5 Attribute | H5T_NATIVE_DOUBLE | Scalar | The specifies the maximum value allowed for data with this unit. Please see Units::getRangeMax() or Units::setRangeMax() for more details. |
| RangeMin | HDF5 Attribute | H5T_NATIVE_DOUBLE | Scalar | The specifies the minimum value allowed for data with this unit. Please see Units::getRangeMin() or Units::setRangeMin() for more details. |
| ScaleFromStandard | HDF5 Attribute | H5T_NATIVE_DOUBLE | Scalar | Specifies the scaling value to scale these data values back to the standard unit value. Please see Units::getScaleFromStandard() or Units::setScaleFromStandard() for more details. |
| Name | HDF5 Attribute | H5T_C_S1 | Scalar | Provides a textual name for this unit. |
| Type | HDF5 Attribute | H5T_C_S1 | Scalar | Gets the actual type of the unit, which must be one of the following:
|
| Name | HDF5 Container | HDF5 Type | HDF5 DataSpace | Description |
| GrayDisplayedBand | HDF5 Attribute | H5T_NATIVE_UINT | Scalar | This is the on-disk number of the band that should be displayed to the user when the display mode is grayscale. Please refer to the RawData for more details. |
| RedDisplayedBand | HDF5 Attribute | H5T_NATIVE_UINT | Scalar | This is the on-disk number of the band that should be displayed to the user for the red channel when the display mode is rgb. Please refer to the RawData for more details. |
| GreenDisplayedBand | HDF5 Attribute | H5T_NATIVE_UINT | Scalar | This is the on-disk number of the band that should be displayed to the user for the green channel when the display mode is rgb. Please refer to the RawData for more details. |
| BlueDisplayedBand | HDF5 Attribute | H5T_NATIVE_UINT | Scalar | This is the on-disk number of the band that should be displayed to the user for the blue channel when the display mode is rgb. Please refer to the RawData for more details. |
| DisplayMode | HDF5 Attribute | H5T_C_S1 | Scalar | This defines whether the raw data should be displayed in grayscale or rgb mode. See the DisplayedBand attributes above for details on what part of the raw data will be displayed to the user. If the value of this attribute is "grayscale" then the raw data will be displayed in grayscale mode. If the value of this attribute is "rgb" then the raw data will be displayed in rgb mode. |
| XPixelSize | HDF5 Attribute | H5T_NATIVE_DOUBLE | Scalar | This specifies how the data should be scaled on the x-dimension or column when displayed to the user. A XPixelSize of 1.0 means to draw the raw data at its original size. A XPixelSize of 2.0 means to draw the raw data twice as large in the x-dimension as its original size. |
| YPixelSize | HDF5 Attribute | H5T_NATIVE_DOUBLE | Scalar | This specifies how the data should be scaled on the y-dimension or row when displayed to the user. A YPixelSize of 1.0 means to draw the raw data at its original size. A YPixelSize of 2.0 means to draw the raw data twice as large in the y-dimension as its original size. |
| Member Name | HDF5 Type | HDF5 DataSpace | Description |
| resolution | H5T_NATIVE_UINT | Scalar | This is the number of rows and columns within a band that must be skipped when an application wishes to generate band statistics. A value of 0 indicates that the application can choose whether to skip rows or columns when generating band statistics. A value of 1, would indicate that an application should read rows 0, 2, 4, 6, 8, etc of a band when generating statistics. A value of 2, would indicate that an application should read rows 0, 3, 6, 9, 12, etc of a band when generating statistics. |
| badValues | H5T_NATIVE_INT | Variable length | This indicates the raw data values that should be ignored by the application when generating statistics for a band. This is a variable length array and can contain an unlimited number of values. The values stored are int, but the raw data can be stored in a non integral format. If that is the case, the application should truncate the raw data values before comparing them to the bad values present in this member. |
This HDF5 Dataset must be have the same number of values in it as the size of the band dimension in RawData.
| Member Name | HDF5 Type | HDF5 DataSpace | Description |
| onDiskNumber | H5T_NATIVE_UINT | Scalar | This is the on-disk number of the band in RawData that these statistics were calculated from. |
| average | H5T_NATIVE_DOUBLE | Scalar | This is the average of all the values found in the given band in the RawData. |
| min | H5T_NATIVE_DOUBLE | Scalar | This is the minimum value found in the given band in the RawData. |
| max | H5T_NATIVE_DOUBLE | Scalar | This is the maximum value found in the given band in the RawData. |
| standardDeviation | H5T_NATIVE_DOUBLE | Scalar | This is the standard deviation of all the values found in the given band in the RawData. |
| percentiles | H5T_NATIVE_DOUBLE | Variable length (NOTE: This is defined as a variable length type, but it must contain 1001 values). | This array is used to store percentile information for the values found in a given band. For example, if you get the value at position 749. Then 75% of the values found in the given band have a value less than this entry. Conversely, 25% of the values found in the given band have a value greater than this entry. For another example, if you get the value at position 329. Then 33% of the values found in the given band have value less than this entry. Conversely, 67% of the values found in the given band have a value greater than this entry. |
| binCenters | H5T_NATIVE_DOUBLE | Variable length (NOTE: This is defined as a variable length type, but it must contain 256 values). | This array is used to store bin information for a band. The difference between the minimum and maximum value in a band is divided into 256 sections. Each value in the array indicates the raw value that corresponds to the center value of each bin. |
| histogramCounts | H5T_NATIVE_UINT | Variable length (NOTE: This is defined as a variable length type, but it must contain 256 values). | This array is used to store information related to each bin as defined in the binCenters member. Each value in the array indicates the number of raw values in the band that fit into the bin. The range of each bin is defined by the binCenters member documented above. |
This HDF5 Dataset will only contain actual band statistics values for those bands that already had their statistics information calculated at the time that the Ice file was created. This means that this HDF5 Dataset is not required to be the same size as the bands dimension in the RawData and in fact may not exist at all in the Ice file. When interpreting the band statistics values from this HDF5 Dataset keep in mind they were calculated using the resolution and bad values for the band as stored in Band Statistics Metadata.
| Name | HDF5 Type | HDF5 DataSpace | Description |
| Name | H5T_C_S1 | Scalar | The name of this layer. |
| LayerType | H5T_C_S1 | Scalar | The type of this layer. The only supported values are "PseudocolorLayer" and "ThresholdLayer". |
| XScaleFactor | H5T_NATIVE_DOUBLE | Scalar | This value will be the X scale factor(with respect to the view) to use when drawing the layer. |
| YScaleFactor | H5T_NATIVE_DOUBLE | Scalar | This value will be the Y scale factor (with respect to the view) to use when drawing the layer. |
| XOffset | H5T_NATIVE_DOUBLE | Scalar | This value will be the X offset (with respect to the view) to use when drawing the layer. |
| YOffset | H5T_NATIVE_DOUBLE | Scalar | This value will be the Y offset (with respect to the view) to use when drawing the layer. |
| Name | HDF5 Type | HDF5 DataSpace | Description |
| Dataset | H5T_C_S1 | Scalar | The full path of the cube containing the cube data for this layer. This value must be "/Datasets/Cube1". |
| Name | HDF5 Container | HDF5 Type | HDF5 DataSpace | Description |
| Symbol | HDF5 Attribute | H5T_C_S1 | Scalar | The symbol type to use when displaying this pseudocolor layer. Valid values are "solid", "x", "crosshair", "asterisk", "horizontalLine", "verticalLine", "forwardSlash", "backSlash", "box", "boxX", "boxCrosshair", "boxAsterisk", "boxHorizontalLine", "boxVerticalLine", "boxForwardSlash", and "boxBackSlash". |
| ClassNames | HDF5 Dataset | H5T_C_S1 | 1 dimension | This dataset will contain the name of each pseudocolor layer class. |
| ClassValues | HDF5 Dataset | H5T_NATIVE_INT | 1 dimension | This dataset will contain the value of each pseudocolor layer class. For each occurrence of the value in the Dataset, a symbol of the specified color will be displayed. |
| ClassColors | HDF5 Dataset | H5T_C_S1 | 1 dimension | This dataset will contain the color of each pseudocolor layer class. This string is in the format "#AARRGGBB", where AA, RR, GG, and BB correspond to unsigned, 8-bit hexadceimal values for the alpha (opacity), red, green, and blue values respectively. |
| ClassIsDisplayed | HDF5 Dataset | H5T_NATIVE_UCHAR | 1 dimension | This dataset will contain a 0 or 1 for each pseudocolor layer class. 0 indicates that the class should not be displayed. All other values indicate that the class should be displayed. |
| Name | HDF5 Type | HDF5 DataSpace | Description |
| Dataset | H5T_C_S1 | Scalar | The full path of the cube containing the cube data for this layer. This value must be "/Datasets/Cube1". |
| Name | HDF5 Container | HDF5 Type | HDF5 DataSpace | Description |
| Symbol | HDF5 Attribute | H5T_C_S1 | Scalar | The symbol type to use when displaying the threshold layer. Valid values are "solid", "x", "crosshair", "asterisk", "horizontalLine", "verticalLine", "forwardSlash", "backSlash", "box", "boxX", "boxCrosshair", "boxAsterisk", "boxHorizontalLine", "boxVerticalLine", "boxForwardSlash", and "boxBackSlash". |
| Color | HDF5 Attribute | H5T_C_S1 | Scalar | The pixel marker color for the threshold layer. This string is in the format "#AARRGGBB", where AA, RR, GG, and BB correspond to unsigned, 8-bit hexadceimal values for the alpha (opacity), red, green, and blue values respectively. |
| PassArea | HDF5 Attribute | H5T_C_S1 | Scalar | The pass area identifies the pixel values in the threshold layer that will be used to flag pixels relative to the threshold value(s). Valid values are "Lower", "Upper", "Middle", and "Outside". |
| RegionUnits | HDF5 Attribute | H5T_C_S1 | Scalar | The regions units specify how the threshold values are measured against the pixel values. Valid values are "Raw", "Percentage", "Percentile", and "StdDev". |
| FirstThreshold | HDF5 Attribute | H5T_NATIVE_DOUBLE | Scalar | For pass areas of "Lower" and "Upper", this is the single threshold value that is used to flag pixels. For pass areas of "Middle" and "Outside", this is the lesser threshold value that is used in conjunction with the second threshold value to flag pixels. |
| SecondThreshold | HDF5 Attribute | H5T_NATIVE_DOUBLE | Scalar | For pass areas of "Middle" and "Outside", this is the greater threshold value that is used in conjunction with the first threshold value to flag pixels. For pass areas of "Lower" and "Upper", this value is ignored. |
| Table 1. Original SampleData | |||||||
| Column D(On-Disk Row #, On-Disk Column #) O(Original Row #, Original Column #) | |||||||
| Row | D(0, 0) O(0, 0) | D(0, 1) O(0, 1) | D(0, 2) O(0, 2) | D(0, 3) O(0, 3) | D(0, 4) O(0, 4) | D(0, 5) O(0, 5) | D(0, 6) O(0, 6) |
| D(1, 0) O(1, 0) | D(1, 1) O(1, 1) | D(1, 2) O(1, 2) | D(1, 3) O(1, 3) | D(1, 4) O(1, 4) | D(1, 5) O(1, 5) | D(1, 6) O(1, 6) | |
| D(2, 0) O(2, 0) | D(2, 1) O(2, 1) | D(2, 2) O(2, 2) | D(2, 3) O(2, 3) | D(2, 4) O(2, 4) | D(2, 5) O(2, 5) | D(2, 6) O(2, 6) | |
| D(3, 0) O(3, 0) | D(3, 1) O(3, 1) | D(3, 2) O(3, 2) | D(3, 3) O(3, 3) | D(3, 4) O(3, 4) | D(3, 5) O(3, 5) | D(3, 6) O(3, 6) | |
| D(4, 0) O(4, 0) | D(4, 1) O(4, 1) | D(4, 2) O(4, 2) | D(4, 3) O(4, 3) | D(4, 4) O(4, 4) | D(4, 5) O(4, 5) | D(4, 6) O(4, 6) | |
Now imagine if that data was loaded into the application, but skipping the first 2 rows and the first 3 columns and only loading 2 rows and 3 columns. That loaded data is now shown below, notice the addition of Active numbers which are only valid while data is loaded in the application:
| Table 2. SampleData from Table 1 Loaded Into Application (Skipped first 2 rows and first 3 columns and only loading 2 rows and 3 columns) | |||
| Column D(On-Disk Row #, On-Disk Column #) O(Original Row #, Original Column #) A(Active Row #, Active Column #) | |||
| Row | D(2, 3) O(2, 3) A(0, 0) | D(2, 4) O(2, 4) A(0, 1) | D(2, 5) O(2, 5) A(0, 2) |
| D(3, 3) O(3, 3) A(1, 0) | D(3, 4) O(3, 4) A(1, 1) | D(3, 5) O(3, 5) A(1, 2) | |
Below you will see a snapshot of Table 1 with the data that was loaded into Table 2 bolded:
| Table 3. Data from Table 1 bolded to show what was loaded into Table 2 | |||||||
| Column D(On-Disk Row #, On-Disk Column #) O(Original Row #, Original Column #) | |||||||
| Row | D(0, 0) O(0, 0) | D(0, 1) O(0, 1) | D(0, 2) O(0, 2) | D(0, 3) O(0, 3) | D(0, 4) O(0, 4) | D(0, 5) O(0, 5) | D(0, 6) O(0, 6) |
| D(1, 0) O(1, 0) | D(1, 1) O(1, 1) | D(1, 2) O(1, 2) | D(1, 3) O(1, 3) | D(1, 4) O(1, 4) | D(1, 5) O(1, 5) | D(1, 6) O(1, 6) | |
| D(2, 0) O(2, 0) | D(2, 1) O(2, 1) | D(2, 2) O(2, 2) | D(2, 3) O(2, 3) | D(2, 4) O(2, 4) | D(2, 5) O(2, 5) | D(2, 6) O(2, 6) | |
| D(3, 0) O(3, 0) | D(3, 1) O(3, 1) | D(3, 2) O(3, 2) | D(3, 3) O(3, 3) | D(3, 4) O(3, 4) | D(3, 5) O(3, 5) | D(3, 6) O(3, 6) | |
| D(4, 0) O(4, 0) | D(4, 1) O(4, 1) | D(4, 2) O(4, 2) | D(4, 3) O(4, 3) | D(4, 4) O(4, 4) | D(4, 5) O(4, 5) | D(4, 6) O(4, 6) | |
Now imagine that this data was exported from the application and written back to a file on the disk. The table below shows that, notice the removal of active numbers because the data is no longer loaded in the application:
| Table 4. SampleData in Table 2 now written to the disk | |||
| Column D(On-Disk Row #, On-Disk Column #) O(Original Row #, Original Column #) | |||
| Row | D(0, 0) O(2, 3) | D(0, 1) O(2, 4) | D(0, 2) O(2, 5) |
| D(1, 0) O(3, 3) | D(1, 1) O(3, 4) | D(1, 2) O(3, 5) | |
Now imagine if this new data that was just written to the disk was reloaded into the application but the second column was skipped., The table for this is shown below:
| Table 5. SampleData in Table 4 loaded back into the application but skipping the second column | ||
| Column D(On-Disk Row #, On-Disk Column #) O(Original Row #, Original Column #) A(Active Row #, Active Column #) | ||
| Row | D(0, 0) O(2, 3) A(0, 0) | D(0, 2) O(2, 5) A(0, 1) |
| D(1, 0) O(3, 3) A(1, 0) | D(1, 2) O(3, 5) A(1, 1) | |
Below you will see a snapshot of Table 1 with the rows still present in Table 5 highlighted in bold.
| Table 6. Data from Table 1 bolded to show data still present in Table 5. | |||||||
| Column D(On-Disk Row #, On-Disk Column #) O(Original Row #, Original Column #) | |||||||
| Row | D(0, 0) O(0, 0) | D(0, 1) O(0, 1) | D(0, 2) O(0, 2) | D(0, 3) O(0, 3) | D(0, 4) O(0, 4) | D(0, 5) O(0, 5) | D(0, 6) O(0, 6) |
| D(1, 0) O(1, 0) | D(1, 1) O(1, 1) | D(1, 2) O(1, 2) | D(1, 3) O(1, 3) | D(1, 4) O(1, 4) | D(1, 5) O(1, 5) | D(1, 6) O(1, 6) | |
| D(2, 0) O(2, 0) | D(2, 1) O(2, 1) | D(2, 2) O(2, 2) | D(2, 3) O(2, 3) | D(2, 4) O(2, 4) | D(2, 5) O(2, 5) | D(2, 6) O(2, 6) | |
| D(3, 0) O(3, 0) | D(3, 1) O(3, 1) | D(3, 2) O(3, 2) | D(3, 3) O(3, 3) | D(3, 4) O(3, 4) | D(3, 5) O(3, 5) | D(3, 6) O(3, 6) | |
| D(4, 0) O(4, 0) | D(4, 1) O(4, 1) | D(4, 2) O(4, 2) | D(4, 3) O(4, 3) | D(4, 4) O(4, 4) | D(4, 5) O(4, 5) | D(4, 6) O(4, 6) | |
From the above example, you can see that the original number of a given row and column were preserved in the data from Table 1 to Table 5. If we look at Table 1, the original numbers for the rows would be (0, 1, 2, 3, 4) while the original numbers for the columns would be (0, 1, 2, 3, 4, 5, 6). If we look at Table 4, the original numbers for the rows would be (2, 3) while the original numbers for the columns would be (3, 4, 5).