next up previous index
Next: Hyperslab Selection Up: Property Lists Previous: Create a Compressed Dataset

Activate Checksum for Error Detection

The program in this section activates checksum for error detection on writes to and reads from a dataset. The program begins by creating a simple dataspace and then filling a data array of rank 2 with integers following the formula $a_{ij} = 6 \times i + j + 1$. The HDF5 file cksum.h5 is created and then we get down to constructing a property list for the dataset. The list contains two requests. The first request is chunking, and the second request is that a filter should be applied to IO operations. The requested filter is H5Z_FILTER_FLETCHER32 , which implements checksum error detection. Finally we create a dataset /dset with requested properties. The write takes place, upon which we close the property list, the dataset and the file.

In the second part of the program we open the file and the the dataset. We create a property list again, but this time we use it in order to inspect whether checksum has been enabled on the data set - the result of the inspection is printed on standard output. The data is then read, but not printed on standard output. We only print the exit status of H5Dread. Then we close the property list, the dataset and the file.

Here is the program itself (this program originates from the NCSA HDF5 Tutorial):

/* 
   h5cksum: This example uses the Fletcher32 checksum algorithm
            for error detection
 */

#include "hdf5.h"
#define FILE "cksum.h5"

main() {

   hid_t       file_id, dataset_id, dataspace_id, dxpl, property_id;
   hsize_t     dims[2];
   herr_t      status;
   hsize_t     chkdim[2] ={2,3};
   int         i, j, dset_data[4][6];
   H5Z_EDC_t   edc;
   

     /* Create the data space for the dataset. */
   dims[0] = 4;
   dims[1] = 6;
   dataspace_id = H5Screate_simple(2, dims, NULL);

   /* Initialize the dataset. */
   for (i = 0; i < 4; i++)
      for (j = 0; j < 6; j++)
         dset_data[i][j] = i * 6 + j + 1;

   /* Create a new file using default properties. */
   file_id = H5Fcreate (FILE, H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT);

   /* Create the dataset with checksum filter */
   property_id = H5Pcreate(H5P_DATASET_CREATE);
   status = H5Pset_chunk (property_id, 2, chkdim); 
   status = H5Pset_filter (property_id, H5Z_FILTER_FLETCHER32, 0, 0, NULL);

   dataset_id = H5Dcreate (file_id, "/dset", H5T_STD_I32BE, dataspace_id, 
                           property_id);

   status = H5Dwrite(dataset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT, 
                     dset_data);

   status = H5Pclose (property_id);
   status = H5Dclose(dataset_id);
   status = H5Fclose(file_id);

   file_id = H5Fopen(FILE, H5F_ACC_RDWR, H5P_DEFAULT);
   dataset_id = H5Dopen (file_id, "/dset");

   dxpl = H5Pcreate (H5P_DATASET_XFER);
   edc = H5Pget_edc_check (dxpl);
   if (edc == 1) printf ("Checksum is enabled\n");
   if (edc == 0) printf ("Checksum is disabled\n");

   status = H5Dread(dataset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, dxpl, 
                    dset_data);
   printf ("H5Dread returns: %i\n", status); 

   status = H5Pclose (dxpl);
   status = H5Dclose(dataset_id);
   status = H5Fclose(file_id);

}
Here is how the program is compiled, linked and run:
gustav@bh1 $ h5cc -o h5_cksum h5_cksum.c
gustav@bh1 $ ./h5_cksum
Checksum is enabled
H5Dread returns: 0
gustav@bh1 $
The generated HDF5 file has a simple and small dump:
gustav@bh1 $ h5dump cksum.h5
HDF5 "cksum.h5" {
GROUP "/" {
   DATASET "dset" {
      DATATYPE  H5T_STD_I32BE
      DATASPACE  SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
      DATA {
         1, 2, 3, 4, 5, 6,
         7, 8, 9, 10, 11, 12,
         13, 14, 15, 16, 17, 18,
         19, 20, 21, 22, 23, 24
      }
   }
}
}
gustav@bh1 $
You can see that the dataset has checksum filter attached to it when you run h5ls on the file:
gustav@bh1 $ h5ls -v -r cksum.h5
Opened "cksum.h5" with sec2 driver.
/dset                    Dataset {4/4, 6/6}
    Location:  0:1:0:976
    Links:     1
    Modified:  2003-11-24 18:20:54 EST
    Chunks:    {2, 3} 24 bytes
    Storage:   96 logical bytes, 112 allocated bytes, 85.71% utilization
    Filter-0:  fletcher32-3  {}
    Type:      32-bit big-endian integer
gustav@bh1 $
The program itself should be easy to understand. It calls two new functions we haven't encountered yet. The first one is H5Pset_filter and the second one is H5Pget_edc_check.

Function  H5Pset_filter adds a filter to the filter pipeline, through which data will flow to and from the dataset. The first argument is a property list we're working ont. The second argument is the name of the filter to be added. You can choose one of the following:

H5Z_FILTER_DEFLATE
This is the gzip filter, which we have installed implicitly in the other program by calling H5Pset_deflate.
H5Z_FILTER_SHUFFLE
This is a data shuffling filter. It may help compress data more effectively.
H5Z_FILTER_FLETCHER32
This is the checksum filter we install in this program.
H5Z_FILTER_SZIP
This is a yet another data compression filter that uses the SZIP algorithm.
The third argument is a bit-vector, i.e., an integer, that can be used to specify additional properties of the filter. In the case of this program it is simply 0.

The last argument is an array of integers that can be used to pass additional values to the filter and the next to last argument is the length of this array. We don't have such an array in this program, so we simply put NULL in its place.

In summary our call to H5Pset_filter looks as follows:

   status = H5Pset_filter (property_id, H5Z_FILTER_FLETCHER32, 0, 0, NULL);

The second new function call , H5Pget_edc_check, is used in the reading part of the program in order to find if checksum has been enabled on the dataset. This function is very easy to use. It takes only one argument, the property list itself, and returns a logical value, TRUE for checksum enable and FALSE for checksum disabled.

But also observe another difference. When the property list itself is created, it is created against the  H5P_DATASET_XFER constant - which represents the default for the raw data transfer property list.

This is what this part of the code looks like:

   dxpl = H5Pcreate (H5P_DATASET_XFER);
   edc = H5Pget_edc_check (dxpl);
   if (edc == 1) printf ("Checksum is enabled\n");
   if (edc == 0) printf ("Checksum is disabled\n");


next up previous index
Next: Hyperslab Selection Up: Property Lists Previous: Create a Compressed Dataset
Zdzislaw Meglicki
2004-04-29