Tutorial 3 - Accessing cube data

Files:

This tutorial demonstrates an important part of most Opticks plug-ins, accessing data in a raster cube. We'll build a plug-in which calculates some basic statistics over band 1 of a data cube. We'll see how to use DataAccessor and switchOnEncoding.h.

If you would like to run this tutorial as you follow along with the code, see Running Tutorials With Opticks. If you would like to build and experiment with this tutorial as you follow along with the code, see Building and Running Tutorial plug-ins. If you are running this tutorial in Opticks, you will need to select "Tutorial 3" from the Tutorial toolbar to execute it.

namespace
{
   template<typename T>
   void updateStatistics(T* pData, double& min, double& max, double& total)
   {
      min = std::min(min, static_cast<double>(*pData));
      max = std::max(max, static_cast<double>(*pData));
      total += *pData;
   }
};

This defines a functor which will be used later in the plug-in. It updates some statistical data based on a pixel value in the cube.

   setAbortSupported(true);

This allows the user to abort the plug-in. We'll see how to properly handle aborts below.

   pInArgList->addArg<RasterElement>(Executable::DataElementArg(), "Generate statistics for this raster element");

This adds a DataElement argument. RasterElement is a sub-class of DataElement. Opticks will allow any sub-class of DataElement in the Executable::DataElementArg() argument.

bool Tutorial3::getOutputSpecification(PlugInArgList*& pOutArgList)
{
   pOutArgList = Service<PlugInManagerServices>()->getPlugInArgList();
   VERIFY(pOutArgList != NULL);
   pOutArgList->addArg<double>("Minimum", "The minimum value");
   pOutArgList->addArg<double>("Maximum", "The maximum value");
   pOutArgList->addArg<unsigned int>("Count", "The number of pixels");
   pOutArgList->addArg<double>("Mean", "The average value");
   return true;
}

We are specifying output arguments in this tutorial. These are useful if you want to chain plug-ins together in a wizard and use the output of one plug-in as the input to another plug-in. We are using doubles for most of these values so we can store statistics for any non-complex data encoding.

   RasterDataDescriptor* pDesc = static_cast<RasterDataDescriptor*>(pCube->getDataDescriptor());

Each DataElement has a DataDescriptor which store information about the DataElement. A RasterElement will always have a RasterDataDescriptor so we can safely use static_cast instead of dynamic_cast.

   FactoryResource<DataRequest> pRequest;
   DataAccessor pAcc = pCube->getDataAccessor(pRequest.release());

Here we are creating a DataRequest object. A DataRequest sets the parameters for the cube access such as the rows, columns, and bands we need access to and how many of each we will need at any particular time. This provides hints to the data importers about access patterns so Opticks can load only the data needed at a particular time. We'll talk more about importers and pagers in Creating a raster data importer plug-in. The DataRequest reference documentation explains what the defaults are for the various attributes. This code requests an accessor over the entire first band of the data cube in BSQ format. We are only calculating statistics on the first band in order to simplify the plug-in.

   double min = std::numeric_limits<double>::max();
   double max = -min;
   double total = 0.0;
   for (unsigned int row = 0; row < pDesc->getRowCount(); ++row)

We initialize various statistics accumulators and begin accessing data over all the rows.

      if (isAborted())
      {
         std::string msg = getName() + " has been aborted.";
         pStep->finalize(Message::Abort, msg);
         if (pProgress != NULL)
         {
            pProgress->updateProgress(msg, 0, ABORT);
         }

         return false;
      }

This block of code checks to see if the plug-in was aborted. This might be because the user clicked the cancel button or another plug-in may have called Executable::abort().

      if (!pAcc.isValid())
      {
         std::string msg = "Unable to access the cube data.";
         pStep->finalize(Message::Failure, msg);
         if (pProgress != NULL)
         {
            pProgress->updateProgress(msg, 0, ERRORS);
         }

         return false;
      }

This code checks to make sure the DataAccessor is still valid. If the DataRequest could not be provided, the DataAccessor will be invalid. If DataAccessorImpl::nextRow() attempts to move past the end of the requested block of data, the DataAccessor will become invalid.

      if (pProgress != NULL)
      {
         pProgress->updateProgress("Calculating statistics", row * 100 / pDesc->getRowCount(), NORMAL);
      }

      for (unsigned int col = 0; col < pDesc->getColumnCount(); ++col)
      {
         switchOnEncoding(pDesc->getDataType(), updateStatistics, pAcc->getColumn(), min, max, total);
         pAcc->nextColumn();
      }

This inner loop processes all the columns in the current row. In the DataRequest we specified that we would be accessing one row and all columns at once so DataAccessorImpl::getColumn() will return a pointer to data from the current column to the end of the current row. We could have called DataAccessorImpl::getRow() to get a pointer to the entire row then called updateStatistics() with that pointer. We'd need to put the column loop inside updateStatistics(). The method presented is usually preferable because we could have requested a DataAccessor with only a single pixel (1 concurrent row and 1 concurrent column) and the code above would still work. This would allow a pager to load less data into memory if necessary.

The switchOnEncoding() function takes an EncodingType and cases the void* returned from DataAccessorImpl::getColumn() into the appropriate data type for the specified EncodingType. It then calls the updateStatistics() function with the remaining arguments. This is a common pattern for accessing data in a raster cube. Finally, DataAccessorImpl::nextColumn() moves the DataAccessor's internal pointer to the next column.

      pAcc->nextRow();
   }
   unsigned int count = pDesc->getColumnCount() * pDesc->getRowCount();
   double mean = total / count;

Here we move the DataAccessor's internal pointer to the next row completing the outer loop. After we have processed the entire cube we calculate the mean.

   if (pProgress != NULL)
   {
      std::string msg = "Minimum value: " + StringUtilities::toDisplayString(min) + "\n"
                      + "Maximum value: " + StringUtilities::toDisplayString(max) + "\n"
                      + "Number of pixels: " + StringUtilities::toDisplayString(count) + "\n"
                      + "Average: " + StringUtilities::toDisplayString(mean);
      pProgress->updateProgress(msg, 100, NORMAL);
   }
   pStep->addProperty("Minimum", min);
   pStep->addProperty("Maximum", max);
   pStep->addProperty("Count", count);
   pStep->addProperty("Mean", mean);
   
   pOutArgList->setPlugInArgValue("Minimum", &min);
   pOutArgList->setPlugInArgValue("Maximum", &max);
   pOutArgList->setPlugInArgValue("Count", &count);
   pOutArgList->setPlugInArgValue("Mean", &mean);

   pStep->finalize();
   return true;

The remainder of the execute() method reports the statistical information to the user in the Progress if available, in the message log, and in the output arguments.