Creating a raster data importer plug-in

Files:

This page describes the basic RasterElement importer provided in the plug-in sampler. This is a basic importer which simulates loading from a file and provides a fixed 1 band data array. The header file contains declarations for three classes.

class SampleRasterPage : public RasterPage
class SampleRasterPager : public RasterPager
class SampleRasterElementImporter : public RasterElementImporterShell

The first two classes are not needed when the data portion of the file format is "raw" and can be imported with the generic file importer. If the file format is not compatible with the generic importer, a compressed format for example, you may need to write a custom RasterPager. These classes should be declared in separate header files but have been included here so the sample code is all in one location. RasterPagerShell is often used when writing a custom RasterPager. It is not used here so we can isolate the RasterPager code from the plug-in code which is normally required by RasterPagerShell.

Now we'll exam the implementation of these classes.

#define NUM_ROWS 10
#define NUM_COLS 5

SampleRasterPage::SampleRasterPage(int row, int col) :
   mRow(row),
   mCol(col)
{
   for (unsigned char r = 0; r < NUM_ROWS; ++r)
   {
      for (unsigned char c = 0; c < NUM_COLS; ++c)
      {
         mData.push_back(r * NUM_COLS + c);
      }
   }
}

SampleRasterPage::~SampleRasterPage()
{
}

void* SampleRasterPage::getRawData()
{
   return reinterpret_cast<void*>(&mData[mRow * NUM_COLS + mCol]);
}

unsigned int SampleRasterPage::getNumRows()
{
   return NUM_ROWS - mRow;
}

unsigned int SampleRasterPage::getNumColumns()
{
   return NUM_COLS;
}

unsigned int SampleRasterPage::getNumBands()
{
   return 1;
}

unsigned int SampleRasterPage::getInterlineBytes()
{
   return 0;
}
// End of SampleRasterPage

A RasterPage is the basic unit of data for a RasterElement. This can contain only the requested data but it may contain more than the requested data when this is more efficient, for example when an entire band must be decompressed it is often more efficient to create a RasterPage for the entire band than it is to decompress the band multiple times.

SampleRasterPager::SampleRasterPager()
{
}

SampleRasterPager::~SampleRasterPager()
{
}

RasterPage* SampleRasterPager::getPage(DataRequest* pOriginalRequest,
                                       DimensionDescriptor startRow,
                                       DimensionDescriptor startColumn,
                                       DimensionDescriptor startBand)
{
   return new SampleRasterPage(startRow.getOriginalNumber(), startColumn.getOriginalNumber());
}

void SampleRasterPager::releasePage(RasterPage* pPage)
{
   delete dynamic_cast<SampleRasterPage*>(pPage);
}

int SampleRasterPager::getSupportedRequestVersion() const
{
   return 1;
}
// End of SampleRasterPager

A RasterPager is a factory object for RasterPage. The RasterPager::getPage() method returns a RasterPage for a specific data request. The RasterPage::getRawData() method must return a pointer to the specified offset but may return additional data. See the RasterPage and RasterPager documentation for further details.

REGISTER_PLUGIN_BASIC(OpticksPlugInSampler, SampleRasterElementImporter);

SampleRasterElementImporter::SampleRasterElementImporter()
{
   setName("Sample RasterElement Importer");
   setCreator("Opticks Community");
   setVersion("Sample");
   setCopyright("Copyright (C) 2008, Ball Aerospace & Technologies Corp.");
   setExtensions("Sample Files (*.sample)");
   setDescription("Import sample data. It's really a static in-memory array that simulates an importer.");
   setSubtype("Sample");
   setDescriptorId("{D1EBFE37-E365-4E49-92B5-B8DE0CA0B03D}");

The importer plug-in should call ImporterShell::setExtensions() to declare the file extensions typical for the file format. The user will always have an "All Files (*)" option so this is only a convenience filter.

unsigned char SampleRasterElementImporter::getFileAffinity(const std::string& filename)
{
   FactoryResource<Filename> pFilename;
   pFilename->setFullPathAndName(filename);
   if (pFilename->getExtension() == "sample")
   {
      return Importer::CAN_LOAD;
   }
   return Importer::CAN_NOT_LOAD;
} // End getFileAffinity

The Importer::getFileAffinity() method is used by the auto importer to decide which importer should be used to load a specific file. The most basic getFileAffinity() implementation calls getImportDescriptors(). If the resulting vector is empty, it returns Importer::CAN_NOT_LOAD and if the vector is not empty, it returns Importer::CAN_LOAD.

The getImportDescriptors() method may be time consuming. If this is the case, getFileAffinity() should use another, faster method and make a best guess about the file. This method is not called if the importer is explicitly selected.

bool SampleRasterElementImporter::isProcessingLocationSupported(ProcessingLocation location) const
{
   return location == IN_MEMORY;
} // End isProcessingLocationSupported

This method is normally not re-implemented. The default indicates that all processing locations are supported. The sample plug-in indicates that only in-memory processing is supported. The reason is described in more detail at the end of this howto.

std::vector<ImportDescriptor*> SampleRasterElementImporter::getImportDescriptors(const std::string& filename)
{
   std::vector<ImportDescriptor*> descriptors;
   ImportDescriptorResource pDesc(filename, TypeConverter::toString<RasterElement>());
   VERIFYRV(pDesc.get() != NULL, descriptors);
   RasterDataDescriptor* pDataDesc = RasterUtilities::generateRasterDataDescriptor(filename, NULL, NUM_ROWS,
      NUM_COLS, INT1UBYTE, IN_MEMORY);
   if (pDataDesc == NULL)
   {
      return descriptors;
   }
   pDesc->setDataDescriptor(pDataDesc);
   if (RasterUtilities::generateAndSetFileDescriptor(pDataDesc, filename, "", LITTLE_ENDIAN_ORDER) == NULL)
   {
      return descriptors;
   }
   descriptors.push_back(pDesc.release());
   return descriptors;
} // End getImportDescriptors

The getImportDescriptors() is the most important method an importer implements. It creates a DataDescriptor and surrounding ImportDescriptor for all data sets in the specified file. Each data set must also have a FileDescriptor. This sample shows a typical sequence.

An ImportDescriptorResource is created and populated with a DataDescriptor. RasterUtilities contains a number of functions which are usually used to create the DataDescriptor and FileDescriptor for a raster data set. If a file format's data can be loaded with the generic importer, all of the relevant attributes should be set on the FileDescriptor to indicate the header bytes, inter-line bytes, etc. This information is used by the generic RasterPager to map the data from disk to memory. The metadata for the data set should also be set in this method. A DynamicObject is created using a FactoryResource and the appropriate fields are populated. Call DataDescriptor::setMetadata() to assign the DynamicObject as the data set's metadata.

Often a file will contain multiple data sets which need to be created in a hierarchy. If a parent DataElement is also a part of the same file, it won't be available to set as the parent when creating a DataDescriptor. If this is the case, the ImportDescriptorResource contructor which takes a vector of strings should be used. The strings in the vector declare a parent path where each string is the name of a parent in the path. This is used to define a hierarchy before all the elements in the hierarchy have been created. The entire hierarchy must either exist before the importer is called or must be created by the importer.

bool SampleRasterElementImporter::createRasterPager(RasterElement* pRaster) const
{
   VERIFY(pRaster != NULL);

   SampleRasterPager* pPager = new SampleRasterPager();
   return pRaster->setPager(pPager);
}

If the importer is using the generic RasterPager, this method should not be overwritten. If an importer is using a custom RasterPager, this method creates the RasterPager. Often, a custom RasterPager uses RasterPagerShell and is a plug-in. createRasterPager() will use an ExecutableResource() to create the RasterPager. If a RasterPager is a plug-in and inherits from RasterPagerShell, it will not be destroyed after execute(). This forces the module containing the RasterPager code to remain in memory. If the module does not remain in memory, the application will crash when accessing the RasterElement data. In this sample, we are not implementing RasterPager as a plug-in so that the code is easier to follow. This is not a problem in this sample because it can not load data on-disk. The RasterElementImporterShell will create the pager and use it to copy the data into memory. The pager is then destroyed before execute() returns.