SMILA/Documentation/Importing/CompoundExtractorService

CompoundExtractor Service

Interface: org.eclipse.smila.importing.CompoundExtractor

A CompoundExtractor service provides two kinds of methods:

• check if an object's filename, URL or mimetype idenfifies it as a compound object that can be extracted by the service.
• extract the compound: Given an InputStream with the compound content produce records for the elements.

The element records can contain the following attributes:

• fileName: the complete name of the entry in the compound object, usually something like a filesystem path
• isCompound: set to true if the element is a supported compound object itself.
• size: uncompressed size of the element
• time: last modification timestamp, as a datetime value.
• compressedSize: compressed size of the element
• comment: a comment for the element in the compound
• isRootCompound: set to true if the record describes the processed compound object itself.
• compounds: a sequence of the compound files to look into to reach this element. For example, if the compound /data/compound.zip contains a file archived/subcompound.zip which contain a file x.html, the compounds list for x.html would be:

  [/data/compound.zip, archived/subcompound.zip]

Not all attributes need to be set for all compound types.

Implementations

SimpleCompoundExtractorService

Bundle: org.eclipse.smila.importing.compounds.simple

This extractor service uses the classes provided by the JDK's java.util.zip package to extract compound objects. This means that it can currently support ZIP files and GZ files (not TAR.GZ, though).

Supported Mimetypes:

• application/zip
• application/x-gunzip
• application/x-gzip

If the mimetype is not provided by the caller at all or it is only application/octet-stream it uses the current MimeType Identifier service to recognize the real mimetype from the filename extension.

The compound types are treated differently:

• For ZIP files, it creates one record for the ZIP file itself and one record for each contained element.
• For GZ files, it creates one record for the compressed file with the original filename of the GZ file but no content, and one record for the content of the uncompressed file.

Configuration

The simple extractor service can be configured by means of a properties file configuration/org.eclipse.smila.importing.compounds.simple/extractor.properties.

The configuration properties are as follows:

• zip.encoding the encoding to use when extracting ZIP files that do not use UTF-8 (default: UTF-8)
  • example: zip.encoding=CP850
  • This property will only yield an effect when SMILA is run on a JRE prior to JRE 7. Since JRE 7 using non-UTF-8 ZIPs with special characters will almost always throw an IllegalArgumentException, since JRE 7's zip solution does not honor the properties used by the previous solution, but software written for java prior to java 7 has no other means on configuring another code page to use for such zip files.
• tmp.dir the temporary directory to extract compounds to, per default a directory named org.eclipse.smila.importing.compounds.simple is created in the user's temporary folder (e.g. on Windows 7 something like C:\Users\<username>\AppData\Local\Temp\org.eclipse.smila.importing.compounds.simple).
  • example: tmp.dir=/temp/SMILA.compound.extractor/

CommonsCompressCompoundExtractorService

Bundle: org.eclipse.smila.importing.compounds.compress

Supported Compound Formats:

• Archives
  • zip
  • tar
  • cpio
  • java-archive
• Compressions
  • bzip2
  • gzip

Configuration

The simple extractor service can be configured by means of a properties file configuration/org.eclipse.smila.importing.compounds.compress/extractor.properties.

The configuration properties are as follows:

• zip.encoding the encoding to use when extracting ZIP files that do not use UTF-8 (default: UTF-8)
  • example: zip.encoding=CP850
  • This property will only yield an effect when SMILA is run on a JRE prior to JRE 7. Since JRE 7 using non-UTF-8 ZIPs with special characters will almost always throw an IllegalArgumentException, since JRE 7's zip solution does not honor the properties used by the previous solution, but software written for java prior to java 7 has no other means on configuring another code page to use for such zip files.
• tmp.dir the temporary directory to extract compounds to, per default a directory named org.eclipse.smila.importing.compounds.compress is created in the user's temporary folder (e.g. on Windows 7 something like C:\Users\<username>\AppData\Local\Temp\org.eclipse.smila.importing.compounds.compress).
  • example: tmp.dir=/temp/SMILA.compound.extractor/