Endrov image data internals

From Endrov

Image data is supported by the Imageset plugin. These are the interesting classes:

  • Imageset - Endrov object, top-level type, contains information about the recording and all the channels (EvChannel)
  • EvChannel - Contains information about one recorded channel, such as GFP. Also contains all images (EvImage) for the channel.
  • EvImagePlane - An image with resolution, binning, displacement etc. Keeps track of pixel data sources.
  • EvPixel - Pixel data only. Does all pixel data conversion.

Possible future class: EvStack, an optimization. makes data generation even lazier

Image data I/O

Endrov is designed all the way up to work with large datasets. Images are loaded only when they are needed, can be cached, and can be swapped out (Technical note: By having Endrov swapping them instead of the operating system, the address space is not exhausted and 32-bit is perfectly enough for arbitrarily large imagesets). There are also "shadow images", images that point to other images for data. Whenever these are modified the data will first be copied (copy-on-write) to keep the original unchanged. Copying images is hence dirt-cheap and encourages that plugins rather copy entire imagesets and make small modifications instead of writing modifications into the original data. All these mechanisms are hidden in EvImage and normal programmers need only set the modified-flag whenever data is changed.

Those implementing new dataformat plugins need to implement EvIOImage. EvImage will ask this class for data when the user requests it. Care has been taken so that dataformat plugins can ignore shadow images, caching and swapping, it is done on a higher level.

Lazy filter evaluation

EvIOImage need not read from disk - it can also be an image processing operation. Applying even the simplest filter to 50Gb images takes an awful lot of time and you can surely not see the change in real-time. Endrov solves this by mandating that image data is generated lazily whenever possible i.e. the operation is made an EvIOImage and postponed until the data is actually needed. This way a contrast-brightness adjustment will only be applied to the shown image at first, and it will run in real-time.

Pixel data

One goal has been to make pixel data conversion automatic on a high level. This means users need not spend effort on it, and plugin writers can ignore that several formats exist. If new formats are added in the future then no plugins has to be modified (64-bit integers are considered). Plugins has to specify legal input and possible output, and Endrov will do the conversion on-the-fly in-between plugins. EvPixel handles the conversion, and also has optimizations to avoid data copying for read-only images.

When data conversion is needed, Endrov plays safe when possible to make sure data is not lost. It will move along this axis:

  • unsigned 8-bit integer -> signed 16-bit integer -> signed 32-bit integer -> float -> double

but never further than needed to converse space. If going to the right is impossible, it will move to the left - this means potential data loss and the user has to make sure the data is within range himself; this is however the best thing Endrov can do in this case. Hidden is the AWT type which is useful for loading images, but is never used for processing and always converted away from.

Unsigned integers, except 8-bit, are removed in Endrov. The reasons are that you almost always want signs when processing images, and Java has really bad support for unsigned integers.

EvPixel stores data in 1D scalar arrays which can be obtained by getArray*(). Java does not support proper 2D arrays so this is needed to achieve the highest speed. The pixel position within this array is width*y+x, which can also be obtained by getPixelIndex(x,y). This is a rather expensive call and calculation, plugin writers are encouraged to only do the multiplication once for each line, and then do only an addition for each pixel. Some operations do not use the x,y-position, by all means just iterate over the 1D array in this case.