Policy: OST is never to be a full database. it is an object dump, without reference control.
=> keep references to a real database whenever data becomes too unrelated.

channel has: (can be overriden)
 * settings for hardware
 * OME SWP
 


===================================================================

<imageset id="im">
	<projectID>...</p>
	<userID>....</>
	<groupID>...</>
	<experimentID></>
	<description>N2: Caenorhabditis elegans wild type for rotation analysis, DIC only</description>


	<-- not in OME, but for our convenience
	<sampleID>N2_071116</sampleID>  
	<rnaiID></rnaiID>
	<entry_date>20.05.2008</>
	
	<tag name="4d"/>  <--- plain tags with no value
	
</imageset>


// imageset should be mirrored into imserv.txt

<channel id="DIC" ostblobid="">

	<compression>100</>

	<hardware name="openlab.objective" prop="magnification">63</a>
	<hardware name="openlab.objective" prop="SerialNumber">...</a>
	<hardware name="openlab.objective" prop="Model">...</a>
	<hardware name="openlab.objective" prop="NA">...</a>
	
	<hardware name="openlab.camera" prop="PixelSize">6.45</a>   // um/px?

	...overridable..., need not be written on all levels

	<resX>15.46</resX>
	<resY>15.46</resY>
	<resZ>2.0</resZ>
	<hardware name="openlab.optivar" prop="magnification">1.6</a>
	<hardware name="openlab.temperature1" prop="temp">21</>    <--- property of some equipment. what is used by micromanager?
	<hardware name="openlab.camera" prop="exposuretime">21</>     <--- micromanager decides EV property name and unit

	OR:
	<hardware name="openlab.temperature1" temp="21"/>    <--- more compact


	<timestamp>1195295075</>

	<frame>
		<slice> ...maybe
			????
			date removed
			????



--- save method so that the acquisition can be repeated. 
--- this would log our data as well
<recordmethod id="j">
	<minexp>
	<maxexp>
	<enabled>
	

========================================

harder to read out exptime but very precise: .getFirstProperty("exposuretime")

 
imserv tags: everything in <imageset>? changing 

custom tags, there is no equivalent in OME. but I thought there was
 
 
 
===================== every hardware 
 
 
Manufacturer=" xsd:string [1]"
ModelName=" xsd:string [1]"
LotNumber=" xsd:string [0..1]"
SerialNumber=" xsd:string [0..1]"


then there should be a database with all hardware



 
 
========== use case

want graph of some property. want to know which properties exist.
can have function for both.

channel menu -> extract variable -> create new object (stack,time,value)


======== general data
data and graph are separate things. graph settings refer to data object.

data providers? overkill?



Manufacturer=" xsd:string [1]"
ModelName=" xsd:string [1]"
LotNumber=" xsd:string [0..1]"
SerialNumber=" xsd:string [0..1]"


=========== our spec



 
<hardware name="stand"
Manufacturer="Zeiss"
ModelName="Axioplan 2ie"
SerialNumber="000403"/>

<hardware name="100x"
Manufacturer="Zeiss"
ModelName="Apochromat 100x/1.4 oil DIC"
ModelNumber="440782-1094-813"    //listed on site:   440782-9902-000
WorkingDistance="0.17"
CoverglassThickness="0.17" 
Immersion="Oil"
Transmittance="600,0.85"
NA="1.4"
Magnification="63"/>

<hardware name="63x"
Manufacturer="Zeiss"
ModelName="Plan-Apochromat 63x/1.4 oil DIC"
ModelNumber="440762-1101-326"    //listed on site: 440762-9904-000   
WorkingDistance="0.19"
CoverglassThickness="0.17" 
ParfocalLength="45.06"
Immersion="Oil"
Transmittance="600,0.85"
NA="1.4"
Magnification="63"/>

<hardware name="40x"
Manufacturer="Zeiss"
ModelName="Plan-neofluar 40x/1.3 oil DIC"
Immersion="Oil"
Transmittance="600,0.85"
NA="1.3"
Magnification="63"/>

<hardware name="20x"
Manufacturer="Zeiss"
ModelName="Plan-neofluar 20x/0.5"
CoverglassThickness="0.17" 
NA="0.5"
Magnification="63"/>

<hardware name="5x"
Manufacturer="Zeiss"
ModelName="Achrostigmat 5x/0.12"
ModelNumber="440120"
Immersion="Oil"
NA="0.12"
Magnification="5"/>

<hardware name="stageController"
Manufacturer="ASI"
ModelName="PZM-2000"
ModelNumber="12VDC-1A-R"
SerialNumber="061206-021"/>

<hardware name="stage"
Manufacturer="ASI"
ModelName="ASI-S1630"/>


<hardware
Manufacturer="Hamamatsu"
ModelName="ORCA II"
ModelNumber="C4442-95-12ER"
SerialNumber="940355"
PixelSize="6.45"
/>

<hardware
Manufacturer="Zeiss"
ModelName="Axiocam color"
ModelNumber="Axiocam color r1.4 412-312"
SerialNumber="210023862"
PixelSize="6.45"
/>

========


unit -> multiple hw

e.g. objective turret -> prism, objective


also, each DIC prism has an ID etc




own config file to deal with advanced properties and 1024-limit in MM.


if multiple device providers, need to start them all first before loading devices. how?
1. load Hardware plugin
2. register HardwareProvider plugins
3. Hardware loads config
how??


=> be happy with separate stacks for now?

* does order of devices matter for MM? maybe not

<micromanager>
 <adapter name="" >
  <device name="" id="mycamera">
   <label num="1" alias="">
    <hardware Manufacturer=.../>
    <hardware Manufacturer=.../>
   </label>
   <set property="" value=""/>
  </device>
 </adapter>
 <synchro>
  <device id=""/>
 </syncho>
</micromanager>



ImageSynchro,ZeissObjectives
ImageSynchro,ZeissShutter

<hardware>

 <device class="endrov.driverMicromanager.">



 <configgroup name="Camera">    <----- should be seen as a device
  <option name="MedRes">
   <set device="ix.camera" property="Binning" value="2"/>
   <set device="ix.camera" property="" value=""/>
  </option>
 </configgroup>
</hardware>

ConfigGroup,Camera,MedRes,Camera,Binning,2

* hardware that doesn't fit anywhere, like stand, can be put in core device or similar 
* order must be stored down. or at least COM ports must go first
* device name as property
* MM name same as EV name, MM can handle dots
* mmstudio is BSD license
* java code exists to load config file! MicroscopeModel.java
* special setup for micromanager


Delay device someTime

Delay settings?
synch settings?
Properties at startup?
COM port settings?
config groups?
MM Roles: won't be any?

!!! !!!Do not rely on database for lookup, just to set up device initially!!! !!!


=========== mm prop blocks

void 	definePropertyBlock (const char *blockName, const char *propertyName, const char *propertyValue)
std::vector< std::string > 	getAvailablePropertyBlocks () const
PropertyBlock 	getPropertyBlockData (const char *blockName) const 

=========== variables and units

* The constant Unknown is rather useless. Rather specify null in this case.
* New constants can be added at need, there is no need for Other
* variable names should be chosen so units do not clash between different types of hardware


ModelName				[string, -]
ModelNumber				[string, -]
Manufacturer			[string, -]
SerialNumber			[string, -]
LotNumber 				[string, -]
WorkingDistance			[mm]
CoverglassThickness		[mm]
Immersion				Oil | Water | WaterDipping | Air | Multi | Glycerol
Transmittance			"{wavelength [nm],fraction,}" // https://www.micro-shop.zeiss.com/us/us_en/objektive.php?cp_sid=&f=od&p[]=440762-9904-000
Emittance				"{wavelength [nm],fraction,}" 
NA						[-]
Magnification			[-]
CalibratedMagnification	[-]
Temperature				[C]
Time					[unixtime, seconds]
Pulse					CW | Single | QSwitched | Repetitive | ModeLocked
Binning					#x#   two numbers 
LaserMedia				GaAs | GaAlAs | NdGlass | NdYAG | ErGlass | ErYAG | HoYLF | HoYAG | Ruby | TiSapphire | Alexandrite 
ArcType					Hg | Xe | HgXe
ContrastMethod			Brightfield | Phase | DIC | HoffmanModulation | ObliqueIllumination | PolarizedLight | Darkfield | Fluorescence
IlluminationType		Transmitted | Epifluorescence | Oblique | NonLinear 
PhotometricInterpretation	Monochrome | RGB | ARGB | CMYK | HSV | ColorMap
DetectionMethod			CCD | IntensifiedCCD | AnalogVideo | PMT | Photodiode | Spectroscopy | LifetimeImaging | CorrelationSpectroscopy | FTIR | EM-CCD | APD | CMOS 
Gain					[-]
DigitalGain				[-]
Voltage					[V]
Filament				Incandescent | Halogen
ParfocalLength			[mm]

use whenever applicable:

{'WideField'|'LaserScanningMicroscopy'|'LaserScanningConfocal'|'SpinningDiskConfocal'|'SlitScanConfocal'|'MultiPhotonMicroscopy'|'StructuredIllumination'|'SingleMoleculeImaging'|'TotalInternalReflection'|'FluorescenceLifetime'|'SpectralImaging'|'FluorescenceCorrelationSpectroscopy'|'NearFieldScanningOpticalMicroscopy'|'SecondHarmonicGenerationImaging'|'Other'}) [0..1]"


Type="list of: [  xsd:string (value comes from list: {'FRAP'|'Photoablation'|'Photoactivation'|'Uncaging'|'OpticalTrapping'|'Other'}) ] [1]"> 

 
 ====== 
 
 rmd.txt -> data/rmd-chan.txt
 
 

 
================================ externalizing XML

<imageset ostext="1">
</imageset>

then all data goes in rmd-1.xml. 
 
One XML-file is 500kb with just basic 4d. 28kb compressed.
Really quick loading: externalize everything. let it be EvCustomObject first that can lazily load
content. hmmm...

<imageset>
 <datablock name="data" ostext="1">
 </datablock>
</imageset>

any object can lazily load content this way. no need to go through Custom!
can mark objects as modified, no need to rewrite content of externalized data.
objects can hint if they should be externalized. or use a metric based on returned Element.

 
==============================================================
UPGRADE STRATEGY
==============================================================
 

1. make sure can write OST 4.0
2. upgrade 3.2 -> 4.0
3. new OSTdaemon 4.0


DONE===
* version 3.3
   => data/ and _ostchildren/ ok
   => resXY has the right unit
   => resxyz, dispxyz stored, override supported
   => no resxyz in imageset
   => no channel inside imageset
   => remove imageset.timestep, slicespacing
   => move NA, objective, optivar, campix, slicespacing, 
   => rename sample, description
   => channel rmd.txt removed
   
TODO
   => one blob for each channel    = this would be enough to solve the flow problem! 
   blob-ch<channelname>


* version 4.0
   => one blob for each channel
   => in file: slice# instead of z-pos  

   
* channel & stack has resXY with the right unit

 
* move rmd.txt in channel directories into data/
   # part of 3.2 -> 4.0
 
 
* OSTdaemon should output image files correctly.
   # Just need to change base directory for slices, name is ok
   
* OSTdaemon, output correct RMD?
   # currently it outputs the very first stoneage format.
   # one file for each channel. copies rmd.txt into each channel directory
   # NEW:
   # copy instead to data/channelRMD-CHANNELNAME
   # write rmd.xml from data/channelRMD-*



* Endrov need to batch-update existing files
   # Next endrov will support OST 3.2 -> 4.0 conversion only
      => rename files
      => temperature as sensor reading
      => exposure as camera setting AND <exposure>
      
      but also choose 
      
      
   # 


# if Channel has no blob, make it blob-CHANNELNAME. 
 
 
 
 ==========================================================
 === new imserv
 
 * improve OST, allow lazy loading also of XML data.
rmd.ostxml should become really small, on the order of 4kb.
use the tricks given above.

=>

eliminate imserv.txt, read the OST instead which should be very fast.
store the stuff in imageset
 
 
 
====== tbu update strategy ============

* make sure everything ost 3.3
* backup RMDs
* eliminate empty channels

* eliminate unwanted tags

openlabtimeout
objfactors
maxtotalstacks
framestart
opvarfactors
openlabautomation
ORSversion
slicenum

* rename tags

description -copy-> preDescription

* later create hardware settings

<objnumber> deleted, use
<hardware objective bla bla bla state="5"> ?

<objnumber> deleted, use
<hardware optivar bla bla bla state="5"> ?

* infer tags?
<strainID></strainID>


* store imserv data in <imageset>
* extend <imageset> to be able to handle tags



entry_date -> tbu_entry_date :: string
author -> authorID
tag -> tag
tag desc -> description. concatenate with current description. 
warn about other tags 
 
 
 
 ?xml version="1.0" encoding="UTF-8"?>
<imserv>
  <tag name="good" />
  <tag name="4d" />
  <tag name="entry_date" value="20.05.2008" />
  <tag name="desc" value="" />
  <tag name="author" value="Yong-Guang Tong" />
</imserv>
 