From email to Daniel Huff:
Starting from deepMedic, I know that version v0.7.4 that we use for segmentation runs with tensorflow==1.15.5 which in turn is fixed to CUDA-10.0 and cudnn-7.6 which don't even support ampere GPU architecture embedded in A5000.
So, deepMedic will not work on a GPU, but might run on CPU. The attached file shows python virtualenv layout used on vangogh.
Python-3.6.14 was used on vangogh. To get it, download it from python web page, do the install. Virtualenv is created and software installed via:
cd software/install/Python-3.6.14
bin/python -m ensurepip --default-pip
bin/python -m pip install --upgrade pip
./python -m venv ~/venv/segDM
. ~/venv/segDM/bin/activate
pip install --upgrade pip
#comment out deepmedic and pkg-resources lines with #
pip install -r requirementsDM.txt
Install deepmedic (still in venv):
cd ~/software/src
git clone https://github.com/Kamnitsask/deepmedic/
cd deepmedic
git checkout tags/v0.7.4 -b branch_v0.7.4
pip install .
Now deepmedic should work:
deepMedicRun -h
Open setup.json and add
"deepMedicRun":"/home/nixWorker/venv/segDM/bin/deepMedicRun"
Sometimes user is not nixWorker, so pay attention.
For parameter file, a good template is available at irAEMM git repository. Upload to configuration and add to parameterFile list.
It is safe to use iraemm virtual env, so standard script path entry to scripts list would be:
iraemm:_softwareSrc_/irAEMM/pythonScripts/runSegmentationDM.py
Although, runSegmentationDM.py is not so picky in terms of additional python libraries and venv could be better tailored, should it be needed.
In formating the Runs entry, from obviously using the configured parameterFile and scripts list entries, runType should be python and server should point to the server used in the install step (works on rubens, not done on vangogh since most of the data is at onko-nix.
There is the parameterOverload field to play with, particularly in setting entryFilter values. Here, one should specify both the name of the configuration file, the variable and the value, which should be JSON formatted, including (double) quote marks. For filtering on patientCode and visitCode, the entry in parameter overload should be:
segmentationIRAEMM_ONKO_retro_DM.js:entryFilter=[{"variable":"patientCode","value":"1000_12","oper":"eq"},
{"variable":"visitCode","value":"VISIT_0","oper":"eq"}]
For filtering on lists using in filter, see here, one would have to use semicolons to separate values in filter. Since semicolon is already a marker for spliting variables in parameterOverload, runSegmentationDM.py uses parseConfig to replace __semiColon__ later on, so use this in place of semicolon within variables, like:
segmentationIRAEMM_ONKO_retro_DM.js:entryFilter=[{"variable":"patientCode","value":"1000_12","oper":"eq"},
{"variable":"visitCode","value":"VISIT_0__semiColon__VISIT_1","oper":"in"}]
Sometimes, analysisModule fails to find appropriate binaries to run the code. Check in Server Javascript Console where exact shell command is reported, and update externalModules/analysisModules/scripts/analysisModule/analysisFcns.js. The strategy was somewhat robustified, but some servers might still use different paths for invoking python on the server. A log of the transaction is recorded in ~tomcat8/logs/analysisModule/runRef.log, where runRef is assigned and recorded in the runs list.
Analysis interface connects the analysisModule and the underlying python script by running:
~/software/src/analysisInterface/runAnalysis.py CALLER ID
It is run both when a local job is submitted as well as when the remote job is started, ensuring symmetry in job execution. In remote mode, output of runAnalysis is reported by the websocket server in ~/logs/socketServer.log. To debug, above command can be run by the same user that is running websocket, typically nixWorker, where CALLER is an abbreviation of the labkey server that emitted the request and ID is the Key of the line in runs list.
~/software/src/analysisInterface/runAnalysis.py onko-nix 659
Analysis interface will prepare ~/analysis/runRef directory where segmentation file will be copied and runRef.log will show output of the underlying script, which is later copied back to calling server and available under log link.
One can execute runSegmentationDM.py directly on the server,
~/venv/nnUNet/bin/python ~/software/src/irAEMM/pythonScripts/runSegmentationDM.py ~/analysis/runRef/configuration.json
where nnUNet is a suggested, but by far not the only virtual environment (see above), and runRef should be replaced by a valid runRef from the runs list. This is only a help to get a valid configuration.json file, where configuration should be replaced by the actual name of the file (see parameterFile above). One can edit both the py script and configuration file until the code runs appropriately. In this case, the output is directly in the terminal.
| Attached Files | ||
requirementsDM.txt
