expand JEOL plugin to read pts files from NMap

[ben-buse]

Describe the functionality you would like to see.

Make JEOL read plugin read pts files from NMap

Describe the context

JEOL EPMA uses NMap for EDS data cube, the EDS files are stored as .apb .pts .emsa and .sp3. The current JEOL plugin fails to read the Pts file

Additional information

screen shot

[ben-buse]

Files

[ben-buse]

if try to load in hyperspy console the pts file

[ben-buse]

File here https://uob-my.sharepoint.com/:u:/g/personal/glxbb_bristol_ac_uk/IQDN7VnHXcDtR7RK3JNb3lJdAZm_rF1J1FRYSYBpJwX64xI?e=7BNAWS

[ericpre]

Moving this issue to https://github.com/hyperspy/rosettasciio because this is where the IO now lives.

[ericpre]

@ben-buse, from a quick look, it seems that this data contains metadata which are not supported. This would typically needs someone who can access the software to figure out what is the difference with other files, etc.
Is it something that occurs with all files or only this one? Sometime, it can be that some processing add some metadata, etc.

[CSSFrancis]

I tried just wrapping the failing lines with a try/except and added some warnings to see if I could at least get the data to load. It seems like reading the rois is what causing the failure. We might be able to load those given a bit more information on how those are stored?

@ben-buse Can you maybe consider reaching out to JEOL as well? It usually helps significantly if we have additional information from them on decoding file formats. And they tend to be a bit more responsive to customer requests.

WARNING | RosettaSciIO | Error 'numpy.uint16' object does not support item assignment occurred when assigning value [0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] to keyword 'Roi' in JEOL file. The value is skipped. (rsciio.jeol._api:704)
WARNING | RosettaSciIO | Keyword 'Roi' is not found in the current dictionary. This keyword is skipped. (rsciio.jeol._api:719)
WARNING | RosettaSciIO | Error 'numpy.uint16' object does not support item assignment occurred when assigning value [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] to keyword 'RoiType' in JEOL file. The value is skipped. (rsciio.jeol._api:704)
WARNING | RosettaSciIO | Keyword 'RoiType' is not found in the current dictionary. This keyword is skipped. (rsciio.jeol._api:719)
WARNING | RosettaSciIO | Error 'numpy.uint16' object does not support item assignment occurred when assigning value [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1] to keyword 'RoiLine' in JEOL file. The value is skipped. (rsciio.jeol._api:704)
WARNING | RosettaSciIO | Keyword 'RoiLine' is not found in the current dictionary. This keyword is skipped. (rsciio.jeol._api:719)
WARNING | RosettaSciIO | Error 'numpy.uint16' object does not support item assignment occurred when assigning value [0 0 0 0 0 0 0 0] to keyword 'RoiUser' in JEOL file. The value is skipped. (rsciio.jeol._api:704)
WARNING | RosettaSciIO | Keyword 'RoiUser' is not found in the current dictionary. This keyword is skipped. (rsciio.jeol._api:719)
WARNING | RosettaSciIO | Error 'numpy.uint16' object does not support item assignment occurred when assigning value [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0] to keyword 'RoiWeight' in JEOL file. The value is skipped. (rsciio.jeol._api:704)
WARNING | RosettaSciIO | Keyword 'RoiWeight' is not found in the current dictionary. This keyword is skipped. (rsciio.jeol._api:719)
WARNING | RosettaSciIO | Error 'str' object does not support item assignment occurred when assigning value 20.0 to keyword 'AccKV' in JEOL file. The value is skipped. (rsciio.jeol._api:704)
WARNING | RosettaSciIO | Keyword 'AccKV' is not found in the current dictionary. This keyword is skipped. (rsciio.jeol._api:719)
WARNING | RosettaSciIO | index failed (rsciio.jeol._api:728)
WARNING | RosettaSciIO | Error 'str' object does not support item assignment occurred when assigning value 9.999999717180685e-10 to keyword 'AccNA' in JEOL file. The value is skipped. (rsciio.jeol._api:704)
WARNING | RosettaSciIO | Keyword 'AccNA' is not found in the current dictionary. This keyword is skipped. (rsciio.jeol._api:719)
WARNING | RosettaSciIO | index failed (rsciio.jeol._api:728)
WARNING | RosettaSciIO | Error 'str' object does not support item assignment occurred when assigning value 2300 to keyword 'Mag' in JEOL file. The value is skipped. (rsciio.jeol._api:704)
WARNING | RosettaSciIO | Keyword 'Mag' is not found in the current dictionary. This keyword is skipped. (rsciio.jeol._api:719)
WARNING | RosettaSciIO | index failed (rsciio.jeol._api:728)
WARNING | RosettaSciIO | Error 'str' object does not support item assignment occurred when assigning value 10.996960639953613 to keyword 'WD' in JEOL file. The value is skipped. (rsciio.jeol._api:704)
WARNING | RosettaSciIO | Keyword 'WD' is not found in the current dictionary. This keyword is skipped. (rsciio.jeol._api:719)
WARNING | RosettaSciIO | Error 'str' object does not support item assignment occurred when assigning value 20.0 to keyword 'AccKV' in JEOL file. The value is skipped. (rsciio.jeol._api:704)
WARNING | RosettaSciIO | Keyword 'AccKV' is not found in the current dictionary. This keyword is skipped. (rsciio.jeol._api:719)
WARNING | RosettaSciIO | index failed (rsciio.jeol._api:728)
WARNING | RosettaSciIO | Error 'str' object does not support item assignment occurred when assigning value 9.999999717180685e-10 to keyword 'AccNA' in JEOL file. The value is skipped. (rsciio.jeol._api:704)
WARNING | RosettaSciIO | Keyword 'AccNA' is not found in the current dictionary. This keyword is skipped. (rsciio.jeol._api:719)
WARNING | RosettaSciIO | index failed (rsciio.jeol._api:728)
WARNING | RosettaSciIO | Error 'str' object does not support item assignment occurred when assigning value 2300 to keyword 'Mag' in JEOL file. The value is skipped. (rsciio.jeol._api:704)
WARNING | RosettaSciIO | Keyword 'Mag' is not found in the current dictionary. This keyword is skipped. (rsciio.jeol._api:719)
WARNING | RosettaSciIO | index failed (rsciio.jeol._api:728)
WARNING | RosettaSciIO | Error 'str' object does not support item assignment occurred when assigning value 10.996960639953613 to keyword 'WD' in JEOL file. The value is skipped. (rsciio.jeol._api:704)
WARNING | RosettaSciIO | Keyword 'WD' is not found in the current dictionary. This keyword is skipped. (rsciio.jeol._api:719)

[ben-buse]

@CSSFrancis Thanks, I will ask JEOL, but not hopeful

[ben-buse]

@ben-buse, from a quick look, it seems that this data contains metadata which are not supported. This would typically needs someone who can access the software to figure out what is the difference with other files, etc. Is it something that occurs with all files or only this one? Sometime, it can be that some processing add some metadata, etc.

@ericpre If it helps here are two other files Pos0001 https://uob-my.sharepoint.com/:u:/g/personal/glxbb_bristol_ac_uk/IQDnOod8-aosTJhbXSw2IQqyAePhsKqA9vihxVqZdlB-FE4?e=pCUOgP and Pos0002 https://uob-my.sharepoint.com/:u:/g/personal/glxbb_bristol_ac_uk/IQAze4IkLIwVQ4a_IgmdmUImAaAIhz0BdBHuOzqQ6p3uJBI?e=KPXBwI

which correspond to

and

[Yuji-Tan]

I guess "roi" represents atomic number to be shown in figure.
Index 12, 14, 26 has a value 1 and these indices should be Mg, Si, and Fe.