Error during 'Preview' of .ebsp file

[bschue]

Hello!

I am currently attempting to process a .ebsp file in EMSphinx built from source.

During the 'Preview' process on the first page of the Indexing Wizard I get a vector subscript out of range error. Upon hitting preview.. the process will continue until the very end where the patterns should be displaying.

I have been trying to hunt down the location this particular bug but no luck yet. Any comments on where it could be taking place or what is happening?

I am familiar with the code use and have successfully processed the example data, but with experimental data gathered by our group we get thrown this error.

Thanks for any comments!

Ben

[marcdegraef]

Hi Ben, one immediate question: are the patterns in the .ebsp file compressed?  If they are compressed, then we can not read the .ebsp file; best thing to do is re-export it without compression... If the file is not compressed, then I have to defer to Will to take a look at the code. Regards, Marc.

…

On 8/31/21 1:08 PM, bschue wrote: Hello! I am currently attempting to process a .ebsp file in EMSphinx built from source. During the 'Preview' process on the first page of the Indexing Wizard I get a vector subscript out of range error. Upon hitting preview.. the process will continue until the very end where the patterns should be displaying. Capture <https://user-images.githubusercontent.com/52439071/131545455-1d926df3-0269-4664-a075-19b04e5d4a30.PNG> I have been trying to hunt down the location this particular bug but no luck yet. Any comments on where it could be taking place or what is happening? I am familiar with the code use and have successfully processed the example data, but with experimental data gathered by our group we get thrown this error. Thanks for any comments! Ben — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#12>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB26VWF3OH7NQMWM5INMWA3T7UEBTANCNFSM5DESEQGA>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

-- Marc De Graef Professor Department of Materials Science and Engineering 130 Roberts Engineering Hall Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213-3890 USA Ph: (412) 268-8527 Fx: (412) 268-7596 ***@***.*** web:http://materials.cmu.edu/degraef Admin. Asst.: Marygrace Antkowski Phone: (412) 268-7240

[bschue]

Hi Marc,

Thanks for the really quick reply! I believe this set was uncompressed.. but before I say that for sure I'll gather another dataset and see if I get the same error.

I do have another series of patterns which I exported (an incredible pain storage wise) and will use your Matlab code to generate a .data file.

Let me play around with that and gather another uncompressed .ebsp dataset and report back.

Thanks, Marc!

Ben

[bschue]

Hey @marcdegraef

Finally got back to this issue. I have rescanned a couple datasets with uncompressed patterns. Same issue is arising with the
"vector subscript out of range" when previewing patterns.

Hopefully this isn't too complicated of a fix..

Thanks for all of your help!

Ben

[jwestraadt]

Hi Ben,

I managed to get it working by incorporating this pull request. It looks like the height and width of the patterns is switched around in the histogram equalization code. I think the Ni example data has square patterns, so it is not an issue, but the .ebsp files typically have rectangular patterns where this becomes an issue.

I found another issue with pattern batch sizes when working with .ebsp files before I finally got it working, but will raise it as a separate issue. If I make the batch size equal to a factor of the total number of patterns, then it completes the analysis.

Johan

[bschue]

Hey Johan,

Thank you for the update! I'll incorporate your pull request and see how it plays with my data. I will report back with any findings.

Cheers,
Ben

[bschue]

Hi Ben,

I managed to get it working by incorporating this pull request. It looks like the height and width of the patterns is switched around in the histogram equalization code. I think the Ni example data has square patterns, so it is not an issue, but the .ebsp files typically have rectangular patterns where this becomes an issue.

I found another issue with pattern batch sizes when working with .ebsp files before I finally got it working, but will raise it as a separate issue. If I make the batch size equal to a factor of the total number of patterns, then it completes the analysis.

Johan

Hey Johan,

Solution worked for me but towards the end of analysis I get another error. Is this what you have seen as well?
Cheers,

[jwestraadt]

Hi Ben,

I have created another issue for this error: #13

You have 255x196 = 44100 patterns in the .ebsp file. Under "Indexing parameters" on the right, try setting the "batchsize" to say 100, which is a factor of the total number of patterns. At the moment it is set to 0, which will automatically determine the batchsize. This number might not be a factor of the total number of patterns. Some initial debugging shows that this error might be related to reading the last partial batch from the .ebsp in the following loop of the index_ebsd.cpp file.

	size_t batches = idxData.pat->numPat() / nml.batchSize;//how many batches are needed
	if(batches * nml.batchSize < idxData.pat->numPat()) ++batches;//extra batch for leftovers
	for(size_t i = 0; i < batches; i++) {//loop over batches
		const size_t start = i * nml.batchSize;//first pattern
		const size_t end = std::min(start + nml.batchSize, idxData.pat->numPat());//last pattern
		pool.schedule(std::bind(idxData.workItem, std::placeholders::_1));//queue indexing
	}

Please also make sure to output the vendor file as ".ang" and not ".ctf" as it will give another error. I have logged this as a separate issue #14. You might also find the discussions on the pattern center determination on the Oxford system useful.