Suitable reflector lists

[hakonanes]

Hi @drowenhorst-nrl!

We just released kikuchipy v0.9.0 compatible with PyEBSDIndex v0.2.0. So far, the new particle swarm optimization works like a charm.

This is more of a question than an issue. It relates to how users of PyEBSDIndex should find suitable reflector lists for indexing. My suggestion is to use the diffsims and orix libraries, specifically the ReciprocalLatticeVector class, which needs a Phase instance. At least for the moment.

So, I've tested multi-phase indexing with three phases in the list:

• Aluminium (Laue class m-3m, blue)
• Alpha-Al(Fe,Mn)Si (m-3, green)
• Pi-Al8FeMg3Si6 (-6m2, purple)

It also works like a charm!

But, only after I specified my own reflector lists or polefamilies. I found the following reflectors to correctly differentiate between the phases above and provide plausible indexing:

• Aluminium: the default ones, {111}, {200}, {220}, {311}
• Pi-Al8FeMg3Si6: {2-12}, {004}, {002}, {4-20}, {2-20}, {200}
• Alpha-Al(Fe,Mn)Si: {600}, {523}, {530}, {835}, {10 0 0}

These reflectors were found automatically using ReciprocalLatticeVector. I've found the approach quite robust and to give results comparable to EMsoft's building of reflector lists (which makes sense since it is based on that implementation). The procedure is simple:

1. Get all reflectors with a minimum interplanar spacing d
2. Exclude kinematically forbidden reflectors (not implemented for hexagonal yet)
3. Exclude reflectors with a kinematical structure factor lower than a certain fraction of the maximum structure factor

Minimal syntax (CIF file: pi_al8femg3si6.cif.txt):

from diffsims.crystallography import ReciprocalLatticeVector
from orix.crystal_map import Phase

phase = Phase.from_cif("pi_al8femg3si6.cif")

ref = ReciprocalLatticeVector.from_min_dspacing(phase, 1)
ref.calculate_structure_factor()  # Parametrizations from Lobato and van Dyck (2014) or International Tables for Crystallography
F = abs(ref.structure_factor)
ref2 = ref[F > 0.48 * F.max()]

Do you have any thoughts on how users should find suitable reflector lists? Do you think linking to a relevant tutorial in kikuchipy's docs (say, this one) would be a good temporary solution until something is included in PyEBSDIndex itself?

[Fred-Ad]

Hi Hakon, and amazing work in Kikuchipy !! This implementation of a Hough solution for non FCC structure is wonderfull !
The tutorial is a good solution at first. Moreover, the code is not so complex. where does the 0.48 comes form to filter the reflectors ? Do you fix it arbitrally ?
The best solution at the end would be a list of structures included directly in KikuchiPy/pyeEBSIndex, like the commercial softwares. For example, fed by users each time they study a new structure thanks to KikuchiPy/PyEBSDIndex.
Would it be legal to use and reimplement the list of reflectors form EDAX for the most used structures ?

See you.

Frédéric Adamski

[drowenhorst-nrl]

@hakonanes These are my favorite types of questions ... where things are mostly working, and someone else has done a lot of good work to make them work better. Yes - the original intent was simply to leave the decision of what poles to include as an exercise for the user. And glad a couple of users have put in the good work on this!

Couple of notes:
FIRST

Do you have any thoughts on how users should find suitable reflector lists?
My initial thought was that the vast majority of people would be happy with the FCC, BCC, HCP built in, and that beyond that they would manually "yank" the proper reflectors from their commercial software of choice. So, on otherwords, I was going to be lazy about it.

Do you think linking to a relevant tutorial in kikuchipy's docs (say, this one) would be a good temporary solution until something is included in PyEBSDIndex itself?

Why not make it the long term solution? I have no desire to re-invent/recode all the work that kikuchipy has already done, and I am keenly aware that I do not want PyEBSDIndex to have large mission creep.

However, looking at @Fred-Ad response above, I wondering if it makes sense to make a simple file format that can be stored to disk for know phases? I am somewhat resistant to directly co-op the EDAX (or Oxford, or ...) file format simply because I know that they do charge for those, and they are not an open format. But ... using a similar LIST does not seem like a violation. Otherwise, there is the EMSoft SHT database - this would also be a good starting point? https://github.com/EMsoft-org/SHTdatabase

I think having it as a separate repo is a good idea, similar to the SHT repo, as there can be a bit of control/curation of what gets added to the global list, but certainly would not prevent users from using their own local files. Thoughts?
@hakonanes - what do you think about a file export?

SECOND
I am actually surprised this worked as well as it did! I had not attempted differentiating between a "cubic-low" vs "cubic-high" symmetry phase. I realize the poles were automated, but {6 0 0} vs {10 0 0}, when it comes to band angles, should be identical. I will have to check, but I am pretty sure when I create the triplet lookup libraries, one of these would just get filtered out. Do you want to try indexing without {10 0 0}, and see if it changes any answers?

We have seen some issues with some phase differentiation on some other data (patterns were not as nice, and classic pseudo symmetry issues). I am still planning on trying to attack a bandwidth metric within the indexing, which may help with this ... but I am trying to think through if I nested the operations in such a way that it is hard to cross-reference bandwidth matching across phases? I'll have to think through this some more. Besides ... bandwidths are hard given that they will change depending on where they are on the detector. So many little details to figure out there ...

THIRD
And it looks like using the 3-index notation for Hex works! Thanks for another test.

[hakonanes]

Thank you both for the positive response. Then I suggest to link to the suggested kikuchipy tutorial somewhere in PyEBSDIndex' docs.

where does the 0.48 comes form to filter the reflectors ? Do you fix it arbitrally ?

Quite arbitrarily, as I was looking for a "happy medium" of families. I set the required fraction of the strongest reflector linear intensity |F| to >50% so that I get six instead of ten families of planes for the Pi phase. The bands after wrapping the reflectors in a simulator and plotting them with a linear intensity scaling |F|:

Six families	Ten families

Note that this initial test is just to see if PyEBSDIndex finds the correct phase. It is by no means a statistical study...

Do you want to try indexing without {10 0 0}, and see if it changes any answers?

The returned crystal orientations and pattern fits for Alpha-Al(Mn,Fe)Si are identical for the reflector lists with and without {10 0 0}. I hadn't noticed the superfluous family before posting: thanks. One way to remove reflectors with identical angles is to ask for the indices of the unique unit reflectors and only keep those. The following should work:

_, idx = ref_alfesi.unit.unique(return_index=True)  # Both (6 0 0) and (10 0 0) -> (13 0 0)
ref_alfesi = ref_alfesi[idx]  # {10 0 0} family excluded

Regarding hosting user-submitted reflector lists:

A very good idea, @Fred-Ad. In the beginning, I would suggest to host it on GitHub and allow downloading them via e.g. the kikuchipy.data module. A new list can be submitted via pull request after a discussion of the benefit and applicability of the list. Longer term, I think something along the lines of an online database (hosted separate from kikuchipy) with some standardized API and a submission form would be a better solution. Then software like AstroEBSD, PyEBSDIndex, and kikuchipy can use the list on an "equal footing".

@drowenhorst-nrl, I haven't looked at the SHT file database before. It seems well structured. Do you know the contents of an SHT file? Does it contain a reflector list? It may be that the file contains more information than we currently need for this list. We would anway need a reader of the files in Python before thinking about using this database.

And it looks like using the 3-index notation for Hex works!

😀