Multitenant and Multiledger anoncreds legacy_indy profile issues

[jamshale]

When testing the anoncreds upgrade for indy credentials in a traction instance. This involves multi-tenancy, multi-ledger and endorsement there was an issue where the tenant profile switched to the root profile after upgrading. The issue was happening in this function https://github.com/openwallet-foundation/acapy/blob/main/acapy_agent/anoncreds/default/legacy_indy/registry.py#L810 related to the IndyLedgerRequestsExecutor.

What this class does exactly and why it's switching profile isn't exactly clear. Perhaps this process shouldn't be happening and was an implementer mistake.

We should figure out exactly what this class does and why this problem is happening.

This PR was opened long ago and I think it could be related. https://github.com/openwallet-foundation/acapy/pull/3649/changes

It was ultimately left stale because maintainers at the time thought it had a bit of and anti pattern but may be helpful for reference.

[jamshale]

My understanding right now is that when using multi-tenant and multi-ledger mode and writing to the ledger you must use

ledger = profile.inject_or(BaseLedger)

normally and it will re-try on multiple ledgers until it finds the one where the did lives.
When reading from the ledger the client must use the IndyLedgerRequestsExecutor

multitenant_mgr = profile.inject_or(BaseMultitenantManager)
if multitenant_mgr:
    ledger_exec_inst = IndyLedgerRequestsExecutor(profile)
else:
    ledger_exec_inst = profile.inject(IndyLedgerRequestsExecutor)
ledger_id, ledger = await ledger_exec_inst.get_ledger_for_identifier(
    schema_id,
    txn_record_type=GET_SCHEMA,
)

This is a poor pattern and error prone but I'm not going to attempt to change it at this time or completely study these classes. I don't think there is enough indy work going on for it to be worth the time, other than making sure multi-ledger is tested better and this problem doesn't occur elsewhere.

[swcurran]

The best path is to eliminate unqualified and did:sov DIDs. With did:indy, the problem of checking all the ledgers goes away. The problem is that those two approaches are so prevalent in the code. Hopefully they are being eliminated in practice.