Optimizing XPath performance using XSLT extensions

[Honesty-of-the-Cavernous-Tissue]

The

trafilatura/trafilatura/htmlprocessing.py

Line 95 in 3d7e786

def prune_unwanted_nodes(

function takes a lot of time when processing large amounts of text, i used a common-crawl shard to test and used md5sum to verify the consistency of the changes.

[adbar]

Hi @Honesty-of-the-Cavernous-Tissue, while your idea is good in general order matters for these XPath expressions.

I advise against it, you would have to be sure to pass the tests and not to degrade the evaluation for your PR to be accepted.

[adbar]

@Honesty-of-the-Cavernous-Tissue You'd still need to fix type hinting for Python 3.13 and test of there is a regression on the benchmark (see tests Readme).

[adbar]

@Honesty-of-the-Cavernous-Tissue Are you still working on the PR?