Backflow of VMR -> Runtime is undoing changes in runtime

[ericstj]

Pull request

dotnet/runtime#123100

Type of problem

Incorrect contents in the PR

Description of the issue

dotnet/runtime#123100 (comment) has the details.

No changes to this file happened in VMR, code flowed from runtime -a-> VMR -b-> runtime. Between a and b, this file was modified in runtime. The change at b (which was already present in a) flowed back to runtime and regressed changes in runtime.

[premun]

This is the codeflow diagram with the commits in question highlighted:

[ericstj]

@premun help me understand this diagram. Does this explain what's happening here, did we do something wrong?

[premun]

@ericstj nothing wrong on your part. I usually put these with investigations so it's easy to go back to.
This diagram shows forward flows (green) and backflows (red) and helps us understand what happened when.
The orange commits are the ones you identified in your comment.

I am still looking into this. I found the bug but don't have the fix validated fully yet.

[ericstj]

What are the straight red lines?

[premun]

They should be arrows back but the graph engine somehow can't draw horizontal arrowheads.

FYI there's a script here: https://github.com/dotnet/arcade-services/blob/main/eng/Create-CodeflowGraphVizGraph.ps1

It can generate the graph for two local repos.

[premun]

So the problem seems to be in the piece that's trying to detect+fix reverts. It tries to revere-apply the latest flow to see if something was missed during the rebase of the working branch.
However, if the target branch had new changes on top of those files, the reverse-application would fail but not because there's missing stuff.

We have two options:

• Try the reverse application with zero context so that neighbouring changes don't fail it.
• Try the reverse application on the version of the file from the point of the last crossing flow, not on top of the tip of the target branch (which could contain these new extra changes to that file). This means to first remove the pink changes and try reverse-application and then reapplying them back (instead of reverse applying on top of the pink block):
  

[premun]

We have another, more minimal case, better for debugging:

dotnet/templating#9748

What happens here is that Marc accidentally makes the 2xx branch 10.0.300
that forward flows, he realizes a mistake, fixes it
but after that, it backflows and we go back to 10.0.300

[premun]

The example with templating uncovered a problem - dotnet/dotnet#4380

So not sure we can still use it as a repro.