[BUG] Removing a rule set from a Front Door route doesn't work

[mderriey]

Library name and version

Azure.ResourceManager.Cdn 1.5.1

Describe the bug

I have a route with a single rule set that I want to dissociate from the route.

Here's the code I'm using:

FrontDoorRouteResource route = /* get the route */;
ResourceIdentifier ruleSetToDissociate = /* Get the ID */;

if (route.Data.RuleSets.Any(x => x.Id == ruleSetToDissociate))
{
    // 1. Build a PATCH with the exact same configuration as the route
    var routePatch = new FrontDoorRoutePatch
    {
        EnabledState = route.Data.EnabledState,
        ForwardingProtocol = route.Data.ForwardingProtocol,
        HttpsRedirect = route.Data.HttpsRedirect,
        LinkToDefaultDomain = route.Data.LinkToDefaultDomain,
        OriginGroupId = route.Data.OriginGroupId,
        OriginPath = route.Data.OriginPath,
    };

    foreach (var pattern in route.Data.PatternsToMatch)
    {
        routePatch.PatternsToMatch.Add(pattern);
    }

    foreach (var protocol in route.Data.SupportedProtocols)
    {
        routePatch.SupportedProtocols.Add(protocol);
    }

    foreach (var domain in route.Data.CustomDomains)
    {
        routePatch.CustomDomains.Add(domain);
    }

    // 2. EXCEPT for the rule sets where we ignore the one we don't want anymore
    foreach (var existingRuleSet in route.Data.RuleSets)
    {
        if (existingRuleSet.Id != ruleSetResourceId)
        {
            routePatch.RuleSets.Add(existingRuleSet);
        }
    }

    // 3. Issue the PATCH request
    await route.UpdateAsync(WaitUntil.Completed, routePatch, ct);
}

Expected behavior

The rule set is not associated with the route anymore.

Actual behavior

The rule set is still associated with the route.

I looked into this and found the following:

1. Rule sets will only be serialized if the collection is "defined", see

   azure-sdk-for-net/sdk/cdn/Azure.ResourceManager.Cdn/src/Generated/Models/FrontDoorRoutePatch.Serialization.cs

   Lines 66 to 75 in 9b622f0

   	if (Optional.IsCollectionDefined(RuleSets))
   	{
   	writer.WritePropertyName("ruleSets"u8);
   	writer.WriteStartArray();
   	foreach (var item in RuleSets)
   	{
   	((IJsonModel<WritableSubResource>)item).Write(writer, options);
   	}
   	writer.WriteEndArray();
   	}

2. IsCollectionDefined only returns false if it's a ChangeTrackingList<T> where IsUndefined is true,

   azure-sdk-for-net/sdk/cdn/Azure.ResourceManager.Cdn/src/Generated/Internal/Optional.cs

   Lines 15 to 18 in 9b622f0

   	public static bool IsCollectionDefined<T>(IEnumerable<T> collection)
   	{
   	return !(collection is ChangeTrackingList<T> changeTrackingList && changeTrackingList.IsUndefined);
   	}

3. RuleSets is indeed a ChangeTrackingList<T>,

   azure-sdk-for-net/sdk/cdn/Azure.ResourceManager.Cdn/src/Generated/Models/FrontDoorRoutePatch.cs

   Line 54 in 9b622f0

   RuleSets = new ChangeTrackingList<WritableSubResource>();

4. ChangeTrackingList<T> is marked as IsUndefined by default until an operation is performed on it, which ensures the backing List<T> is initialized, https://github.com/Azure/azure-sdk-for-net/blob/9b622f049bda3420d2bb4b5c29fe59a89fafa710/sdk/cdn/Azure.ResourceManager.Cdn/src/Generated/Internal/ChangeTrackingList.cs

In other words, not performing any actions on the RuleSets collection prevents it from being serialized, and the service doesn't remove the rule set from the route.

It can be worked around by making sure the ChangeTrackingList<T> is defined:

[...]

// Force it to be defined so it will be serialized to an empty array even if we don't add any rule sets below
routePatch.RuleSets.Clear();

// 2. EXCEPT for the rule sets where we ignore the one we don't want anymore
foreach (var existingRuleSet in route.Data.RuleSets)
{
    if (existingRuleSet.Id != ruleSetResourceId)
    {
        routePatch.RuleSets.Add(existingRuleSet);
    }
}

// 3. Issue the PATCH request
await route.UpdateAsync(WaitUntil.Completed, routePatch, ct);

Reproduction Steps

See code above.

Environment

• Azure App Service running on Windows.
• .NET Framework 4.8

[mderriey]

Seems related to #44489.

[jsquire]

Thank you for your feedback. Tagging and routing to the team member best able to assist.

[mderriey]

It seems the "problem" exists across all client libraries with collections when there's a single item that we want to remove.

If we dont interact with the collection (because we didn't add the item we wanted to remove), it's consiredered undefined and doesn't get serialized at all, and the service rightfully thinks there's no change necessary.

I understand why we made that choice, given collections are not nullable and not settable; we need a way to know whether the client wants to make a change to the collection.

I'm not advocating for a change, but I think documentation could help. I found cases in the codebase I'm working on where colleagues hit this, and ended up manually calling the API with an HttpClient as they thought the library had a bug.