Skip to content

feat: analytics category refactor #14337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
iankhou wants to merge 2 commits into
base: gen2-migration
Choose a base branch
from
Open

feat: analytics category refactor #14337

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
44c6cef
feat: analytics category refactor
iankhou Nov 12, 2025
eacd814
fix: output for StreamArn was not an Arn
iankhou Nov 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions ...-cli/src/__tests__/commands/gen2-migration/refactor/resolvers/cfn-output-resolver.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,12 @@ describe('CFNOutputResolver', () => {
Ref: 'snstopic',
},
},
KinesisStreamArn: {
Description: 'Kinesis Stream Arn',
Value: {
'Fn::GetAtt': ['MyKinesisStream', 'Arn'],
},
},
},
Resources: {
MyS3Bucket: {
Expand Down Expand Up @@ -221,6 +227,29 @@ describe('CFNOutputResolver', () => {
},
},
},
MyKinesisStream: {
Type: 'AWS::Kinesis::Stream',
Properties: {
Name: 'MyKinesisStream',
ShardCount: 1,
},
},
KinesisStreamPolicy: {
Type: 'AWS::IAM::Policy',
Properties: {
PolicyName: 'KinesisStreamPolicy',
PolicyDocument: {
Statement: [
{
Effect: 'Allow',
Action: 'kinesis:PutRecord',
Resource: { 'Fn::GetAtt': ['MyKinesisStream', 'Arn'] },
},
],
},
Roles: [{ Ref: 'AuthenticatedRole' }],
},
},
},
};
const expectedTemplate: CFNTemplate = {
Expand Down Expand Up @@ -264,6 +293,10 @@ describe('CFNOutputResolver', () => {
Description: 'SnsTopicArn',
Value: 'arn:aws:sns:us-east-1:12345:snsTopic',
},
KinesisStreamArn: {
Description: 'Kinesis Stream Arn',
Value: 'arn:aws:kinesis:us-east-1:12345:stream/MyKinesisStream',
},
},
Resources: {
MyS3Bucket: {
Expand Down Expand Up @@ -412,6 +445,29 @@ describe('CFNOutputResolver', () => {
BucketName: 'test-bucket',
},
},
MyKinesisStream: {
Type: 'AWS::Kinesis::Stream',
Properties: {
Name: 'MyKinesisStream',
ShardCount: 1,
},
},
KinesisStreamPolicy: {
Type: 'AWS::IAM::Policy',
Properties: {
PolicyName: 'KinesisStreamPolicy',
PolicyDocument: {
Statement: [
{
Effect: 'Allow',
Action: 'kinesis:PutRecord',
Resource: 'arn:aws:kinesis:us-east-1:12345:stream/MyKinesisStream',
},
],
},
Roles: [{ Ref: 'AuthenticatedRole' }],
},
},
},
};

Expand Down Expand Up @@ -452,6 +508,10 @@ describe('CFNOutputResolver', () => {
OutputKey: 'snsTopicArn',
OutputValue: 'arn:aws:sns:us-east-1:12345:snsTopic',
},
{
OutputKey: 'KinesisStreamArn',
OutputValue: 'arn:aws:kinesis:us-east-1:12345:stream/MyKinesisStream',
},
],
[
{
Expand Down
11 changes: 10 additions & 1 deletion packages/amplify-cli/src/commands/gen2-migration/refactor/generators/template-generator.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import {
CFNStackStatus,
CFNTemplate,
ResourceMapping,
CFN_ANALYTICS_TYPE,
} from '../types';
import MigrationReadmeGenerator from './migration-readme-generator';
import { pollStackForCompletionState, tryUpdateStack } from '../cfn-stack-updater';
Expand All @@ -33,7 +34,7 @@ import ora from 'ora';
const CFN_RESOURCE_STACK_TYPE = 'AWS::CloudFormation::Stack';
const GEN2_AMPLIFY_AUTH_LOGICAL_ID_PREFIX = 'amplifyAuth';

const CATEGORIES: CATEGORY[] = ['auth', 'storage'];
const CATEGORIES: CATEGORY[] = ['auth', 'storage', 'analytics'];
Copy link
Author

@iankhou iankhou Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strings in this array are used to check against the prefix of category stacks in both Gen1 and Gen2. In Gen1, the analytics stack always starts with "analyticsamplify", but in Gen2, it can be any name. Codegen will need to handle the naming accordingly.

const TEMPLATES_DIR = '.amplify/migration/templates';
const SEPARATOR = ' to ';

Expand All @@ -48,18 +49,21 @@ const AUTH_RESOURCES_TO_REFACTOR = [
];
const AUTH_USER_POOL_GROUP_RESOURCES_TO_REFACTOR = [CFN_AUTH_TYPE.UserPoolGroup];
const STORAGE_RESOURCES_TO_REFACTOR = [CFN_S3_TYPE.Bucket];
const ANALYTICS_RESOURCES_TO_REFACTOR = [CFN_ANALYTICS_TYPE.Stream];
const GEN1_RESOURCE_TYPE_TO_LOGICAL_RESOURCE_IDS_MAP = new Map<string, string>([
[CFN_AUTH_TYPE.UserPool.valueOf(), 'UserPool'],
[CFN_AUTH_TYPE.UserPoolClient.valueOf(), 'UserPoolClientWeb'],
[CFN_AUTH_TYPE.IdentityPool.valueOf(), 'IdentityPool'],
[CFN_AUTH_TYPE.IdentityPoolRoleAttachment.valueOf(), 'IdentityPoolRoleMap'],
[CFN_AUTH_TYPE.UserPoolDomain.valueOf(), 'UserPoolDomain'],
[CFN_S3_TYPE.Bucket.valueOf(), 'S3Bucket'],
[CFN_ANALYTICS_TYPE.Stream.valueOf(), 'KinesisStream'],
]);
const LOGICAL_IDS_TO_REMOVE_FOR_REVERT_MAP = new Map<CATEGORY, CFN_RESOURCE_TYPES[]>([
['auth', AUTH_RESOURCES_TO_REFACTOR],
['auth-user-pool-group', AUTH_USER_POOL_GROUP_RESOURCES_TO_REFACTOR],
['storage', [CFN_S3_TYPE.Bucket]],
['analytics', ANALYTICS_RESOURCES_TO_REFACTOR],
Copy link
Author

@iankhou iankhou Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Used for rollback, not used yet.

]);
const GEN2_NATIVE_APP_CLIENT = 'UserPoolNativeAppClient';
const GEN1_USER_POOL_GROUPS_STACK_TYPE_DESCRIPTION = 'auth-Cognito-UserPool-Groups';
Expand All @@ -82,6 +86,9 @@ class TemplateGenerator {
storage: {
resourcesToRefactor: STORAGE_RESOURCES_TO_REFACTOR,
},
analytics: {
resourcesToRefactor: ANALYTICS_RESOURCES_TO_REFACTOR,
},
} as const;

constructor(
Expand Down Expand Up @@ -214,12 +221,14 @@ class TemplateGenerator {
assert(sourceCategoryStacks && sourceCategoryStacks?.length > 0, 'No source category stack found');
assert(destinationCategoryStacks && destinationCategoryStacks?.length > 0, 'No destination category stack found');
for (const { LogicalResourceId: sourceLogicalResourceId, PhysicalResourceId: sourcePhysicalResourceId } of sourceCategoryStacks) {
// find the valid, migrate-able categories in Gen1 stack
const category = CATEGORIES.find((category) => sourceLogicalResourceId?.startsWith(category));
if (!category) continue;
assert(sourcePhysicalResourceId);
let destinationPhysicalResourceId: string | undefined;
let userPoolGroupDestinationPhysicalResourceId: string | undefined;

// find the corresponding category stack in Gen2 stack
const correspondingCategoryStackInDestination = destinationCategoryStacks.find(
({ LogicalResourceId: destinationLogicalResourceId }) => destinationLogicalResourceId?.startsWith(category),
);
Expand Down
5 changes: 5 additions & 0 deletions packages/amplify-cli/src/commands/gen2-migration/refactor/resolvers/cfn-output-resolver.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,11 @@ class CfnOutputResolver {
return {
Arn: `arn:aws:lambda:${this.region}:${this.accountId}:function:${resourceIdentifier}`,
};
case 'AWS::Kinesis::Stream':
return {
// output is already in ARN format
Arn: resourceIdentifier,
};
default:
Copy link
Author

@iankhou iankhou Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably default to resourceIdentifier instead of undefined... but we will take a deeper look at this later.

return undefined;
}
Expand Down
7 changes: 6 additions & 1 deletion packages/amplify-cli/src/commands/gen2-migration/refactor/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ export enum NON_CUSTOM_RESOURCE_CATEGORY {
AUTH = 'auth',
STORAGE = 'storage',
AUTH_USER_POOL_GROUP = 'auth-user-pool-group',
ANALYTICS = 'analytics',
Copy link
Author

@iankhou iankhou Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Included this here so that the custom resource flow is not triggered.

}

export type CATEGORY =
Expand Down Expand Up @@ -101,6 +102,10 @@ export enum CFN_S3_TYPE {
Bucket = 'AWS::S3::Bucket',
}

export enum CFN_ANALYTICS_TYPE {
Stream = 'AWS::Kinesis::Stream',
Copy link
Author

@iankhou iankhou Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only stateful resource for Kinesis-based analytics. The rest are IAM roles and policies.

}

export enum CFN_IAM_TYPE {
Role = 'AWS::IAM::Role',
}
Expand All @@ -113,7 +118,7 @@ export enum CFN_LAMBDA_TYPE {
Function = 'AWS::Lambda::Function',
}

export type CFN_RESOURCE_TYPES = CFN_AUTH_TYPE | CFN_S3_TYPE | CFN_IAM_TYPE | CFN_SQS_TYPE | CFN_LAMBDA_TYPE;
export type CFN_RESOURCE_TYPES = CFN_AUTH_TYPE | CFN_S3_TYPE | CFN_ANALYTICS_TYPE | CFN_IAM_TYPE | CFN_SQS_TYPE | CFN_LAMBDA_TYPE;

export type AWS_RESOURCE_ATTRIBUTES = 'Arn';

Expand Down
Loading