Skip to content

[WIP] paasta validate in GO#35

Open
chlgit wants to merge 1 commit intomasterfrom
u/chl/PAASTA-16166_paasta_validate_in_GO
Open

[WIP] paasta validate in GO#35
chlgit wants to merge 1 commit intomasterfrom
u/chl/PAASTA-16166_paasta_validate_in_GO

Conversation

@chlgit
Copy link
Copy Markdown
Contributor

@chlgit chlgit commented Jan 10, 2020

Sample output looks like: https://fluffy.yelpcorp.com/i/4p6pWN0hC0qW2XgsHflpLTLgL0vtrqFN.html

There are a couple of things missing so far:
1 ) Some checks haven't been implemented yet.
2) Unit Test
3) There are issues to validate Kubernetes and Tron schema. For example, regexp, the standard go package for regular expression that the schema validator uses, doesn't support negative loopahead '?!' for performance issue (It can't be implemented by O(n) time complexity).
4) The green "Yes" and the red "No" need to change to some symbols.

@chlgit chlgit requested a review from kawaiwanyelp January 10, 2020 21:58
@chlgit chlgit self-assigned this Jan 10, 2020
Copy link
Copy Markdown
Contributor

@kawaiwanyelp kawaiwanyelp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The core logic looks good by there are still a number of Pythonisms in here.

Comment thread cmd/validate/validate.go
return true
}

func main() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You shouldn't use a main function here, since the main function has already been defined here: https://github.com/Yelp/paasta-tools-go/blob/master/cmd/paasta/main.go#L28. Besides, this is not supposed to be the main entry point into the PaaSTA command line tool. Golang isn't like Python, where each file can be a separate script. Instead, you should probably define a validate function, that is called by the "real" main function when paasta validate is run.

Comment thread cmd/validate/validate.go
SOAConfigPath string
}

func parseFlags(opts *ValidateOptions) error {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should take no arguments here and return a *ValidateOptions. It doesn't make sense to initialize the struct outside and pass it in here I think. You can also drop the error return type, since it seems this function has no possibly errors to return (you always return nil).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was working on metastatus binary in spare time and have this: https://github.com/Yelp/paasta-tools-go/blob/u/maksym/paasta-metastatus/pkg/cli/options.go

usage example in this file: https://github.com/Yelp/paasta-tools-go/blob/u/maksym/paasta-metastatus/cmd/paasta-metastatus/main.go#L14

maybe some of that code could be ported if useful

Comment thread cmd/validate/validate.go
Comment on lines +23 to +26
flag.StringVar(&opts.Service, "service", "", "service to validate")
flag.StringVar(&opts.SOAConfigPath, "yelpsoa-config-root", "", "yelpsoa-configs path")
flag.Parse()
return nil
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validate is meant to be a subcommand of paasta, but you've defined top-level flags here. My suggestion is to use a FlagSet (https://golang.org/pkg/flag/#FlagSet) here instead, which is equivalent to Python's subparser.

Comment thread cmd/validate/validate.go
if soa_dir == current_path {
return soa_dir, nil
}
return soa_dir, errors.New("Unknown service")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I'd add the service name to the error message. You can format string using fmt.Sprintf.

Optionally. if you wanted to take it a step further, considering this is a common error in PaaSTA, you could define an "UnknownServiceError" as a struct with a Service field, that returns this error message.

Comment thread cmd/validate/validate.go
return filepath.Join(soa_dir, service), nil
}

current_path, _ := os.Getwd()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: To follow Go patterns, you should always handle an error, even if that simply means just returning it.

Comment thread cmd/validate/validate.go
return false
}
if result.Valid() {
green := color.New(color.FgGreen).SprintFunc()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Just like in Python PaaSTA, you may want to eventually turn this into a global function (e.g. PaastaGreen), instead of declaring a local copy here. The same holds for all the other color functions you're using.

Comment thread cmd/validate/validate.go
// validateSchema checks if the specified config file has a valid schema.
// file_path is the path to file to validate.
// file_type is what schema type should we validate against
func validateSchema(file_name string, file_type string) bool {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Golang, you should use Camel case instead of Snake case (Pythonic) for everything, including variables (fileName or filename, instead of file_name). You should check your other variables too - there are a lot of cases in this file.

Comment thread cmd/validate/validate.go
// validateAllSchemas Finds all recognized config files in service directory,and validates their schema.
// service_path is the path to location of configuration files.
func validateAllSchemas(service_path string) bool {
matches, _ := filepath.Glob(service_path + "/*.yaml")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Use filepath.Join, instead of string concatenation.

Comment thread cmd/validate/validate.go
matches, _ := filepath.Glob(service_path + "/*.yaml")
return_code := true
for _, file_name := range matches {
file_info, _ := os.Lstat(file_name)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, you should at least return your errors.

Comment thread cmd/validate/validate.go
Comment on lines +135 to +139
if strings.HasPrefix(basename, file_type) == true {
if validateSchema(file_name, file_type) == false {
return_code = false
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of using an if-statement here, you should use a case-statement here, matching over the file type.
Also, you shouldn't need to use == true and == false; just use C-style boolean operators.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or maybe just

if strings.HasPrefix(basename, file_type) && !validateSchema(file_name, file_type) {
    return_code = false
}

?

@@ -0,0 +1,103 @@
{
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are these taken from https://github.com/Yelp/paasta/blob/master/paasta_tools/api/api_docs/swagger.json?

if yes, i would prefer if we copied the whole file and had a test that checks if it's out of sync with content from the url above.

Comment thread cmd/validate/validate.go
SOAConfigPath string
}

func parseFlags(opts *ValidateOptions) error {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was working on metastatus binary in spare time and have this: https://github.com/Yelp/paasta-tools-go/blob/u/maksym/paasta-metastatus/pkg/cli/options.go

usage example in this file: https://github.com/Yelp/paasta-tools-go/blob/u/maksym/paasta-metastatus/cmd/paasta-metastatus/main.go#L14

maybe some of that code could be ported if useful

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants