PRP: Secret extractor for DataBricks User Account Personal Access Token

[0xXA]

• Secret name: DataBricks User Account Personal Access Token
• DataBricks Explained:
  • What is DataBricks ?

   Databricks is a cloud-based data & AI platform built on top of Apache Spark.
   It’s designed to help companies store, process, analyze, and build machine learning models using massive amounts of data.
   Think of it as a workspace where data engineers, data scientists, and analysts all work together on big data.
   
   It runs on top of cloud providers like: AWS, Azure, and Google Cloud.
   Companies use Databricks for Big Data Processing, Data Warehousing & Analytics, and Machine Learning & AI.
  

  • What type of accounts are there in DataBricks ?

   There are two types of accounts that are used for authentication:
  
   	User account: For interactive CLI commands and API calls.
   	Service principal: For automated CLI commands and API calls without human interaction.
  

  • Does DataBricks Personal Access Token have a prefix ?

   Yes, all DataBricks Personal Access Token regardless of User Account or Service Principal have the following format:
   	A "dapi" prefix followed by an Alphanumeric String. In concret regex terms: dapi[a-zA-Z0-9]
  

• Risk in exposing the secret:

   A leaked PAT is basically account takeover via API. Depending on the token owner's permissions, an attacker may be able to:
   Access Sensitive Data (customer's private data), Steal Intellectual Property (source code), Destroy or Corrupt Data,
   Run Malicious Workloads, and Move Laterally in the Cloud (S3 and GCS buckets).
  

• Validation method, if any:

• Send a POST request to /api/2.0/token/create endpoint with URL below as host to validate the secret:
  https://accounts.cloud.databricks.com // For AWS
  https://accounts.gcp.databricks.com // For Google Cloud
  https://accounts.azuredatabricks.net // For Microsoft Azure
  One must also include this in the header: PAT, and Account id
  We must note that Account id is only required for User Account authentication.
  Also, validation for user accounts is more complex than Service Principal because we need to send requests to all three endpoints.
• If the response code is 200 or 400 or 401, then it's a valid token

• Resources:
  • AWS DataBricks Documentation
  • GCP DataBricks Documentation
  • Azure DataBricks Documentation
  • AWS DataBricks Account Type & Authentication Documentation
  • GCP DataBricks Account Type & Authentication Documentation
  • AWS DataBricks PAT Authentication Documentation
  • GCP DataBricks PAT Authentication Documentation
  • Token Format Example
  • Token Creation Endpoint Documentation

[github-actions]

Welcome to the OSV-SCALIBR patch reward program!
Your issue has been added to our triage queue and will reviewed
shortly. The panel usually takes a decision once per week, so it
can take up to a week before you hear back from us.
Please, do not start the work until the panel has reached a
decision. Although we always welcome contributions, unapproved
work is not eligible for a reward.
~The OSV-SCALIBR PRP team

[erikvarga]

Hi @0xXA ,

We'd be interested in accepting your 4 DataBricks related submissions as one PRP as with the new panel decision we're processing the most relevant secrets for the same platform as a single submission.

How complex do you expect the detection and validation logic for these to be? In my understanding it'd be doing regexp searches (with proximity matches to context strings for Databricks in the case of generic tokens like Oauth) and it'd be querying a number of different HTTP endpoints for secret validation, is that correect?

[0xXA]

How complex do you expect the detection and validation logic for these to be? In my understanding it'd be doing regexp searches (with proximity matches to context strings for Databricks in the case of generic tokens like Oauth) and it'd be querying a number of different HTTP endpoints for secret validation, is that correect?

Indeed this but I’m yet to code these plugins so it’s just a guess for now. Thus, we can say that it’s going to be complex.

[0xXA]

I tried implementing, this is going to be complex since I can't use simplevalidate library. I need to implement my own Validate() for testing all three urls and return the successful result (if any) or error otherwise. Will have to write test cases for this Validate() too.

[erikvarga]

Thanks for the info. Yep, it sounds like you'll either need to implement your own validation library or extend the existing simplevalidate lib to support multiple endpoints. In either case my assessment is that this should qualify for the Secrets requiring complex detection methods category.

[github-actions]

Congratulations, your request has been approved! 🎉
This means that you can start working on this contribution.
❗ Please take a moment to fill the participation form
If you are unsure where to start, we have compiled a set of
useful guides in our documentation:

• Writing a new vulnerability detector plugin
• Writing a new inventory extraction plugin
• Writing a new secret detector plugin
• General style guide

~The OSV-SCALIBR PRP team

[0xXA]

Thanks for the info. Yep, it sounds like you'll either need to implement your own validation library or extend the existing simplevalidate lib to support multiple endpoints. In either case my assessment is that this should qualify for the Secrets requiring complex detection methods category.

I’m thinking of implementing a nvalidate library which will be a generalised version of simple validate. To ensure backward compatibility with other plugins, I’ll rewrite simplevalidate as a wrapper around nvalidate.

[erikvarga]

How about simply introducing an Endpoints []string field, to be used instead of the existing Endpoint string if it's set? That way existing users of the library can continue using it as is and we don't need a new wrapper.

[0xXA]

How about simply introducing an Endpoints []string field, to be used instead of the existing Endpoint string if it's set? That way existing users of the library can continue using it as is and we don't need a new wrapper.

Hmm. In either case we will have to heavily rewrite Validate(). So it’s better I introduce a new library.

Moreover, Having Endpoint and Endpoints in a single structure is confusing.

[erikvarga]

Isn't the added logic just that we'd run endpoint queries in a loop? That doesn't sounds like a large rewrite so it should be less code than setting up a wrapper for the existing simplevalidate.

Moreover, Having Endpoint and Endpoints in a single structure is confusing.

Agree that this is somewhat confusing, but the idea was that existing users can be migrated to the Endpoints field over time and Endpoint can eventually be removed. I think that's less confusing than having two simplevalidate-esqe libs that only differ in a single field.

[0xXA]

Isn't the added logic just that we'd run endpoint queries in a loop? That doesn't sounds like a large rewrite so it should be less code than setting up a wrapper for the existing simplevalidate.

nvalidate.go:

// Copyright 2026 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//      http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// Package nvalidate contains a Validator for secrets that can be validated with
// simple HTTP queries and result code comparison.
package nvalidate

import (
	"context"
	"errors"
	"fmt"
	"io"
	"net/http"
	"slices"
	"strings"

	"github.com/google/osv-scalibr/veles"
)

// Validator validates a secret by sending HTTP requests to multiple endpoints
// and checking response status codes.
// It implements veles.Validator.
type Validator[S veles.Secret] struct {
	// The API endpoints to query.
	Endpoints []string
	// Function that constructs multiple endpoints for a given secret.
	// Exactly one of Endpoints or EndpointFunc must be provided.
	// If EndpointFunc returns an error, Validate returns ValidationFailed and the error.
	EndpointFunc func(S) ([]string, error)
	// The HTTP request method to send (e.g. http.MethodGet, http.MethodPost)
	HTTPMethod string
	// HTTP headers to set in the query based on the secret.
	HTTPHeaders func(S) map[string]string
	// The body to set in the query based on the secret.
	// If Body returns an error, Validate returns ValidationFailed and the error.
	Body func(S) (string, error)
	// Status codes that should result in a "ValidationValid" validation result.
	ValidResponseCodes []int
	// Status codes that should result in a "ValidationInvalid" validation result.
	InvalidResponseCodes []int
	// Additional custom validation logic to perform on the response body. Will run if none of the
	// status codes from ValidResponseCodes and InvalidResponseCodes have been found.
	StatusFromResponseBody func(body io.Reader) (veles.ValidationStatus, error)
	// The HTTP client to use for the network queries. Uses http.DefaultClient if nil.
	HTTPC *http.Client
}

// Validate validates a secret with multiple HTTP requests.
func (v *Validator[S]) Validate(ctx context.Context, secret S) (veles.ValidationStatus, error) {
	if v.HTTPC == nil {
		v.HTTPC = http.DefaultClient
	}

	if (len(v.Endpoints) == 0 && v.EndpointFunc == nil) ||
		(len(v.Endpoints) > 0 && v.EndpointFunc != nil) {
		return veles.ValidationFailed, errors.New("exactly one of Endpoints or EndpointFunc must be specified")
	}

	endpoints := v.Endpoints
	if v.EndpointFunc != nil {
		var err error
		endpoints, err = v.EndpointFunc(secret)
		if err != nil {
			return veles.ValidationFailed, err
		}
		if len(endpoints) == 0 {
			return veles.ValidationFailed, errors.New("EndpointFunc returned no endpoints")
		}
	}

	var reqBody string
	var err error
	if v.Body != nil {
		reqBody, err = v.Body(secret)
		if err != nil {
			return veles.ValidationFailed, err
		}
	}

	var sawInvalid bool
	var endpointErrors []string
	singleEndpoint := len(endpoints) == 1
	for _, endpoint := range endpoints {

		if ctx.Err() != nil {
			return veles.ValidationFailed, ctx.Err()
		}

		var bodyReader io.Reader
		if len(reqBody) > 0 {
			bodyReader = strings.NewReader(reqBody)
		}

		req, err := http.NewRequestWithContext(ctx, v.HTTPMethod, endpoint, bodyReader)
		if err != nil {
			return veles.ValidationFailed, fmt.Errorf("http.NewRequestWithContext: %w", err)
		}

		if v.HTTPHeaders != nil {
			for key, val := range v.HTTPHeaders(secret) {
				req.Header.Set(key, val)
			}
		}

		res, err := v.HTTPC.Do(req)
		if err != nil {
			if ctx.Err() != nil {
				return veles.ValidationFailed, err
			}
			endpointErrors = append(endpointErrors, fmt.Sprintf("%s: request failed: %v", endpoint, err))
			continue
		}
		defer res.Body.Close()

		if slices.Contains(v.ValidResponseCodes, res.StatusCode) {
			return veles.ValidationValid, nil
		}

		if slices.Contains(v.InvalidResponseCodes, res.StatusCode) {
			sawInvalid = true
			continue
		}

		if v.StatusFromResponseBody != nil {
			status, bodyErr := v.StatusFromResponseBody(res.Body)
			if singleEndpoint {
				return status, bodyErr
			}
			if bodyErr != nil {
				endpointErrors = append(endpointErrors, fmt.Sprintf("%s: body parse failed: %v", endpoint, bodyErr))
				continue
			}
			if status == veles.ValidationValid {
				return veles.ValidationValid, nil
			}
			if status == veles.ValidationInvalid {
				sawInvalid = true
				continue
			}
			if status == veles.ValidationFailed {
				continue
			}
		}
		endpointErrors = append(endpointErrors, fmt.Sprintf("%s: unexpected HTTP status %d", endpoint, res.StatusCode))
	}

	if sawInvalid {
    	return veles.ValidationInvalid, nil
	}

	if len(endpointErrors) > 0 {
		return veles.ValidationFailed, fmt.Errorf("%s", strings.Join(endpointErrors, "; "))
	}

	return veles.ValidationFailed, errors.New("no endpoint produced a definitive result")
}

simplevalidate.go:

// simple HTTP queries and result code comparison.
package simplevalidate

import (
	"context"
	"errors"
	"io"
	"net/http"

	nv "github.com/google/osv-scalibr/veles/secrets/common/nvalidate"
	"github.com/google/osv-scalibr/veles"
)

// Validator validates a secret of a given type by sending HTTP requests and
// checking the response status code.
// It implements veles.Validator.
type Validator[S veles.Secret] struct {
	// The API endpoint to query.
	Endpoint string
	// Function that constructs the endpoint for a given secret.
	// Exactly one of Endpoint or EndpointFunc must be provided.
	// If EndpointFunc returns an error, Validate returns ValidationFailed and the error.
	EndpointFunc func(S) (string, error)
	// The HTTP request method to send (e.g. http.MethodGet, http.MethodPost)
	HTTPMethod string
	// HTTP headers to set in the query based on the secret.
	HTTPHeaders func(S) map[string]string
	// The body to set in the query based on the secret.
	// If Body returns an error, Validate returns ValidationFailed and the error.
	Body func(S) (string, error)
	// Status codes that should result in a "ValidationValid" validation result.
	ValidResponseCodes []int
	// Status codes that should result in a "ValidationInvalid" validation result.
	InvalidResponseCodes []int
	// Additional custom validation logic to perform on the response body. Will run if none of the
	// status codes from ValidResponseCodes and InvalidResponseCodes have been found.
	StatusFromResponseBody func(body io.Reader) (veles.ValidationStatus, error)
	// The HTTP client to use for the network queries. Uses http.DefaultClient if nil.
	HTTPC *http.Client
}

// Validate validates a secret with a simple HTTP request.
func (v *Validator[S]) Validate(ctx context.Context, secret S) (veles.ValidationStatus, error) {

	if (v.Endpoint == "" && v.EndpointFunc == nil) ||
		(v.Endpoint != "" && v.EndpointFunc != nil) {
		return veles.ValidationFailed, errors.New("exactly one of Endpoint or EndpointFunc must be specified")
	}

	nvValidator := &nv.Validator[S]{
		HTTPMethod:             v.HTTPMethod,
		HTTPHeaders:            v.HTTPHeaders,
		Body:                   v.Body,
		ValidResponseCodes:     v.ValidResponseCodes,
		InvalidResponseCodes:   v.InvalidResponseCodes,
		StatusFromResponseBody: v.StatusFromResponseBody,
		HTTPC:                  v.HTTPC,
	}

	if v.Endpoint != "" {
		nvValidator.Endpoints = []string{v.Endpoint}
	} else {
		nvValidator.EndpointFunc = func(s S) ([]string, error) {
			endpoint, err := v.EndpointFunc(s)
			if err != nil {
				return nil, err
			}
			if endpoint == "" {
				return nil, errors.New("EndpointFunc returned empty endpoint")
			}
			return []string{endpoint}, nil
		}
	}

	return nvValidator.Validate(ctx, secret)
}

I can do as you say, at the end of the day it's your project.