Review: Better Stack dashboard vs DealBot JTBD requirements

[timfong888]

Better Stack Dashboard vs DealBot JTBD Requirements

Dashboard reviewed: https://telemetry.betterstack.com/dashboards/wt6QCw
Note: Dashboard title says "Staging" — this is not yet production.
Linear: XFN-296

---

Questions

1. Confirm that we are only measuring now TTFB as displayed. Originally there was a TTLB which I know doesn't capture transfer time but it had been something to know for customers running production websites.
2. Can we harmonize data from faults into a single dashboard?

---

Gaps — Prioritized

P0 — Data not collected or not charted

Gap	Detail
Faults / PDP fault rate	No PDP fault rate anywhere on the dashboard. This is a core JTBD metric (<=1% threshold).
CommP Retrieval	No direct SP retrieval path tested separately. Only IPFS retrieval is shown. PRD threshold is >=97%.
TTLB (last byte) chart	ipfsRetrievalLastByteMs exists in the raw data (visible in Debugging table) but has no chart. Current "IPFS Retrieval TTFB" only measures first byte — doesn't capture full 10MB transfer time. PRD threshold is p90 <=2s.

P1 — Eligibility layer missing

Gap	Detail
No threshold indicators	No green/red/yellow pass/fail signals. Raw numbers only — user has to remember that 90%/97%/2s are the thresholds.
No eligibility header	No summary "ELIGIBLE: X/Y providers" count. This is the single most important thing a viewer needs to see at a glance.
Provider names	Providers shown as numeric IDs (4, 7, 5, 2…). Need human-readable SP names or miner IDs.

P2 — Format polish

Gap	Detail
failures/total format	Rates show percentages only (100, 94.82, 88.89) with no sample-size context. Show failures/total (rate%) so viewers can judge significance.
Thresholds legend	No thresholds shown anywhere on the dashboard. Add a visible legend or inline labels so viewers don't need the PRD open.
Deal Success naming	Panel says data_storage_success_rate_pct — conflates "deal acceptance" with "data storage," which are two different concepts.

[andrianvaleanu]

Staging data usually looks messy, but tracking TTFB instead of TTLB for 10MB files is a big miss since it ignores the actual transfer speed. If you need something simpler for checking basic uptime and status pages while you're sorting out these custom metrics, Pulsetic https://pulsetic.com/ works well for that.

[timfong888]

@andrianvaleanu -- yes, we raised this as a discussion item. Because it depended on location we didn't make it a requirement for SPs acceptance or rejection from receiving deals, it is something we want to consider for latency metrics 10MB being an average size for websites.

[timfong888]

@SgtPooki -- This dashboard will be a click-through from the dealbot page which summarizes eligibility? So this will not replace dealbot.filecoin.cloud? It's a drill-down?

[BigLep]

@timfong888: thanks for raising. I (or Russell and I both) can probably respond or meet tomorrow to answer most of these questions. I am going to use my available time today to focus on some open PR reviews.

What has been implemented so far is following the simplified approach outlined in #239

[SgtPooki]

@SgtPooki -- This dashboard will be a click-through from the dealbot page which summarizes eligibility? So this will not replace dealbot.filecoin.cloud? It's a drill-down?

I clarified a bit on slack but i'll post here too:

dealbot dashboard (on staging.dealbot.filoz.org and dealbot.filoz.org) will look like what is screencaptured in #313

betterstack public provider dashboard (clicking the “view metrics dashboard” button in the screenshot in the above PR will take you to https://telemetry.betterstack.com/dashboards/wt6QCw for staging (or prod for prod). The metrics dashboard also has a table on it.. it should be the same that is embedded in the screenshot (the embed link used is https://telemetry.betterstack.com/dashboards/1MnUG3)

logs button/link will take you to https://telemetry.betterstack.com/dashboards/1eH3gR (for staging)

I believe we can re-use the same dashboards for staging/prod, we just need to update the source. That will be nailed down when we get things deployed to dealbot prod

[timfong888]

[BigLep]

@timfong888 : I didn't put my head into this today. If I don't do so beforehand, we'll definitely connect during our 2026-03-05 sync and and I'll post the notes here.

[BigLep]

@timfong888 : we'll cover this verbally 2026-03-05, but these are the notes to myself when going through the feedback: (I came in afterward and put in our discussion)

Confirm that we are only measuring now TTFB as displayed. Originally there was a TTLB which I know doesn't capture transfer time but it had been something to know for customers running production websites.

🎬 We can add TTLB.

• Sound good

Can we harmonize data from faults into a single dashboard?

❓ What is meant by this?

• Tim looked at the dashboard before we had data retention metrics.

No PDP fault rate anywhere on the dashboard

This is there now. Tim looked before it had been added.

No direct SP retrieval path tested separately

❓ I assume you mean no direct /piece retrieval?

• That's right. We don't have that. We are doing direct /ipfs retrieval. Agreed that is fine for now.

No green/red/yellow pass/fail signals.

You're right, we don't have it and we don't think they support it in terms of colored row. Next steps would be:

• 🎬 Check with betterstack support team
• 🎬 Add text note inline that specifies the thresholds
• We agreed to add the inline text about thresholds.

No summary "ELIGIBLE: X/Y providers" count

🎬 We can investigate if a query can be run that show this. Likely is possible.

• Agreed this can be added to the action list.

Need human-readable SP names or miner IDs.

Makes sense. That info doesn't live in betterstack and I don't think we want to invest in getting it there.

• Explained that isn't possible for now. The dealbot.filoz.org landing page can have this as a column (see Dealbot UI: basic table of SPs for easy links to dashboards #279 ).

Rates show percentages only

I see these. It looks like the label wasn't very clear before.

• Agreed we are good now.

No thresholds shown anywhere on the dashboard

🎬 Agreed we should a text box at the minimum.

• Agreed that static text should be added.

Deal Success naming

What do you want us to rename? We are using the 3 names from https://github.com/FilOzone/dealbot/tree/main/docs/checks: data storage, data retention, and data retrieval

• Tim couldn't find it again. We'll action if discrepancy or inconsistency is found in future.

---

Personal notes when looking at this top to bottom:

• Lets get a link to https://github.com/FilOzone/dealbot/blob/main/docs/checks/events-and-metrics.md to describe the metrics
• Lets add all the timing metrics we have?
• IngentMS is empty
• Wherever we show percentiles, also need to make clear the count for the percentile
• Default the dashboard to looking the last week

[BigLep]

2026-03-05 notes: they were added inline above and actions were enumerated below.

[BigLep]

@SgtPooki : Below are all the actions I think engineering needs to take based on this issue so far, my conversation with @timfong888 today 2026-03-05, and me spending time looking at this for myself. I assume once we do these items we'll likely get other feedback.

Presentation items

• Add all our timing metrics from https://github.com/FilOzone/dealbot/blob/main/docs/checks/production-configuration-and-approval-methodology.md
• Add static text linking and stating the threasholds

Metric	Threshold	Minimum Sample Size
Data Storage Success Rate	≥ 97%	200
Data Retention Fault Rate	≤ 0.2%	500
Retrieval Success Rate	≥ 97%	200

Metric Threshold Minimum Sample Size
Data Storage Success Rate ≥ 97% 200
Data Retention Fault Rate ≤ 0.2% 500
Retrieval Success Rate ≥ 97% 200

• Add a count of the number of eligible providers
• Default the dashboard to looking at the last 3 days
• Give visual indication for whether an SP is meeting acceptance threshold

There are multiple ways to do this. For example, we could have a single column that represents this or we can give more indication about if each column is meeting the threshold. This is what the latter looks like.

SELECT
  p.provider_id AS "Provider",
  if(
    coalesce(round(100.0 * s.success / nullIf(s.total, 0), 2), 0) >= 97
    AND coalesce(s.total, 0) >= 200
    AND coalesce(round(100.0 * (dr.total - dr.success) / nullIf(dr.total, 0), 2), 100) <= 0.2
    AND coalesce(dr.total, 0) >= 500
    AND coalesce(round(100.0 * r.success / nullIf(r.total, 0), 2), 0) >= 97
    AND coalesce(r.total, 0) >= 200,
    '✅', '❌'
  ) AS "Meeting Criteria",
  concat(
    coalesce(toString(round(100.0 * s.success / nullIf(s.total, 0), 2)), 'N/A'),
    if(coalesce(round(100.0 * s.success / nullIf(s.total, 0), 2), 0) >= 97, ' ✅', ' ❌')
  ) AS "Data Storage success%",
  concat(
    toString(toUInt64(coalesce(s.total, 0))),
    if(coalesce(s.total, 0) >= 200, ' ✅', ' ❌')
  ) AS "Data Storage check count",
  concat(
    coalesce(toString(round(100.0 * dr.success / nullIf(dr.total, 0), 2)), 'N/A'),
    if(coalesce(round(100.0 * (dr.total - dr.success) / nullIf(dr.total, 0), 2), 100) <= 0.2, ' ✅', ' ❌')
  ) AS "Data retention success%",
  concat(
    toString(toUInt64(coalesce(dr.total, 0))),
    if(coalesce(dr.total, 0) >= 500, ' ✅', ' ❌')
  ) AS "Data retention check count",
  concat(
    coalesce(toString(round(100.0 * r.success / nullIf(r.total, 0), 2)), 'N/A'),
    if(coalesce(round(100.0 * r.success / nullIf(r.total, 0), 2), 0) >= 97, ' ✅', ' ❌')
  ) AS "Retrieval success%",
  concat(
    toString(toUInt64(coalesce(r.total, 0))),
    if(coalesce(r.total, 0) >= 200, ' ✅', ' ❌')
  ) AS "Retrieval check count"

Data investigations:

• Understand why IngestMS is empty
• Understand why the retrieval check count significantly higher than the data storage check count

Things that would empower me to go in and make changes

1. I see we have different betterstack dashboards for prod vs staging. Is it possible just have one dashboard for both and set the environment with dropdown selection? (That way we only have to make edits one place.)
2. Should I just go in and make changes, or do we want to have any review process?
3. I guess not needed yet but I imagine we're going to want a dropdown for network once we have staging for example working with calibration and mainnet.