Datasets and schema updates #15

max-ostapenko · 2024-09-19T17:51:07Z

Stabilisation of all schema tables
Implemented in Stable all.pages dataform#8 and Stable all.requests dataform#5

Pruned summary description
Splitting custom_metrics
Structured data columns type changed from STRING to JSON

rviscomi · 2024-10-07T13:46:51Z

Moving back to draft until we've finalized the schema changes

max-ostapenko · 2024-10-10T09:34:59Z

src/content/docs/reference/structs/page-payload.md

+    "_minify_savings": -1,
+    "_minify_total": -1,
+    "_optimization_checked": 1,
+    "_origin_dns": {
+        "cname": [],
+        "https": [],
+        "mx": ["0 ."],
+        "ns": [
+            "a.iana-servers.net.",
+            "b.iana-servers.net."
+        ],
+        "soa": [
+            "ns.icann.org. noc.dns.icann.org. 2024081420 7200 3600 1209600 3600"
+        ],
+        "svcb": [],
+        "txt": [
+            "\"v=spf1 -all\"",
+            "\"wgyf8z8cgvm2qmxpnbnldrcltvk4xqfn\""
+        ]
+    },
+    "_osPlatform": "x86_64 x86_64",
+    "_osVersion": "Linux 6.8.0-1014-gcp",
+    "_os_version": "Linux 6.8.0-1014-gcp",
+    "_render": 400,
+    "_renderBlockingCSS": 0,
+    "_renderBlockingJS": 0,
+    "_requests": 2,
+    "_requestsDoc": 2,
+    "_requestsFull": 2,
+    "_responses_200": 1,
+    "_responses_404": 1,
+    "_responses_other": 0,
+    "_result": 99999,
+    "_run": 1,
+    "_score_cache": -1,
+    "_score_cdn": 100,
+    "_score_combine": -1,
+    "_score_compress": -1,
+    "_score_cookies": -1,
+    "_score_etags": -1,
+    "_score_gzip": 100,
+    "_score_keep-alive": 100,
+    "_score_minify": -1,


Suggested change

"_minify_savings": -1,

"_minify_total": -1,

"_optimization_checked": 1,

"_origin_dns": {

"cname": [],

"https": [],

"mx": ["0 ."],

"ns": [

"a.iana-servers.net.",

"b.iana-servers.net."

],

"soa": [

"ns.icann.org. noc.dns.icann.org. 2024081420 7200 3600 1209600 3600"

],

"svcb": [],

"txt": [

"\"v=spf1 -all\"",

"\"wgyf8z8cgvm2qmxpnbnldrcltvk4xqfn\""

]

},

"_osPlatform": "x86_64 x86_64",

"_osVersion": "Linux 6.8.0-1014-gcp",

"_os_version": "Linux 6.8.0-1014-gcp",

"_render": 400,

"_renderBlockingCSS": 0,

"_renderBlockingJS": 0,

"_requests": 2,

"_requestsDoc": 2,

"_requestsFull": 2,

"_responses_200": 1,

"_responses_404": 1,

"_responses_other": 0,

"_result": 99999,

"_run": 1,

"_score_cache": -1,

"_score_cdn": 100,

"_score_combine": -1,

"_score_compress": -1,

"_score_cookies": -1,

"_score_etags": -1,

"_score_gzip": 100,

"_score_keep-alive": 100,

"_score_minify": -1,

"_optimization_checked": 1,

"_origin_dns": {

"cname": [],

"https": [],

"mx": ["0 ."],

"ns": [

"a.iana-servers.net.",

"b.iana-servers.net."

],

"soa": [

"ns.icann.org. noc.dns.icann.org. 2024081420 7200 3600 1209600 3600"

],

"svcb": [],

"txt": [

"\"v=spf1 -all\"",

"\"wgyf8z8cgvm2qmxpnbnldrcltvk4xqfn\""

]

},

"_osPlatform": "x86_64 x86_64",

"_osVersion": "Linux 6.8.0-1014-gcp",

"_os_version": "Linux 6.8.0-1014-gcp",

"_render": 400,

"_renderBlockingCSS": 0,

"_renderBlockingJS": 0,

"_requests": 2,

"_requestsDoc": 2,

"_requestsFull": 2,

"_responses_200": 1,

"_responses_404": 1,

"_responses_other": 0,

"_result": 99999,

"_run": 1,

"_score_cache": -1,

"_score_cdn": 100,

"_score_compress": -1,

"_score_gzip": 100,

"_score_keep-alive": 100,

Deprecated: https://docs.webpagetest.org/metrics/page-metrics/#optimization-checks%2Fgrades

max-ostapenko · 2024-10-10T09:37:59Z

src/content/docs/reference/structs/page-payload.md

+    "_score_minify": -1,
+    "_score_progressive_jpeg": -1,
+    "_server_rtt": 0,
+    "_start_epoch": 1726370501.1809506,


Suggested change

"_start_epoch": 1726370501.1809506,

Deprecated: https://docs.webpagetest.org/metrics/page-metrics/#test-information

max-ostapenko · 2024-10-10T09:39:18Z

src/content/docs/reference/structs/page-payload.md

+    "_browserVersion": "128.0.0.0",
+    "_browser_name": "Chrome",
+    "_browser_version": "128.0.0.0",


Duplicate: https://docs.webpagetest.org/metrics/page-metrics/#browser-state

Suggested change

"_browserVersion": "128.0.0.0",

"_browser_name": "Chrome",

"_browser_version": "128.0.0.0",

"_browser_name": "Chrome",

"_browser_version": "128.0.0.0",

max-ostapenko · 2024-10-10T09:46:06Z

src/content/docs/reference/structs/page-payload.md

+    "_gzip_total": 648,
+    "_image_savings": 0,
+    "_image_total": 0,
+    "_interactivePeriods": [[0, 3452]],


Suggested change

"_interactivePeriods": [[0, 3452]],

Deprecated: https://docs.webpagetest.org/metrics/page-metrics/#interactivity-metrics

max-ostapenko · 2024-10-10T09:50:02Z

src/content/docs/reference/structs/page-payload.md

+    "_TTFB": 232,
+    "_TTIMeasurementEnd": 3452,
+    "_URL": "https://www.example.com/",
+    "_aft": 0,


Suggested change

"_aft": 0,

Deprecated: https://docs.webpagetest.org/metrics/page-metrics/#visual-metrics

max-ostapenko · 2024-10-10T09:50:16Z

src/content/docs/reference/structs/page-payload.md

+    "_testUrl": "https://www.example.com/",
+    "_test_run_time_ms": 5850,
+    "_tester": "agents-west-1-kl42-10.138.2.146",
+    "_titleTime": 76,


Suggested change

"_titleTime": 76,

Deprecated: https://docs.webpagetest.org/metrics/page-metrics/#visual-metrics

max-ostapenko · 2024-10-10T09:51:00Z

src/content/docs/reference/structs/page-payload.md

+    "_final_base_page_request_id": "44714C6F6C72B045136D19CA9894886A",
+    "_final_url": "https://www.example.com/",
+    "_firstContentfulPaint": 354,
+    "_firstMeaningfulPaint": 354,


Suggested change

"_firstMeaningfulPaint": 354,

Deprecated: https://docs.webpagetest.org/metrics/page-metrics/#visual-metrics

max-ostapenko · 2024-10-10T09:52:45Z

src/content/docs/reference/structs/page-payload.md

+    "_firstContentfulPaint": 354,
+    "_firstMeaningfulPaint": 354,
+    "_firstPaint": 314.80000000447035,
+    "_fullyLoaded": 323,


Suggested change

"_fullyLoaded": 323,

Duplicate: https://docs.webpagetest.org/metrics/page-metrics/#technical-page-metrics

max-ostapenko · 2024-10-10T09:56:25Z

src/content/docs/reference/structs/page-payload.md

+    "_chromeUserTiming.CumulativeLayoutShift": 0,
+    "_chromeUserTiming.LargestContentfulPaint": 354,
+    "_chromeUserTiming.LargestTextPaint": 354,
+    "_chromeUserTiming.TotalLayoutShift": 0,
+    "_chromeUserTiming.commitNavigationEnd": 285,
+    "_chromeUserTiming.domComplete": 320,
+    "_chromeUserTiming.domContentLoadedEventEnd": 320,
+    "_chromeUserTiming.domContentLoadedEventStart": 320,
+    "_chromeUserTiming.domInteractive": 320,
+    "_chromeUserTiming.domLoading": 285,
+    "_chromeUserTiming.fetchStart": 48,
+    "_chromeUserTiming.firstContentfulPaint": 354,
+    "_chromeUserTiming.firstMeaningfulPaint": 354,
+    "_chromeUserTiming.firstMeaningfulPaintCandidate": 354,
+    "_chromeUserTiming.firstPaint": 354,
+    "_chromeUserTiming.loadEventEnd": 320,
+    "_chromeUserTiming.loadEventStart": 320,
+    "_chromeUserTiming.markAsMainFrame": 284,
+    "_chromeUserTiming.navigationStart": 74,
+    "_chromeUserTiming.responseEnd": 277,
+    "_chromeUserTiming.unloadEventEnd": 284,
+    "_chromeUserTiming.unloadEventStart": 284,


Suggested change

"_chromeUserTiming.CumulativeLayoutShift": 0,

"_chromeUserTiming.LargestContentfulPaint": 354,

"_chromeUserTiming.LargestTextPaint": 354,

"_chromeUserTiming.TotalLayoutShift": 0,

"_chromeUserTiming.commitNavigationEnd": 285,

"_chromeUserTiming.domComplete": 320,

"_chromeUserTiming.domContentLoadedEventEnd": 320,

"_chromeUserTiming.domContentLoadedEventStart": 320,

"_chromeUserTiming.domInteractive": 320,

"_chromeUserTiming.domLoading": 285,

"_chromeUserTiming.fetchStart": 48,

"_chromeUserTiming.firstContentfulPaint": 354,

"_chromeUserTiming.firstMeaningfulPaint": 354,

"_chromeUserTiming.firstMeaningfulPaintCandidate": 354,

"_chromeUserTiming.firstPaint": 354,

"_chromeUserTiming.loadEventEnd": 320,

"_chromeUserTiming.loadEventStart": 320,

"_chromeUserTiming.markAsMainFrame": 284,

"_chromeUserTiming.navigationStart": 74,

"_chromeUserTiming.responseEnd": 277,

"_chromeUserTiming.unloadEventEnd": 284,

"_chromeUserTiming.unloadEventStart": 284,

"_chromeUserTiming.CumulativeLayoutShift": 0,

"_chromeUserTiming.LargestContentfulPaint": 354,

"_chromeUserTiming.LargestTextPaint": 354,

"_chromeUserTiming.TotalLayoutShift": 0,

"_chromeUserTiming.commitNavigationEnd": 285,

"_chromeUserTiming.domComplete": 320,

"_chromeUserTiming.domContentLoadedEventEnd": 320,

"_chromeUserTiming.domContentLoadedEventStart": 320,

"_chromeUserTiming.domInteractive": 320,

"_chromeUserTiming.domLoading": 285,

"_chromeUserTiming.fetchStart": 48,

"_chromeUserTiming.firstContentfulPaint": 354,

"_chromeUserTiming.firstMeaningfulPaint": 354,

"_chromeUserTiming.firstMeaningfulPaintCandidate": 354,

"_chromeUserTiming.firstPaint": 354,

"_chromeUserTiming.loadEventEnd": 320,

"_chromeUserTiming.loadEventStart": 320,

"_chromeUserTiming.markAsMainFrame": 284,

"_chromeUserTiming.navigationStart": 74,

"_chromeUserTiming.responseEnd": 277,

"_chromeUserTiming.unloadEventEnd": 284,

"_chromeUserTiming.unloadEventStart": 284,

Deprecated: https://docs.webpagetest.org/metrics/page-metrics/#visual-metrics
Duplicates: https://docs.webpagetest.org/metrics/page-metrics/#technical-page-metrics

Suggested change

"_chromeUserTiming.CumulativeLayoutShift": 0,

"_chromeUserTiming.LargestContentfulPaint": 354,

"_chromeUserTiming.LargestTextPaint": 354,

"_chromeUserTiming.TotalLayoutShift": 0,

"_chromeUserTiming.commitNavigationEnd": 285,

"_chromeUserTiming.domComplete": 320,

"_chromeUserTiming.domContentLoadedEventEnd": 320,

"_chromeUserTiming.domContentLoadedEventStart": 320,

"_chromeUserTiming.domInteractive": 320,

"_chromeUserTiming.domLoading": 285,

"_chromeUserTiming.fetchStart": 48,

"_chromeUserTiming.firstContentfulPaint": 354,

"_chromeUserTiming.firstMeaningfulPaint": 354,

"_chromeUserTiming.firstMeaningfulPaintCandidate": 354,

"_chromeUserTiming.firstPaint": 354,

"_chromeUserTiming.loadEventEnd": 320,

"_chromeUserTiming.loadEventStart": 320,

"_chromeUserTiming.markAsMainFrame": 284,

"_chromeUserTiming.navigationStart": 74,

"_chromeUserTiming.responseEnd": 277,

"_chromeUserTiming.unloadEventEnd": 284,

"_chromeUserTiming.unloadEventStart": 284,

"_chromeUserTiming.CumulativeLayoutShift": 0,

"_chromeUserTiming.LargestContentfulPaint": 354,

"_chromeUserTiming.LargestTextPaint": 354,

"_chromeUserTiming.TotalLayoutShift": 0,

"_chromeUserTiming.commitNavigationEnd": 285,

"_chromeUserTiming.domComplete": 320,

"_chromeUserTiming.domContentLoadedEventEnd": 320,

"_chromeUserTiming.domContentLoadedEventStart": 320,

"_chromeUserTiming.domInteractive": 320,

"_chromeUserTiming.domLoading": 285,

"_chromeUserTiming.fetchStart": 48,

"_chromeUserTiming.firstContentfulPaint": 354,

"_chromeUserTiming.firstMeaningfulPaint": 354,

"_chromeUserTiming.firstMeaningfulPaintCandidate": 354,

"_chromeUserTiming.firstPaint": 354,

"_chromeUserTiming.loadEventEnd": 320,

"_chromeUserTiming.loadEventStart": 320,

"_chromeUserTiming.markAsMainFrame": 284,

"_chromeUserTiming.navigationStart": 74,

"_chromeUserTiming.responseEnd": 277,

"_chromeUserTiming.unloadEventEnd": 284,

"_chromeUserTiming.unloadEventStart": 284,

Suggested change

"_chromeUserTiming.CumulativeLayoutShift": 0,

"_chromeUserTiming.LargestContentfulPaint": 354,

"_chromeUserTiming.LargestTextPaint": 354,

"_chromeUserTiming.TotalLayoutShift": 0,

"_chromeUserTiming.commitNavigationEnd": 285,

"_chromeUserTiming.domComplete": 320,

"_chromeUserTiming.domContentLoadedEventEnd": 320,

"_chromeUserTiming.domContentLoadedEventStart": 320,

"_chromeUserTiming.domInteractive": 320,

"_chromeUserTiming.domLoading": 285,

"_chromeUserTiming.fetchStart": 48,

"_chromeUserTiming.firstContentfulPaint": 354,

"_chromeUserTiming.firstMeaningfulPaint": 354,

"_chromeUserTiming.firstMeaningfulPaintCandidate": 354,

"_chromeUserTiming.firstPaint": 354,

"_chromeUserTiming.loadEventEnd": 320,

"_chromeUserTiming.loadEventStart": 320,

"_chromeUserTiming.markAsMainFrame": 284,

"_chromeUserTiming.navigationStart": 74,

"_chromeUserTiming.responseEnd": 277,

"_chromeUserTiming.unloadEventEnd": 284,

"_chromeUserTiming.unloadEventStart": 284,

"_chromeUserTiming.CumulativeLayoutShift": 0,

"_chromeUserTiming.LargestContentfulPaint": 354,

"_chromeUserTiming.LargestTextPaint": 354,

"_chromeUserTiming.TotalLayoutShift": 0,

"_chromeUserTiming.commitNavigationEnd": 285,

"_chromeUserTiming.firstContentfulPaint": 354,

"_chromeUserTiming.firstMeaningfulPaintCandidate": 354,

"_chromeUserTiming.firstPaint": 354,

"_chromeUserTiming.markAsMainFrame": 284,

"_chromeUserTiming.unloadEventEnd": 284,

"_chromeUserTiming.unloadEventStart": 284,

max-ostapenko · 2024-10-10T09:57:02Z

src/content/docs/reference/structs/page-payload.md

+    "_renderBlockingJS": 0,
+    "_requests": 2,
+    "_requestsDoc": 2,
+    "_requestsFull": 2,


Suggested change

"_requestsFull": 2,

Duplicates: https://docs.webpagetest.org/metrics/page-metrics/#technical-page-metrics

max-ostapenko · 2024-10-10T09:57:30Z

src/content/docs/reference/structs/page-payload.md

+    "_domElements": 12,
+    "_domInteractive": 280,
+    "_domLoading": 0,
+    "_domTime": 0,


Suggested change

"_domTime": 0,

Deprecated: https://docs.webpagetest.org/metrics/page-metrics/#technical-page-metrics

pruned summary

1ea8aff

max-ostapenko changed the title ~~Pruned summary description~~ Datasets and schema updates Sep 30, 2024

max-ostapenko added 5 commits September 30, 2024 14:45

custom metrics split

ca1d132

cleaned up payload

85f7b40

summary pruned

4fd99a1

structs

87388eb

queries update

a2a0dfa

max-ostapenko marked this pull request as ready for review September 30, 2024 16:20

rviscomi marked this pull request as draft October 7, 2024 13:46

max-ostapenko added 4 commits October 9, 2024 17:14

update schemas

69ef369

update schema

edee4c6

examples updated

52a6718

types cleanup

4677c68

max-ostapenko commented Oct 10, 2024

View reviewed changes

max-ostapenko mentioned this pull request Oct 10, 2024

Duplicated and deprecated WPT metrics HTTPArchive/httparchive.org#936

Open

max-ostapenko added 2 commits October 11, 2024 23:00

crawl and page ids removed

666a5f8

page ids removed from metadata

2b3f2cd

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Datasets and schema updates #15

Datasets and schema updates #15

max-ostapenko commented Sep 19, 2024 •

edited

Loading

rviscomi commented Oct 7, 2024

max-ostapenko Oct 10, 2024

max-ostapenko Oct 10, 2024

max-ostapenko Oct 10, 2024

max-ostapenko Oct 10, 2024

max-ostapenko Oct 10, 2024

max-ostapenko Oct 10, 2024

max-ostapenko Oct 10, 2024

max-ostapenko Oct 10, 2024

max-ostapenko Oct 10, 2024

max-ostapenko Oct 10, 2024

max-ostapenko Oct 10, 2024

Datasets and schema updates #15

Are you sure you want to change the base?

Datasets and schema updates #15

Conversation

max-ostapenko commented Sep 19, 2024 • edited Loading

rviscomi commented Oct 7, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

max-ostapenko commented Sep 19, 2024 •

edited

Loading