Skip to content

Commit

Permalink
pbench-report-status: fix md5 calculation.
Browse files Browse the repository at this point in the history
Calculate the md5 on the generated JSON document which includes the
timestamp.

Previously, we were calculating it on the input document which did not
include the timestamp: the result was that the md5 could be identical
between different documents (e.g. the same error message generated
during different runs of a script): since the md5 is used as the ES id
for the document, the indexing of the document could fail as a
duplicate.
  • Loading branch information
ndokos committed Dec 18, 2018
1 parent 4802e4b commit acfd035
Show file tree
Hide file tree
Showing 48 changed files with 126 additions and 126 deletions.
4 changes: 2 additions & 2 deletions server/pbench/bin/gold/test-10.txt
Original file line number Diff line number Diff line change
Expand Up @@ -118,8 +118,8 @@ drwxrwxr-x - tmp
--- pbench log file contents
+++ test-execution.log file contents
ssh pbench-satellite.example.com /var/tmp/pbench-test-server/opt/pbench-server-satellite/bin/pbench-sync-package-tarballs
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-sync-satellite.1900-01/status/3563965c0526d7cb8d4f63bc0a7e8453 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/6a6c5afeb29296d3d3b7b4b08e5272f1 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-sync-satellite.1900-01/status/677e594d3775c9c1717661401f858b68 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/2516423d024eea25ec92d3eb82986f8f --data @/tmp/pbench-report-status.NNNN/payload
--- test-execution.log file contents
+++ test-curl-payload.log file contents
{
Expand Down
4 changes: 2 additions & 2 deletions server/pbench/bin/gold/test-11.txt
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,8 @@ drwxrwxr-x - tmp
+++ test-execution.log file contents
ssh pbench-satellite.example.com /var/tmp/pbench-test-server/opt/pbench-server-satellite/bin/pbench-sync-package-tarballs
ssh pbench-satellite.example.com /var/tmp/pbench-test-server/opt/pbench-server-satellite/bin/pbench-satellite-state-change /var/tmp/pbench-test-server/pbench-satellite/archive/fs-version-001
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-sync-satellite.1900-01/status/b9ff6a9857d11ca1d7ba568654de463d --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/8c40f0bc61a9fe48e778edfd8fe9eb57 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-sync-satellite.1900-01/status/a5c51aef0dad0ffdfaec637977add48c --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/ddb8cfb2144d7623ee1dfd01258643e9 --data @/tmp/pbench-report-status.NNNN/payload
--- test-execution.log file contents
+++ test-curl-payload.log file contents
{
Expand Down
2 changes: 1 addition & 1 deletion server/pbench/bin/gold/test-12.txt
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ drwxrwxr-x - tmp
/var/tmp/pbench-test-server/pbench-local/logs/pbench-server-prep-shim-001/pbench-server-prep-shim-001.error:Failed: /var/tmp/pbench-test-server/pbench-local/quarantine does not exist, or is not a directory
--- pbench log file contents
+++ test-execution.log file contents
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/8c40f0bc61a9fe48e778edfd8fe9eb57 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/ddb8cfb2144d7623ee1dfd01258643e9 --data @/tmp/pbench-report-status.NNNN/payload
--- test-execution.log file contents
+++ test-curl-payload.log file contents
{
Expand Down
2 changes: 1 addition & 1 deletion server/pbench/bin/gold/test-13.txt
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ drwxrwxr-x - tmp
/var/tmp/pbench-test-server/pbench-local/logs/pbench-server-prep-shim-002/pbench-server-prep-shim-002.error:Failed: /var/tmp/pbench-test-server/pbench-local/quarantine does not exist, or is not a directory
--- pbench log file contents
+++ test-execution.log file contents
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/8c40f0bc61a9fe48e778edfd8fe9eb57 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/ddb8cfb2144d7623ee1dfd01258643e9 --data @/tmp/pbench-report-status.NNNN/payload
--- test-execution.log file contents
+++ test-curl-payload.log file contents
{
Expand Down
2 changes: 1 addition & 1 deletion server/pbench/bin/gold/test-14.txt
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ drwxrwxr-x - tmp
/var/tmp/pbench-test-server/pbench-local/logs/pbench-server-prep-shim-001/pbench-server-prep-shim-001.error:Failed: /var/tmp/pbench-test-server/pbench-local/pbench-move-results-receive/fs-version-001 does not exist, or is not a directory
--- pbench log file contents
+++ test-execution.log file contents
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/8c40f0bc61a9fe48e778edfd8fe9eb57 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/ddb8cfb2144d7623ee1dfd01258643e9 --data @/tmp/pbench-report-status.NNNN/payload
--- test-execution.log file contents
+++ test-curl-payload.log file contents
{
Expand Down
2 changes: 1 addition & 1 deletion server/pbench/bin/gold/test-15.txt
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ drwxrwxr-x - tmp
/var/tmp/pbench-test-server/pbench-local/logs/pbench-server-prep-shim-002/pbench-server-prep-shim-002.error:Failed: /var/tmp/pbench-test-server/pbench-local/pbench-move-results-receive/fs-version-002 does not exist, or is not a directory
--- pbench log file contents
+++ test-execution.log file contents
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/8c40f0bc61a9fe48e778edfd8fe9eb57 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/ddb8cfb2144d7623ee1dfd01258643e9 --data @/tmp/pbench-report-status.NNNN/payload
--- test-execution.log file contents
+++ test-curl-payload.log file contents
{
Expand Down
4 changes: 2 additions & 2 deletions server/pbench/bin/gold/test-16.txt
Original file line number Diff line number Diff line change
Expand Up @@ -200,8 +200,8 @@ drwxrwxr-x - tmp
/var/tmp/pbench-test-server/pbench-local/logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log:run-1900-01-01T00:00:00-UTC: Processed 3 tarballs
--- pbench log file contents
+++ test-execution.log file contents
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-unpack-tarballs.1900-01/status/65acd420d4a5bac8c4df9eee8d85dc4c --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/8c40f0bc61a9fe48e778edfd8fe9eb57 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-unpack-tarballs.1900-01/status/852c4be38b544973890b4c4c05f26716 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/ddb8cfb2144d7623ee1dfd01258643e9 --data @/tmp/pbench-report-status.NNNN/payload
--- test-execution.log file contents
+++ test-curl-payload.log file contents
{
Expand Down
4 changes: 2 additions & 2 deletions server/pbench/bin/gold/test-17.txt
Original file line number Diff line number Diff line change
Expand Up @@ -210,8 +210,8 @@ drwxrwxr-x - tmp
/var/tmp/pbench-test-server/pbench-local/logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log:run-1900-01-01T00:00:00-UTC: Processed 3 tarballs
--- pbench log file contents
+++ test-execution.log file contents
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-unpack-tarballs.1900-01/status/65acd420d4a5bac8c4df9eee8d85dc4c --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/8c40f0bc61a9fe48e778edfd8fe9eb57 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-unpack-tarballs.1900-01/status/852c4be38b544973890b4c4c05f26716 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/ddb8cfb2144d7623ee1dfd01258643e9 --data @/tmp/pbench-report-status.NNNN/payload
--- test-execution.log file contents
+++ test-curl-payload.log file contents
{
Expand Down
2 changes: 1 addition & 1 deletion server/pbench/bin/gold/test-18.txt
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ drwxrwxr-x - tmp
+++ pbench log file contents
--- pbench log file contents
+++ test-execution.log file contents
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/8c40f0bc61a9fe48e778edfd8fe9eb57 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/ddb8cfb2144d7623ee1dfd01258643e9 --data @/tmp/pbench-report-status.NNNN/payload
--- test-execution.log file contents
+++ test-curl-payload.log file contents
{
Expand Down
4 changes: 2 additions & 2 deletions server/pbench/bin/gold/test-19.txt
Original file line number Diff line number Diff line change
Expand Up @@ -185,8 +185,8 @@ drwxrwxr-x - tmp
/var/tmp/pbench-test-server/pbench-local/logs/pbench-move-unpacked/pbench-move-unpacked.log:run-1900-01-01T00:00:00-UTC: Processed 4 tar balls
--- pbench log file contents
+++ test-execution.log file contents
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-move-unpacked.1900-01/status/2674b387afce7521b6bf50a09ffc2a77 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/8c40f0bc61a9fe48e778edfd8fe9eb57 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-move-unpacked.1900-01/status/36aa920baf9d156918ecab1d9e3496a5 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/ddb8cfb2144d7623ee1dfd01258643e9 --data @/tmp/pbench-report-status.NNNN/payload
--- test-execution.log file contents
+++ test-curl-payload.log file contents
{
Expand Down
2 changes: 1 addition & 1 deletion server/pbench/bin/gold/test-20.txt
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ drwxrwxr-x - tmp
+++ pbench log file contents
--- pbench log file contents
+++ test-execution.log file contents
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/505f80d5913a207aecb34405238d8405 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/91cbb977a978978c937c8ba573b43b73 --data @/tmp/pbench-report-status.NNNN/payload
--- test-execution.log file contents
+++ test-curl-payload.log file contents
{
Expand Down
4 changes: 2 additions & 2 deletions server/pbench/bin/gold/test-21.txt
Original file line number Diff line number Diff line change
Expand Up @@ -97,8 +97,8 @@ drwxrwxr-x - tmp
/var/tmp/pbench-test-server/pbench-local/logs/pbench-dispatch/pbench-dispatch.log:run-1900-01-01T00:00:00-UTC: Processed 0 result tar balls, 0 successfully (0 partial), with 0 errors, and 0 duplicates
--- pbench log file contents
+++ test-execution.log file contents
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-dispatch.1900-01/status/657c08ef9549227093a6fa057d037507 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/8c40f0bc61a9fe48e778edfd8fe9eb57 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-dispatch.1900-01/status/b344dce76d92d2d0d160eb6ba3c0021c --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/ddb8cfb2144d7623ee1dfd01258643e9 --data @/tmp/pbench-report-status.NNNN/payload
--- test-execution.log file contents
+++ test-curl-payload.log file contents
{
Expand Down
4 changes: 2 additions & 2 deletions server/pbench/bin/gold/test-22.txt
Original file line number Diff line number Diff line change
Expand Up @@ -119,8 +119,8 @@ drwxrwxr-x - tmp
/var/tmp/pbench-test-server/pbench-local/logs/pbench-satellite-cleanup/pbench-satellite-cleanup.log:run-1900-01-01T00:00:00-UTC: pbench-satellite-cleanup ends: Total 6 tarballs cleaned up, with 0 tarball removal errors, 0 md5 file remove errors, 0 state change errors, 0 incoming removal errors, 0 result removal errors and 0 prefix removal errors.
--- pbench log file contents
+++ test-execution.log file contents
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-satellite-cleanup.1900-01/status/2e17f1381c38036b7b8355938bfaa31b --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/5410fceac5a16fbd3293a70c3f2e24de --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-satellite-cleanup.1900-01/status/d1da1e859906a2a5071f65952edac397 --data @/tmp/pbench-report-status.NNNN/payload
curl --silent --show-error -XPOST -H Content-Type: application/json --output /tmp/pbench-report-status.NNNN/response_body --write-out %{http_code} http://elasticsearch.example.com:9280/pbench-unittests.pbench-audit-server.1900-01/status/56eede3d765d9995569bc09785c9ab4a --data @/tmp/pbench-report-status.NNNN/payload
--- test-execution.log file contents
+++ test-curl-payload.log file contents
{
Expand Down
Loading

0 comments on commit acfd035

Please sign in to comment.