Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ui): logs view #506

Merged
merged 49 commits into from
Nov 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
2384e4e
WIP
UncleGedd Oct 29, 2024
b2502be
Update src/pkg/api/handlers.go
UncleGedd Oct 29, 2024
92ca73c
WIP: making progress on dropdown
UncleGedd Oct 29, 2024
c435c42
WIP: this dropdown...
UncleGedd Oct 30, 2024
cb47f55
dropdowns working as expected
UncleGedd Oct 30, 2024
0c1af08
log view looking good
UncleGedd Oct 31, 2024
03263ad
WIP: ugly code but working pretty well
UncleGedd Oct 31, 2024
7d3b68d
WIP: dont miss logs
UncleGedd Oct 31, 2024
5d145b1
WIP: remove ticker
UncleGedd Oct 31, 2024
f57b039
WIP: tiny issue with container dropdown
UncleGedd Nov 1, 2024
795577c
WIP: swap reactive statements for stores
UncleGedd Nov 1, 2024
efb7844
WIP: can pass query params
UncleGedd Nov 1, 2024
3ac7976
autoscroll working
UncleGedd Nov 1, 2024
33f819d
copy working
UncleGedd Nov 1, 2024
05960a3
copy and download working
UncleGedd Nov 1, 2024
84ac247
search works
UncleGedd Nov 1, 2024
4a3253f
refactor search
UncleGedd Nov 2, 2024
5b7b09b
labeled dropdown
UncleGedd Nov 2, 2024
63a1c81
css
UncleGedd Nov 4, 2024
49707ee
css
UncleGedd Nov 4, 2024
58b5ec1
Merge branch 'main' into logs-view
UncleGedd Nov 5, 2024
1bf08d8
refactors backend and adds tests
UncleGedd Nov 5, 2024
8fe2148
update swagger
UncleGedd Nov 5, 2024
1c4a7d0
cleans up frontend
UncleGedd Nov 5, 2024
4c9b26e
more frontend cleanup
UncleGedd Nov 5, 2024
8a551c7
WIP: adding FE tests
UncleGedd Nov 5, 2024
0cf5112
adds FE e2e tests
UncleGedd Nov 6, 2024
9cc5b80
fix unit tests
UncleGedd Nov 6, 2024
761fb5e
fix autoscroll bug when changing containers
UncleGedd Nov 6, 2024
136720c
deploy metrics-server in e2e-in-cluster test
UncleGedd Nov 6, 2024
5c1726f
ensure we use latest client and update e2e test
UncleGedd Nov 6, 2024
e6287a3
Update src/pkg/api/logs/logs.go
UncleGedd Nov 6, 2024
da154c4
minor edits
UncleGedd Nov 6, 2024
8eabe03
refactor Go code to remove race condition and ensure cleanup
UncleGedd Nov 6, 2024
46107c3
Merge branch 'main' into logs-view
UncleGedd Nov 6, 2024
b7c486c
remove goroutine
UncleGedd Nov 6, 2024
97ff055
cleanup logs
UncleGedd Nov 6, 2024
0b9537f
Merge branch 'main' into logs-view
UncleGedd Nov 6, 2024
76bc009
fix e2e test
UncleGedd Nov 6, 2024
117c050
make logs header match datatable
UncleGedd Nov 6, 2024
7265dff
rename file and ensure refactor cleanup
UncleGedd Nov 6, 2024
5b27740
more fixes
UncleGedd Nov 6, 2024
ee3c4af
add space to bottom of logs view
UncleGedd Nov 6, 2024
5c4b135
Merge branch 'main' into logs-view
UncleGedd Nov 6, 2024
bf99260
fix
UncleGedd Nov 6, 2024
b342efc
fix api test
UncleGedd Nov 6, 2024
5f0723d
fix test and refactor cluster dropdown
UncleGedd Nov 6, 2024
5987dd6
fix
UncleGedd Nov 6, 2024
d4ba8f1
revert cluster menu dropdown
UncleGedd Nov 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions src/pkg/api/docs/docs.go
Original file line number Diff line number Diff line change
Expand Up @@ -3037,6 +3037,48 @@ const docTemplate = `{
}
}
},
"/api/v1/resources/workloads/pods/logs": {
"get": {
"description": "Get Pod logs",
"consumes": [
"text/html"
],
"produces": [
"application/json"
],
"tags": [
"workloads"
],
"parameters": [
{
"type": "string",
"description": "pod",
"name": "pod",
"in": "query",
"required": true
},
{
"type": "string",
"description": "namespace",
"name": "namespace",
"in": "query",
"required": true
},
{
"type": "string",
"description": "container",
"name": "container",
"in": "query",
"required": true
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
},
"/api/v1/resources/workloads/pods/{uid}": {
"get": {
"description": "Get Pod by UID",
Expand Down
42 changes: 42 additions & 0 deletions src/pkg/api/docs/swagger.json
Original file line number Diff line number Diff line change
Expand Up @@ -3026,6 +3026,48 @@
}
}
},
"/api/v1/resources/workloads/pods/logs": {
"get": {
"description": "Get Pod logs",
"consumes": [
"text/html"
],
"produces": [
"application/json"
],
"tags": [
"workloads"
],
"parameters": [
{
"type": "string",
"description": "pod",
"name": "pod",
"in": "query",
"required": true
},
{
"type": "string",
"description": "namespace",
"name": "namespace",
"in": "query",
"required": true
},
{
"type": "string",
"description": "container",
"name": "container",
"in": "query",
"required": true
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
},
"/api/v1/resources/workloads/pods/{uid}": {
"get": {
"description": "Get Pod by UID",
Expand Down
28 changes: 28 additions & 0 deletions src/pkg/api/docs/swagger.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2070,6 +2070,34 @@ paths:
description: OK
tags:
- workloads
/api/v1/resources/workloads/pods/logs:
get:
consumes:
- text/html
description: Get Pod logs
parameters:
- description: pod
in: query
name: pod
required: true
type: string
- description: namespace
in: query
name: namespace
required: true
type: string
- description: container
in: query
name: container
required: true
type: string
produces:
- application/json
responses:
"200":
description: OK
tags:
- workloads
/api/v1/resources/workloads/statefulsets:
get:
consumes:
Expand Down
16 changes: 16 additions & 0 deletions src/pkg/api/handlers.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (

"github.com/defenseunicorns/uds-runtime/src/pkg/api/auth"
_ "github.com/defenseunicorns/uds-runtime/src/pkg/api/docs" //nolint:staticcheck
"github.com/defenseunicorns/uds-runtime/src/pkg/api/logs"
"github.com/defenseunicorns/uds-runtime/src/pkg/api/resources"
"github.com/defenseunicorns/uds-runtime/src/pkg/api/rest"
"github.com/defenseunicorns/uds-runtime/src/pkg/k8s/client"
Expand Down Expand Up @@ -137,6 +138,21 @@ func getPod(cache *resources.Cache) func(w http.ResponseWriter, r *http.Request)
return rest.Bind(cache.Pods)
}

// @Description Get Pod logs
// @Tags workloads
// @Accept html
// @Produce json
// @Success 200
// @Router /api/v1/resources/workloads/pods/logs [get]
// @Param pod query string true "pod"
// @Param namespace query string true "namespace"
// @Param container query string true "container"
func getPodLogs(k8sSession *session.K8sSession) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
logs.GetLogs(k8sSession.Clients.Clientset, w, r)
}
}

// @Description Get Deployments
// @Tags workloads
// @Accept html
Expand Down
146 changes: 146 additions & 0 deletions src/pkg/api/logs/logs.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
// Copyright 2024 Defense Unicorns
// SPDX-License-Identifier: AGPL-3.0-or-later OR LicenseRef-Defense-Unicorns-Commercial

package logs

import (
"bufio"
"context"
"errors"
"fmt"
"io"
"log/slog"
"net/http"
"strings"
"time"

"github.com/defenseunicorns/uds-runtime/src/pkg/api/rest"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
)

// validateParams checks if all required parameters are present
func validateParams(pod, namespace, container string) error {
if pod == "" || namespace == "" || container == "" {
return errors.New("pod, namespace, and container are required")
}
return nil
}

// streamExistingLogs fetches and streams existing logs
func streamExistingLogs(ctx context.Context, clientSet kubernetes.Interface, w http.ResponseWriter,
pod, namespace, container string, numLines int64) (time.Time, error) {
logStream, err := clientSet.CoreV1().Pods(namespace).GetLogs(pod, &v1.PodLogOptions{
Container: container,
TailLines: &numLines,
}).Stream(ctx)
if err != nil {
return time.Time{}, fmt.Errorf("failed to get existing logs: %w", err)
}
defer logStream.Close()

flusher := w.(http.Flusher)
scanner := bufio.NewScanner(logStream)
for scanner.Scan() {
fmt.Fprintf(w, "data: %s\n\n", scanner.Text())
}
flusher.Flush()

if err := scanner.Err(); err != nil {
return time.Time{}, fmt.Errorf("error reading existing logs: %w", err)
}

return time.Now(), nil
}

// handleLogStream manages the continuous streaming of new logs
func handleLogStream(ctx context.Context, logStream io.ReadCloser, w http.ResponseWriter) error {
defer logStream.Close()

reader := bufio.NewReader(logStream)
flusher := w.(http.Flusher)
buffer := &strings.Builder{}
ticker := time.NewTicker(500 * time.Millisecond)
defer ticker.Stop()

for {
select {
case <-ctx.Done():
slog.Debug("Logs streaming stopped")
return nil

case <-ticker.C:
if buffer.Len() > 0 {
fmt.Fprintf(w, "data: %s\n\n", buffer.String())
buffer.Reset()
flusher.Flush()
}

default:
// Try to read a line
line, err := reader.ReadBytes('\n')
if err != nil {
if err == io.EOF || errors.Is(err, context.Canceled) {
// Final flush if needed
if buffer.Len() > 0 {
fmt.Fprintf(w, "data: %s\n\n", buffer.String())
flusher.Flush()
}
return nil
}
slog.Debug("Reader returned error", "error", err)
return err
}
buffer.Write(line)
}
}
}

// GetLogs streams logs from a pod
func GetLogs(clientSet kubernetes.Interface, w http.ResponseWriter, r *http.Request) {
// Get and validate query parameters
pod := r.URL.Query().Get("pod")
namespace := r.URL.Query().Get("namespace")
container := r.URL.Query().Get("container")

if err := validateParams(pod, namespace, container); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}

rest.WriteHeaders(w)
ctx := r.Context()

// Verify streaming is supported
_, ok := w.(http.Flusher)
if !ok {
http.Error(w, "Streaming unsupported", http.StatusInternalServerError)
return
}

// Stream existing logs first
lastReadTime, err := streamExistingLogs(ctx, clientSet, w, pod, namespace, container, 100)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}

// Set up streaming for new logs
podLogs := clientSet.CoreV1().Pods(namespace).GetLogs(pod, &v1.PodLogOptions{
Container: container,
Follow: true,
SinceTime: &metav1.Time{Time: lastReadTime},
})
UncleGedd marked this conversation as resolved.
Show resolved Hide resolved

logStream, err := podLogs.Stream(ctx)
if err != nil {
http.Error(w, fmt.Sprintf("Failed to open log stream: %v", err), http.StatusInternalServerError)
return
}

if err := handleLogStream(ctx, logStream, w); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
}
Loading
Loading