Add artifact download and deletion

README.md

@@ -10,7 +10,7 @@ The project combines FastAPI, Docker, Terraform, GitHub Actions, and AWS-compati
 
 The API accepts text artifacts and stores them in an S3-compatible bucket protected by a customer-managed KMS encryption key.
 
-Protected API endpoints require a valid API key supplied through the `X-API-Key` request header. Terraform defines the infrastructure and its security controls as code, while GitHub Actions automatically tests and scans every change.
+Protected API endpoints require a valid API key supplied through the `X-API-Key` request header. Authorized clients can upload, list, retrieve, and delete artifacts. Terraform defines the infrastructure and its security controls as code, while GitHub Actions automatically tests and scans every change.
 
 The project demonstrates:
 
@@ -20,6 +20,7 @@ The project demonstrates:
 - AWS infrastructure provisioning with Terraform
 - Encrypted object storage using S3 and KMS
 - Least-privilege IAM permissions
+- Authenticated artifact upload, listing, retrieval, and deletion
 - Automated unit testing with Pytest
 - Automated LocalStack integration testing
 - Dependency vulnerability scanning with pip-audit
@@ -57,10 +58,11 @@ flowchart LR
 1. A client sends a request to the FastAPI application.
 2. The API checks the `X-API-Key` request header.
 3. Requests with missing or incorrect credentials are rejected.
-4. The API validates the artifact name and content.
-5. Boto3 sends the artifact to the local S3-compatible service.
-6. S3 encrypts the stored object using the customer-managed KMS key.
-7. Authorized clients can list the stored artifact names.
+4. For uploads, the API validates the artifact name and content.
+5. Boto3 sends artifact operations to the local S3-compatible service.
+6. Stored objects are encrypted using the customer-managed KMS key.
+7. Authorized clients can list, retrieve, and delete stored artifacts.
+8. Missing artifacts return a controlled `404 Not Found` response instead of exposing storage-layer errors.
 
 ## Security Controls
 
@@ -75,8 +77,10 @@ flowchart LR
 | Storage security | Public access prevention | All four S3 public-access-block settings are enabled |
 | Transport security | HTTPS-only access | The bucket policy denies requests using insecure transport |
 | Data recovery | Object versioning | S3 bucket versioning preserves previous object versions |
-| Access control | Least privilege | The API role is limited to the required S3 and KMS operations |
+| Access control | Least privilege | The API role is limited to the required S3 list, upload, read, delete, and KMS operations |
+| Encrypted retrieval | KMS decryption | The API role receives `kms:Decrypt` only for the project KMS key |
 | Input security | Request validation | Artifact names and content sizes are restricted using Pydantic |
+| Missing-object handling | Controlled `404` responses | Retrieval and deletion of missing artifacts return `404 Not Found` |
 | Application security | Safe error handling | Storage failures return a controlled HTTP `503` response |
 | Container security | Non-root execution | The API container runs as an unprivileged application user |
 | Network exposure | Localhost binding | API and LocalStack ports are bound to `127.0.0.1` |
@@ -158,6 +162,8 @@ The API key must never be committed to Git or included in screenshots, logs, or
 | `GET` | `/health` | Public | Check whether the API is running | `200 OK` |
 | `POST` | `/artifacts` | API key required | Validate and store a text artifact | `201 Created` |
 | `GET` | `/artifacts` | API key required | List stored artifact names | `200 OK` |
+| `GET` | `/artifacts/{name}` | API key required | Retrieve a stored artifact and its content | `200 OK` |
+| `DELETE` | `/artifacts/{name}` | API key required | Delete a stored artifact | `200 OK` |
 | `GET` | `/docs` | Public | Open the interactive API documentation | `200 OK` |
 
 ### Example Upload Body
@@ -199,13 +205,63 @@ curl \
   http://localhost:8000/artifacts
 ```
 
+### Example Authenticated Artifact Retrieval
+
+```bash
+curl \
+  --header "X-API-Key: $API_KEY" \
+  http://localhost:8000/artifacts/security-report.txt
+```
+
+Expected response:
+
+```json
+{
+  "name": "security-report.txt",
+  "content": "No critical vulnerabilities found."
+}
+```
+
+If the artifact does not exist, the API returns:
+
+```json
+{
+  "detail": "Artifact not found"
+}
+```
+
+with HTTP status `404 Not Found`.
+
+### Example Authenticated Artifact Deletion
+
+```bash
+curl --request DELETE \
+  --header "X-API-Key: $API_KEY" \
+  http://localhost:8000/artifacts/security-report.txt
+```
+
+Expected response:
+
+```json
+{
+  "name": "security-report.txt",
+  "status": "deleted"
+}
+```
+
+If the artifact does not exist, the API returns `404 Not Found`.
+
 ## Testing Strategy
 
-The automated unit test suite contains eight tests covering:
+The automated unit test suite contains twelve tests covering:
 
 - API health checking
 - Successful artifact uploads
 - Artifact listing
+- Successful artifact downloads
+- `404 Not Found` responses for missing downloads
+- Successful artifact deletion
+- `404 Not Found` responses for attempts to delete missing artifacts
 - Rejection of unsafe artifact names
 - Controlled behavior when storage is unavailable
 - Rejection of missing API keys
@@ -228,6 +284,8 @@ Unlike the unit tests, the integration test uses real local infrastructure creat
 - The object uses KMS encryption
 - The temporary test object is deleted afterward
 
+The download and deletion endpoints were also verified manually against the running LocalStack environment to confirm real S3 retrieval, successful deletion, and `404 Not Found` behavior for missing artifacts.
+
 Run the unit tests locally with:
 
 ```bash
@@ -424,8 +482,8 @@ secure-api-devsecops/
 
 - `app/main.py` defines the API endpoints and request validation.
 - `app/security.py` validates API keys.
-- `app/storage.py` connects the application to S3 through Boto3.
-- `infrastructure/` contains the AWS-compatible Terraform configuration.
+- `app/storage.py` connects the application to S3 through Boto3 and implements upload, list, retrieval, and deletion operations.
+- `infrastructure/` contains the AWS-compatible Terraform configuration and least-privilege S3/KMS permissions.
 - `tests/` contains the unit and integration tests.
 - `.github/workflows/ci.yml` defines the CI and security pipeline.
 - `compose.yaml` connects the API container to LocalStack.
@@ -438,13 +496,19 @@ The project currently provides:
 - A containerized FastAPI service
 - Protected artifact-management endpoints
 - API-key authentication
+- Authenticated artifact upload and listing
+- Authenticated artifact retrieval
+- Authenticated artifact deletion with missing-object handling
 - Secure S3-compatible artifact storage
 - Customer-managed KMS encryption
-- Least-privilege IAM resources
+- Least-privilege S3 list, upload, read, and delete permissions
+- KMS decryption permission for encrypted artifact retrieval
+- S3 object versioning for recovery
 - Terraform-based infrastructure provisioning
 - Automated unit and integration testing
 - Automated source, dependency, image, and infrastructure scanning
 - A multi-job GitHub Actions pipeline
+- Pull-request-based development with CI checks before merge
 - Local development without a billing-enabled AWS account
 
 The infrastructure follows AWS-compatible patterns but has not been deployed to a production AWS environment.
@@ -453,11 +517,13 @@ The infrastructure follows AWS-compatible patterns but has not been deployed to
 
 Potential future improvements include:
 
-- Adding object download and deletion endpoints
+- Adding a PostgreSQL database for artifact metadata
 - Replacing the single API key with user-specific authentication and authorization
 - Adding API-key rotation and expiration
 - Adding rate limiting to protect the API from abuse
 - Adding structured security logs and request identifiers
+- Adding code coverage and automated secret scanning
+- Adding S3 lifecycle rules to expire old noncurrent object versions and delete markers for storage cost optimization
 - Adding monitoring, metrics, dashboards, and alerts
 - Deploying the container to Amazon ECS or another managed platform
 - Using GitHub OpenID Connect for secure AWS deployment authentication
@@ -472,13 +538,23 @@ Building this project provided practical experience with:
 
 - Designing a complete DevSecOps workflow
 - Building and containerizing a Python API
-- Protecting endpoints with API-key authentication
+- Protecting endpoints with API key authentication
 - Managing secrets safely in local and CI environments
 - Provisioning AWS-compatible resources using Terraform
 - Applying S3, KMS, IAM, and network security controls
+- Designing least-privilege permissions as application capabilities evolve
+- Implementing authenticated object upload, listing, retrieval, and deletion
+- Handling missing cloud objects with controlled HTTP responses
 - Writing unit and end-to-end integration tests
 - Testing real interactions between FastAPI, Boto3, S3, and KMS
 - Scanning source code, dependencies, containers, and infrastructure
 - Building multi-job GitHub Actions workflows
 - Debugging Docker, WSL, YAML, Terraform, LocalStack, and CI failures
-- Using feature branches and pull requests to deliver changes safely
+- Using feature branches and pull requests to deliver changes safely
+- Considering the tradeoff between S3 versioning for recovery and lifecycle management for future cost optimization
+
+## Author
+
+**Abdulrahman Abuzeid**
+
+GitHub: [abdulrahmancoding](https://github.com/abdulrahmancoding)

app/main.py

@@ -3,7 +3,13 @@
 from pydantic import BaseModel, Field
 
 from app.security import require_api_key
-from app.storage import list_artifacts, upload_artifact
+
+from app.storage import (
+    delete_artifact,
+    download_artifact,
+    list_artifacts,
+    upload_artifact,
+)
 
 
 app = FastAPI(
@@ -60,4 +66,71 @@ def get_artifacts() -> dict[str, list[str]]:
             detail="Storage service unavailable",
         ) from error
 
-    return {"artifacts": names}
+    return {"artifacts": names}
+
+
+@app.get(
+    "/artifacts/{name}",
+    dependencies=[Security(require_api_key)],
+)
+def get_artifact(name: str) -> dict[str, str]:
+    try:
+        content = download_artifact(name)
+    except ClientError as error:
+        if error.response.get("Error", {}).get("Code") in {
+            "NoSuchKey",
+            "404",
+            "NotFound",
+        }:
+            raise HTTPException(
+                status_code=404,
+                detail="Artifact not found",
+            ) from error
+
+        raise HTTPException(
+            status_code=503,
+            detail="Storage service unavailable",
+        ) from error
+    except BotoCoreError as error:
+        raise HTTPException(
+            status_code=503,
+            detail="Storage service unavailable",
+        ) from error
+
+    return {
+        "name": name,
+        "content": content,
+    }
+
+@app.delete(
+    "/artifacts/{name}",
+    dependencies=[Security(require_api_key)],
+)
+def remove_artifact(name: str) -> dict[str, str]:
+    try:
+        delete_artifact(name)
+    except ClientError as error:
+        if error.response.get("Error", {}).get("Code") in {
+            "NoSuchKey",
+            "404",
+            "NotFound",
+        }:
+            raise HTTPException(
+                status_code=404,
+                detail="Artifact not found",
+            ) from error
+
+        raise HTTPException(
+            status_code=503,
+            detail="Storage service unavailable",
+        ) from error
+    except BotoCoreError as error:
+        raise HTTPException(
+            status_code=503,
+            detail="Storage service unavailable",
+        ) from error
+
+    return {
+        "name": name,
+        "status": "deleted",
+    }

app/storage.py

@@ -30,4 +30,26 @@ def upload_artifact(name: str, content: str) -> None:
 
 def list_artifacts() -> list[str]:
     response = get_s3_client().list_objects_v2(Bucket=BUCKET_NAME)
-    return [item["Key"] for item in response.get("Contents", [])]
+    return [item["Key"] for item in response.get("Contents", [])]
+
+
+def download_artifact(name: str) -> str:
+    response = get_s3_client().get_object(
+        Bucket=BUCKET_NAME,
+        Key=name,
+    )
+    return response["Body"].read().decode("utf-8")
+
+
+def delete_artifact(name: str) -> None:
+    client = get_s3_client()
+
+    client.head_object(
+        Bucket=BUCKET_NAME,
+        Key=name,
+    )
+
+    client.delete_object(
+        Bucket=BUCKET_NAME,
+        Key=name,
+    )

infrastructure/main.tf

@@ -121,13 +121,28 @@ data "aws_iam_policy_document" "api_permissions" {
     ]
   }
 
+  statement {
+    sid    = "ReadAndDeleteArtifacts"
+    effect = "Allow"
+
+    actions = [
+      "s3:GetObject",
+      "s3:DeleteObject"
+    ]
+
+    resources = [
+      "${aws_s3_bucket.artifacts.arn}/*"
+    ]
+  }
+
   statement {
     sid    = "UseArtifactEncryptionKey"
     effect = "Allow"
 
     actions = [
       "kms:DescribeKey",
       "kms:Encrypt",
+      "kms:Decrypt",
       "kms:GenerateDataKey"
     ]
 
@@ -139,7 +154,7 @@ data "aws_iam_policy_document" "api_permissions" {
 
 resource "aws_iam_policy" "api_permissions" {
   name        = "secure-api-least-privilege"
-  description = "Allows the API to list and upload encrypted artifacts only"
+  description = "Allows the API to list, upload, download, and delete encrypted artifacts"
   policy      = data.aws_iam_policy_document.api_permissions.json
 }
 data "aws_iam_policy_document" "api_assume_role" {