Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Azure TDX (preview) support #170

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion attestation-agent/attester/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,28 @@ edition = "2021"

[dependencies]
anyhow.workspace = true
async-trait.workspace = true
az-snp-vtpm = { git = "https://github.com/kinvolk/azure-cvm-tooling", rev = "2c2e411", default-features = false, features = ["attester"], optional = true }
base64.workspace = true
ioctl-sys = { version = "0.8.0", optional = true }
log.workspace = true
occlum_dcap = { git = "https://github.com/occlum/occlum", rev = "dbe404f", optional = true }
raw-cpuid = { version = "11", optional = true }
reqwest = { version = "0.11", features = ["json"], optional = true }
serde.workspace = true
serde_json.workspace = true
sev = { git = "https://github.com/virtee/sev", rev = "3dca05d2c93388cb00534ad18f5928fd812e99cc", optional = true }
strum.workspace = true
tdx-attest-rs = { git = "https://github.com/intel/SGXDataCenterAttestationPrimitives", rev = "cc582e8be0c9010295c66fb58c59f74744017600", optional = true }

[dev-dependencies]
tokio = { version = "1", features = ["macros"] }

[features]
default = ["all-attesters"]
all-attesters = ["tdx-attester", "occlum-attester", "az-snp-vtpm-attester", "snp-attester"]

tdx-attester = ["tdx-attest-rs"]
tdx-attester = ["tdx-attest-rs", "dep:reqwest", "dep:ioctl-sys", "dep:raw-cpuid"]
Copy link
Member

@Xynnn007 Xynnn007 Jul 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking about this: if more CSPs are going to add their attester code under tdx, like tdx/az, tdx/ali, tdx/another-company. They all shares a same feature here, but must introduce some dependencies they will not use, e.g. CoCo native does not need reqwest, ioctl-sys and raw-cpuid.

The advantage is the community version binary of attestation-agent can be easy to build (only one!)
The disadvantage is big footprint, which will influence the performance and cost.

I do not have a clear solution here, and maybe we should bring a more robust verified build pipeline (CD) of the components of CoCo Community to solve the trustiness here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think for community builds we should support as many different attesters as possible, so that it is easy to try out coco.

But we will need to have ways to disable/enable things that pull in a lot of dependencies so that companies don't need to build in code that will never be used in their environment. We'll add them over time.

Of these dependencies reqwest is the largest, but it is also already used by kbs_protocol.

I do not have a clear solution here, and maybe we should bring a more robust verified build pipeline (CD) of the components of CoCo Community to solve the trustiness here.

👍

occlum-attester = ["occlum_dcap"]
az-snp-vtpm-attester = ["az-snp-vtpm"]
snp-attester = ["sev"]
3 changes: 2 additions & 1 deletion attestation-agent/attester/src/az_snp_vtpm/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,9 @@ struct Evidence {
vcek: String,
}

#[async_trait::async_trait]
impl Attester for AzSnpVtpmAttester {
fn get_evidence(&self, report_data: String) -> Result<String> {
async fn get_evidence(&self, report_data: String) -> Result<String> {
let report = vtpm::get_report()?;
let report_data_bin = base64::decode(report_data)?;
let quote = vtpm::get_quote(&report_data_bin)?;
Expand Down
5 changes: 3 additions & 2 deletions attestation-agent/attester/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ impl Tee {
match self {
Tee::Sample => Ok(Box::<sample::SampleAttester>::default()),
#[cfg(feature = "tdx-attester")]
Tee::Tdx => Ok(Box::<tdx::TdxAttester>::default()),
Tee::Tdx => Ok(tdx::make_attester()),
#[cfg(feature = "occlum-attester")]
Tee::SgxOcclum => Ok(Box::<sgx_occlum::SgxOcclumAttester>::default()),
#[cfg(feature = "az-snp-vtpm-attester")]
Expand All @@ -57,8 +57,9 @@ impl Tee {
}
}

#[async_trait::async_trait]
pub trait Attester {
fn get_evidence(&self, report_data: String) -> Result<String>;
async fn get_evidence(&self, report_data: String) -> Result<String>;
}

// Detect which TEE platform the KBC running environment is.
Expand Down
3 changes: 2 additions & 1 deletion attestation-agent/attester/src/sample/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,9 @@ struct SampleQuote {
#[derive(Debug, Default)]
pub struct SampleAttester {}

#[async_trait::async_trait]
impl Attester for SampleAttester {
fn get_evidence(&self, report_data: String) -> Result<String> {
async fn get_evidence(&self, report_data: String) -> Result<String> {
let evidence = SampleQuote {
svn: "1".to_string(),
report_data,
Expand Down
9 changes: 5 additions & 4 deletions attestation-agent/attester/src/sgx_occlum/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,9 @@ struct SgxOcclumAttesterEvidence {
#[derive(Debug, Default)]
pub struct SgxOcclumAttester {}

#[async_trait::async_trait]
impl Attester for SgxOcclumAttester {
fn get_evidence(&self, report_data: String) -> Result<String> {
async fn get_evidence(&self, report_data: String) -> Result<String> {
let mut report_data_bin = base64::decode(report_data)?;
if report_data_bin.len() != 48 {
bail!("Occlum SGX Attester: Report data should be SHA384 base64 String");
Expand Down Expand Up @@ -56,13 +57,13 @@ mod tests {
use super::*;

#[ignore]
#[test]
fn test_sgx_get_evidence() {
#[tokio::test]
async fn test_sgx_get_evidence() {
let attester = SgxOcclumAttester::default();
let report_data: Vec<u8> = vec![0; 48];
let report_data_base64 = base64::encode(report_data);

let evidence = attester.get_evidence(report_data_base64);
let evidence = attester.get_evidence(report_data_base64).await;
assert!(evidence.is_ok());
}
}
3 changes: 2 additions & 1 deletion attestation-agent/attester/src/snp/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,9 @@ struct SnpEvidence {
#[derive(Debug, Default)]
pub struct SnpAttester {}

#[async_trait::async_trait]
impl Attester for SnpAttester {
fn get_evidence(&self, report_data: String) -> Result<String> {
async fn get_evidence(&self, report_data: String) -> Result<String> {
let mut report_data_bin = base64::decode(report_data)?;

if report_data_bin.len() != 48 {
Expand Down
105 changes: 105 additions & 0 deletions attestation-agent/attester/src/tdx/az.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
// Copyright (c) Microsoft Corporation.
//
// SPDX-License-Identifier: Apache-2.0
//

use anyhow::*;
use ioctl_sys::ioctl;
use raw_cpuid::{CpuId, Hypervisor};
use serde::{Deserialize, Serialize};
use std::fs::OpenOptions;
use std::io::Error;
use std::os::unix::io::AsRawFd;

use crate::Attester;

// Length of the REPORTDATA used in TDG.MR.REPORT TDCALL
const TDX_REPORTDATA_LEN: usize = 64;

// Length of TDREPORT used in TDG.MR.REPORT TDCALL
const TDX_REPORT_LEN: usize = 1024;

#[repr(C)]
#[derive(Debug)]
pub struct TdxReportReq {
reportdata: [u8; TDX_REPORTDATA_LEN],
tdreport: [u8; TDX_REPORT_LEN],
}

impl TdxReportReq {
pub fn new(reportdata: [u8; TDX_REPORTDATA_LEN]) -> Self {
Self {
reportdata,
tdreport: [0; TDX_REPORT_LEN],
}
}
}

#[derive(Serialize, Deserialize)]
struct AzTdQuoteRequest {
report: String,
}
#[derive(Serialize, Deserialize, Debug)]
struct AzTdQuoteResponse {
quote: String,
}

impl Default for TdxReportReq {
fn default() -> Self {
Self {
reportdata: [0; TDX_REPORTDATA_LEN],
tdreport: [0; TDX_REPORT_LEN],
}
}
}

ioctl!(readwrite tdx_cmd_get_report0 with b'T', 0x01; TdxReportReq);

pub(super) async fn get_hyperv_tdx_evidence(report_data: &[u8]) -> Result<String> {
let file = OpenOptions::new().write(true).open("/dev/tdx_guest")?;
let fd = file.as_raw_fd();
let mut tdx_req = TdxReportReq::new(report_data.try_into()?);
unsafe {
let err = tdx_cmd_get_report0(fd, &mut tdx_req);
if err != 0 {
bail!("TDX Attester: ioctl failed: {}", Error::last_os_error());
}
}
let report = base64::encode_config(tdx_req.tdreport, base64::URL_SAFE_NO_PAD);
let tdquotereq = AzTdQuoteRequest { report };
let req = reqwest::Client::new()
.post("http://169.254.169.254/acc/tdquote")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems that the conversion from report to quote doesn't follow the standard way, s.t. call to host. Why?

Copy link
Member Author

@jepio jepio Jun 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately it doesn't look like there is a single standard way for TDX, see this https://lore.kernel.org/lkml/cover.1684048511.git.sathyanarayanan.kuppuswamy@linux.intel.com/:

Although attestation software can use communication methods like TCP/IP or vsock to send the TDREPORT to QE, not all platforms support these communication models. So TDX GHCI specification [1] defines a method for Quote generation via hypercalls.

It looks like TCP/IP using the IMDS service as a proxy to QE was the best way to implement the communication for preview. It might look different for GA.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the QE on the same platform/host? I mean the conversion from report to quote rely on the verification of the MAC of the report, and the key can only be generated on the same platform via EGETKEY.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think yes, the QE VM is on the same node as the TD VM.

.header(reqwest::header::CONTENT_TYPE, "application/json")
.header(reqwest::header::ACCEPT, "application/json")
.body(serde_json::to_string(&tdquotereq)?)
.send()
.await?;
let tdquoteresp = req.json::<AzTdQuoteResponse>().await?;
let quote = base64::decode_config(tdquoteresp.quote, base64::URL_SAFE_NO_PAD)?;
let evidence = super::TdxEvidence {
cc_eventlog: None,
quote: base64::encode(quote),
};
serde_json::to_string(&evidence).context("TDX Attester: Failed to serialize evidence")
}

pub(super) fn detect_platform() -> bool {
// check cpuid if we are in a Hyper-V guest
let cpuid = CpuId::new();
if let Some(hypervisor) = cpuid.get_hypervisor_info() {
hypervisor.identify() == Hypervisor::HyperV
} else {
false
}
}

#[derive(Debug, Default)]
pub struct TdxAttester {}

#[async_trait::async_trait]
impl Attester for TdxAttester {
async fn get_evidence(&self, report_data: String) -> Result<String> {
let report_data_bin = super::convert_report_data(report_data)?;
get_hyperv_tdx_evidence(&report_data_bin).await
}
}
44 changes: 31 additions & 13 deletions attestation-agent/attester/src/tdx/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,20 @@

use super::Attester;
use anyhow::*;
use raw_cpuid::cpuid;
use serde::{Deserialize, Serialize};
use std::path::Path;
use tdx_attest_rs;

mod az;

const CCEL_PATH: &str = "/sys/firmware/acpi/tables/data/CCEL";

pub fn detect_platform() -> bool {
Path::new("/dev/tdx-attest").exists() || Path::new("/dev/tdx-guest").exists()
const TDX_CPUID_LEAF: u32 = 0x21;
let c = cpuid!(TDX_CPUID_LEAF, 0);
let tdbytes = [c.ebx, c.edx, c.ecx].map(|x| x.to_le_bytes()).concat();
let tdstring = String::from_utf8_lossy(&tdbytes);
tdstring == "IntelTDX "
jepio marked this conversation as resolved.
Show resolved Hide resolved
}

#[derive(Serialize, Deserialize)]
Expand All @@ -24,18 +30,30 @@ struct TdxEvidence {
quote: String,
}

pub fn make_attester() -> Box<dyn Attester + Sync + Send> {
if az::detect_platform() {
Box::<az::TdxAttester>::default()
} else {
Box::<TdxAttester>::default()
}
}

fn convert_report_data(report_data: String) -> Result<Vec<u8>> {
let mut report_data_bin = base64::decode(report_data)?;
if report_data_bin.len() != 48 {
bail!("TDX Attester: Report data should be SHA384 base64 String");
}
report_data_bin.extend([0; 16]);
Ok(report_data_bin)
}

#[derive(Debug, Default)]
pub struct TdxAttester {}

#[async_trait::async_trait]
impl Attester for TdxAttester {
fn get_evidence(&self, report_data: String) -> Result<String> {
let mut report_data_bin = base64::decode(report_data)?;
if report_data_bin.len() != 48 {
return Err(anyhow!(
"TDX Attester: Report data should be SHA384 base64 String"
));
}
report_data_bin.extend([0; 16]);
async fn get_evidence(&self, report_data: String) -> Result<String> {
let report_data_bin = convert_report_data(report_data)?;

let tdx_report_data = tdx_attest_rs::tdx_report_data_t {
d: report_data_bin.as_slice().try_into()?,
Expand Down Expand Up @@ -71,13 +89,13 @@ mod tests {
use super::*;

#[ignore]
#[test]
fn test_tdx_get_evidence() {
#[tokio::test]
async fn test_tdx_get_evidence() {
let attester = TdxAttester::default();
let report_data: Vec<u8> = vec![0; 48];
let report_data_base64 = base64::encode(report_data);

let evidence = attester.get_evidence(report_data_base64);
let evidence = attester.get_evidence(report_data_base64).await;
assert!(evidence.is_ok());
}
}
5 changes: 3 additions & 2 deletions attestation-agent/kbs_protocol/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ impl KbsProtocolWrapper {
})
}

fn generate_evidence(&self) -> Result<Attestation> {
async fn generate_evidence(&self) -> Result<Attestation> {
let key = self
.tee_key
.as_ref()
Expand All @@ -77,6 +77,7 @@ impl KbsProtocolWrapper {

let tee_evidence = attester
.get_evidence(ehd)
.await
.map_err(|e| anyhow!("Get TEE evidence failed: {:?}", e))?;

Ok(Attestation {
Expand Down Expand Up @@ -109,7 +110,7 @@ impl KbsProtocolWrapper {
.http_client()
.post(format!("{kbs_host_url}/{KBS_URL_PREFIX}/attest"))
.header("Content-Type", "application/json")
.json(&self.generate_evidence()?)
.json(&self.generate_evidence().await?)
.send()
.await?;

Expand Down
Loading