Direct and secure file-uploads to AWS S3, Google Cloud Storage and others.
meteor add edgee:slingshotThere are many many packages out there that allow file uploads to S3, Google Cloud and other cloud storage services, but they usually rely on the meteor apps' server to relay the files to the cloud service, which puts the server under unnecessary load.
meteor-slingshot uploads the files directly to the cloud service from the browser without ever exposing your secret access key or any other sensitive data to the client and without requiring public write access to cloud storage to the entire public.
File uploads can not only be restricted by file-size and file-type, but also by other stateful criteria such as the current meteor user.
On the client side we can now upload files through to the bucket:
var uploader = new Slingshot.Upload("myFileUploads");
uploader.send(document.getElementById('input').files[0], function (error, downloadUrl) {
if (error) {
// Log service detailed response.
console.error('Error uploading', uploader.xhr.response);
alert (error);
}
else {
Meteor.users.update(Meteor.userId(), {$push: {"profile.files": downloadUrl}});
}
});These file upload restrictions are validated on the client and then appended to the directive on the server side to enforce them:
Slingshot.fileRestrictions("myFileUploads", {
allowedFileTypes: ["image/png", "image/jpeg", "image/gif"],
maxSize: 10 * 1024 * 1024 // 10 MB (use null for unlimited).
});Important: The fileRestrictions must be declared before the the directive is instantiated.
On the server we declare a directive that controls upload access rules:
Slingshot.createDirective("myFileUploads", Slingshot.S3Storage, {
bucket: "mybucket",
acl: "public-read",
authorize: function () {
//Deny uploads if user is not logged in.
if (!this.userId) {
var message = "Please login before posting files";
throw new Meteor.Error("Login Required", message);
}
return true;
},
key: function (file) {
//Store file into a directory by the user's username.
var user = Meteor.users.findOne(this.userId);
return user.username + "/" + file.name;
}
});With the directive above, no other files than images will be allowed. The policy is directed by the meteor app server and enforced by AWS S3.
Note: If your bucket is created in any region other than US Standard, you will need to set the region key in the directive. Refer the AWS Slingshot Storage Directives
The client side is agnostic to which storage service is used. All it needs for the file upload to work, is a directive name.
There is no limit imposed on how many directives can be declared for each storage service.
Storage services are pluggable in Slingshot and you can add support for own storage service as described in a section below.
You can create file upload progress bars as follows:
Using the Slingshot.Upload instance read and react to the progress:
Template.progressBar.helpers({
progress: function () {
return Math.round(this.uploader.progress() * 100);
}
});Template.myPicture.helpers({
url: function () {
//If we are uploading an image, pass true to download the image into cache.
//This will preload the image before using the remote image url.
return this.uploader.url(true);
}
});This to show the image from the local source until it is uploaded to the server.
If Blob URL's are not available it will attempt to use FileReader to generate
a base64-encoded url representing the data as a fallback.
You can add meta-context to your file-uploads, to make your requests more specific on where the files are to be uploaded.
Consider the following example...
We have an app that features picture albums. An album belongs to a user and only that user is allowed to upload picture to it. In the cloud each album has its own directory where its pictures are stored.
We declare our client-side uploader as follows:
var metaContext = {albumId: album._id}
var uploadToMyAlbum = new Slingshot.Upload("picturealbum", metaContext);On the server side the directive can now set the key accordingly and check if the user is allowed post pictures to the given album:
Slingshot.createDirective("picturealbum", Slingshot.GoogleCloud, {
acl: "public-read",
authorize: function (file, metaContext) {
var album = Albums.findOne(metaContext.albumId);
//Denied if album doesn't exist or if it is not owned by the current user.
return album && album.userId === this.userId;
},
key: function (file, metaContext) {
return metaContext.albumId + "/" + Date.now() + "-" + file.name;
}
});You can check if a file uploadable according to file-restrictions as follows:
var uploader = new Slingshot.Upload("myFileUploads");
var error = uploader.validate(document.getElementById('input').files[0]);
if (error) {
console.error(error);
}The validate method will return null if valid and returns an Error instance
if validation fails.
You will need aAWSAccessKeyId and AWSSecretAccessKey in Meteor.settings
and a bucket with the following CORS configuration:
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>HEAD</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>Declare AWS S3 Directives as follows:
Slingshot.createDirective("aws-s3-example", Slingshot.S3Storage, {
//...
});For extra security you can use temporary credentials to sign upload requests.
var sts = new AWS.STS(); // Using the AWS SDK to retrieve temporary credentials.
Slingshot.createDirective('myUploads', Slingshot.S3Storage.TempCredentials, {
bucket: 'myBucket',
temporaryCredentials: Meteor.wrapAsync(function (expire, callback) {
//AWS dictates that the minimum duration must be 900 seconds:
var duration = Math.max(Math.round(expire / 1000), 900);
sts.getSessionToken({
DurationSeconds: duration
}, function (error, result) {
callback(error, result && result.Credentials);
});
})
});If you are running slingshot on an EC2 instance, you can conveniantly retreive
your access keys with AWS.EC2MetadataCredentials:
var credentials = new AWS.EC2MetadataCredentials();
var updateCredentials = Meteor.wrapAsync(credentials.get, credentials);
Slingshot.createDirective('myUploads', Slingshot.S3Storage.TempCredentials, {
bucket: 'myBucket',
temporaryCredentials: function () {
if (credentials.needsRefresh()) {
updateCredentials();
}
return {
AccessKeyId: credentials.accessKeyId,
SecretAccessKey: credentials.secretAccessKey,
SessionToken: credentials.sessionToken
};
}
});Generate a private key and convert it to a .pem file
using openssl:
openssl pkcs12 -in google-cloud-service-key.p12 -nodes -nocerts > google-cloud-service-key.pem
Setup CORS on the bucket:
gsutil cors set docs/gs-cors.json gs://mybucket
Save this file into the /private directory of your meteor app and add this
line to your server-side code:
Slingshot.GoogleCloud.directiveDefault.GoogleSecretKey = Assets.getText('google-cloud-service-key.pem');Declare Google Cloud Storage Directives as follows:
Slingshot.createDirective("google-cloud-example", Slingshot.GoogleCloud, {
//...
});You will need aRackspaceAccountId (your acocunt number) and
RackspaceMetaDataKey in Meteor.settings.
In order to obtain your RackspaceMetaDataKey (a.k.a. Account-Meta-Temp-Url-Key)
you need an
auth-token
and then follow the
instructions here.
Note that API-Key, Auth-Token, Meta-Data-Key are not the same thing:
API-Key is what you need to obtain an Auth-Token, which in turn is what you need to setup CORS and to set your Meta-Data-Key. The auth-token expires after 24 hours.
For your directive you need container and provide its name, region and cdn.
Slingshot.createDirective("rackspace-files-example", Slingshot.RackspaceFiles, {
container: "myContainer", //Container name.
region: "lon3", //Region code (The default would be 'iad3').
//You must set the cdn if you want the files to be publicly accessible:
cdn: "https://abcdefghije8c9d17810-ef6d926c15e2b87b22e15225c32e2e17.r19.cf5.rackcdn.com",
pathPrefix: function (file) {
//Store file into a directory by the user's username.
var user = Meteor.users.findOne(this.userId);
return user.username;
}
});To setup CORS you also need to your Auth-Token from above and use:
curl -I -X POST -H 'X-Auth-Token: yourAuthToken' \
-H 'X-Container-Meta-Access-Control-Allow-Origin: *' \
-H 'X-Container-Meta-Access-Expose-Headers: etag location x-timestamp x-trans-id Access-Control-Allow-Origin' \
https://storage101.containerRegion.clouddrive.com/v1/MossoCloudFS_yourAccoountNumber/yourContainerCurrently the uploader uses XMLHttpRequest 2 to upload the files, which is not
supported on Internet Explorer 9 and older versions of Internet Explorer.
This can be circumvented by falling back to iframe uploads in future versions, if required.
Latency compensation is available in Internet Explorer 10.
The secret key never leaves the meteor app server. Nobody will be able to upload anything to your buckets outside of your meteor app.
Instead of using secret access keys, Slingshot uses a policy document that is sent to along with the file AWS S3 or Google Cloud Storage. This policy is signed by the secret key and contains all the restrictions that you define in the directive. By default a signed policy expires after 5 minutes.
Cloud storage services are pluggable in Slingshot. You can add support for a cloud storage service of your choice. All you need is to declare an object with the following parameters:
MyStorageService = {
/**
* Define the additional parameters that your your service uses here.
*
* Note that some parameters like maxSize are shared by all services. You do
* not need to define those by yourself.
*/
directiveMatch: {
accessKey: String,
options: Object,
foo: Match.Optional(Function)
},
/**
* Here you can set default parameters that your service will use.
*/
directiveDefault: {
options: {}
},
/**
*
* @param {Object} method - This is the Meteor Method context.
* @param {Object} directive - All the parameters from the directive.
* @param {Object} file - Information about the file as gathered by the
* browser.
* @param {Object} [meta] - Meta data that was passed to the uploader.
*
* @returns {UploadInstructions}
*/
upload: function (method, directive, file, meta) {
var accessKey = directive.accessKey;
var fooData = directive.foo && directive.foo.call(method, file, meta);
//Here you need to make sure that all parameters passed in the directive
//are going to be enforced by the server receiving the file.
return {
// Endpoint where the file is to be uploaded:
upload: "https://example.com",
// Download URL, once the file uploaded:
download: directive.cdn || "https://example.com/" + file.name,
// POST data to be attached to the file-upload:
postData: [
{
name: "accessKey",
value: accessKey
},
{
name: "signature",
value: signature
}
//...
],
// HTTP headers to send when uploading:
headers: {
"x-foo-bar": fooData
}
};
},
/**
* Absolute maximum file-size allowable by the storage service.
*/
maxSize: 5 * 1024 * 1024 * 1024
};Example Directive:
Slingshot.createDirective("myUploads", MyStorageService, {
accessKey: "a12345xyz",
foo: function (file, metaContext) {
return "bar";
}
});Meteor core packages:
- underscore
- tracker
- reactive-var
- check
If you are having any queries about how to use slingshot, or how to get it to work with the different services or any other general questions about it, please post a question on Stack Overflow. You will get a high quality answer there much quicker than by posting an issue here on github.
Bug reports, Feature Requests and Pull Requests are always welcome.
authorize: Function (required unless set in File Restrictions)
maxSize: Number (required unless set in File Restrictions)
allowedFileTypes RegExp, String or Array (required unless set in File
Restrictions)
cdn String (optional) - CDN domain for downloads.
i.e. "https://d111111abcdef8.cloudfront.net"
expire Number (optional) - Number of milliseconds in which an upload
authorization will expire after the request was made. Default is 5 minutes.
region String (optional) - Default is Meteor.settings.AWSRegion or
"us-east-1". See AWS Regions
AWSAccessKeyId String (required) - Can also be set in Meteor.settings.
AWSSecretAccessKey String (required) - Can also be set in Meteor.settings.
region String (optional) - Default is Meteor.settings.AWSRegion or
"us-east-1". See AWS Regions
temporaryCredentials Function (required) - Function that generates temporary
credentials. It takes a signle argument, which is the minumum desired expiration
time in milli-seconds and it returns an object that contains AccessKeyId,
SecretAccessKey and SessionToken.
bucket String (required) - Name of bucket to use. The default is
Meteor.settings.GoogleCloudBucket.
GoogleAccessId String (required) - Can also be set in Meteor.settings.
GoogleSecretKey String (required) - Can also be set in Meteor.settings.
bucket String (required) - Name of bucket to use. The default is
Meteor.settings.GoogleCloudBucket. For AWS S3 the default bucket is
Meteor.settings.S3Bucket.
bucketUrl String or Function (optional) - Override URL to which files are
uploaded. If it is a function, then the first argument is the bucket name. This
url also used for downloads unless a cdn is given.
key String or Function (required) - Name of the file on the cloud storage
service. If a function is provided, it will be called with userId in the
context and its return value is used as the key. First argument is file info and
the second is the meta-information that can be passed by the client.
acl String (optional)
cacheControl String (optional) - RFC 2616 Cache-Control directive
contentDisposition String or Function (optional) - RFC 2616
Content-Disposition directive. Default is the uploaded file's name (inline). If
it is a function then it takes the same context and arguments as the key
function. Use null to disable.
RackspaceAccountId String (required) - Can also be set in Meteor.settings.
RackspaceMetaDataKey String (required) - Can also be set in Meteor.settings.
container String (required) - Name of container to use.
region String (optional) - Data Center region. The default is "iad3".
See other regions
pathPrefix String or Function (required) - Similar to key for S3, but
will always be appended by file.name that is provided by the client.
deleteAt Date (optional) - Absolute time when the uploaded file is to be
deleted. This attribute is not enforced at all. It can be easily altered by the
client
deleteAfter Number (optional) - Same as deleteAt, but relative.
authorize Function (optional) - Function to determines if upload is allowed.
maxSize Number (optional) - Maximum file-size (in bytes). Use null or 0
for unlimited.
allowedFileTypes RegExp, String or Array (optional) - Allowed MIME types. Use
null for any file type.