Skip to content

Commit

Permalink
docs: update README and Contribution Guide and add Code of Conduct
Browse files Browse the repository at this point in the history 
Include the same `CONTRIBUTING.md` and `CODE_OF_CONDUCT.md` we have been using for other repos.

Ref: #8
Semver: patch
  • Loading branch information
Samir Musali committed Apr 27, 2021
1 parent ea6caac commit e5ababb
Show file tree
Hide file tree
Showing 5 changed files with 267 additions and 65 deletions.
138 changes: 138 additions & 0 deletions CODE_OF_CONDUCT.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
# Contributor's Code of Conduct

If you contribute to this repo, you agree to abide by this code of conduct for
this community.

We abide by the
[Contributor Covenant Code of Conduct, 2.0](https://www.contributor-covenant.org/version/2/0/code_of_conduct/).
It is reproduced below for ease of use.

# Contributor Covenant Code of Conduct

## Our Pledge

We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.

We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.

## Our Standards

Examples of behavior that contributes to a positive environment for our
community include:

* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community

Examples of unacceptable behavior include:

* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting

## Enforcement Responsibilities

Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.

Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.

## Scope

This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
[opensource@logdna.com](mailto:opensource@logdna.com).
All complaints will be reviewed and investigated promptly and fairly.

All community leaders are obligated to respect the privacy and security of the
reporter of any incident.

## Enforcement Guidelines

Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:

### 1. Correction

**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.

**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.

### 2. Warning

**Community Impact**: A violation through a single incident or series
of actions.

**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.

### 3. Temporary Ban

**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.

**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.

### 4. Permanent Ban

**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.

**Consequence**: A permanent ban from any sort of public interaction within
the community.

## Attribution

This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.

Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder](https://github.com/mozilla/diversity).

[homepage]: https://www.contributor-covenant.org

For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
https://www.contributor-covenant.org/translations.

55 changes: 42 additions & 13 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
@@ -1,15 +1,44 @@
# Contributing

## Github Workflow

Contributions are always welcome! Be sure to follow the [github workflow](https://guides.github.com/introduction/flow/) when contributing to this project:

* Create an issue, or comment on an issue to indicate what you are working on. This avoids work duplication.
* Fork the repository and clone to your local machine
* You should already be on the default branch `master` - if not, check it out (`git checkout master`)
* Create a new branch for your feature/fix `git checkout -b my-new-feature`)
* Write your feature/fix
* Stage the changed files for a commit (`git add .`)
* Commit your files with a *useful* commit message ([example](https://github.com/Azure/azure-quickstart-templates/commit/53699fed9983d4adead63d9182566dec4b8430d4)) (`git commit`)
* Push your new branch to your GitHub Fork (`git push origin my-new-feature`)
* Visit this repository in GitHub and create a Pull Request.
## Process

We use a fork-and-PR process, also known as a triangular workflow. This process
is fairly common in open-source projects. Here's the basic workflow:

1. Fork the upstream repo to create your own repo. This repo is called the origin repo.
2. Clone the origin repo to create a working directory on your local machine.
3. Work your changes on a branch in your working directory, then add, commit, and push your work to your origin repo.
4. Submit your changes as a PR against the upstream repo. You can use the upstream repo UI to do this.
5. Maintainers review your changes. If they ask for changes, you work on your
origin repo's branch and then submit another PR. Otherwise, if no changes are made,
then the branch with your PR is merged to upstream's main trunk, the main branch.

When you work in a triangular workflow, you have the upstream repo, the origin
repo, and then your working directory (the clone of the origin repo). You do
a `git fetch` from upstream to local, push from local to origin, and then do a PR from origin to
upstream—a triangle.

If this workflow is too much to understand to start, that's ok! You can use
GitHub's UI to make a change, which is autoset to do most of this process for
you. We just want you to be aware of how the entire process works before
proposing a change.

Thank you for your contributions; we appreciate you!

## License

Note that we use a standard [MIT](./LICENSE) license on this repo.

## Coding style

Code style is enforced by [eslint][]. Linting is applied CI builds when a pull request
is made. The rule set being enforced is provided by [eslint-config-logdna][]

## Questions?

The easiest way to get our attention is to comment on an existing, or open a new
[issue][].

[eslint]: https://eslint.org
[eslint-config-logdna]: https://github.com/logdna/eslint-config-logdna
[issue]: https://github.com/logdna/eslint-config-logdna/issues
115 changes: 64 additions & 51 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,62 +2,75 @@

The LogDNA Amazon S3 integration relies on [AWS Lambda](https://docs.aws.amazon.com/lambda/index.html) to route your logs from [S3](https://docs.aws.amazon.com/AmazonS3/latest/dev/Welcome.html) to LogDNA.

## Configure the LogDNA AWS Lambda function
## How to Use
### Deploy the Code
1. Create a [new AWS Lambda function](https://console.aws.amazon.com/lambda/home) and select `Author from scratch`
2. For the basic information:
* Function Name: `logdna_s3` (you can choose what to name it)
* Runtime: `Node.js.10.x`
3. Click on the Lambda function to edit the details:
2. Click on the Lambda function to edit the details:
* Code entry type: `Upload a .ZIP file`
* Upload our LogDNA Lambda function [.ZIP File](https://github.com/logdna/logdna-s3/releases/latest/download/logdna-s3.zip)
* Handler: `index.handler`
* Runtime: `Node.js.10.x`
* Environment variables:
* `LOGDNA_KEY`: `YOUR_INGESTION_KEY_HERE` *(Required)*
* `LOGDNA_HOSTNAME`: Alternative Host Name *(Optional)*
* `LOGDNA_TAGS`: Comma-separated Tags *(Optional)*
* `LOGDNA_URL`: Custom Ingestion URL *(Optional)*
4. For Execution role, assign a role that has the following policies:
* Runtime: `Node.js.14.x`

### Configuration
#### General Configuration
If the lambda is being used to stream from gzipped files:
1. Set `Timeout` to, at least, `30 seconds`
2. Set `Memory` limit to, at least, `512 MB`

**Notes**:
* The number of retries is recommended to be set to 0 since retrying lambda execution can result in duplicate logs. It can be modified in `Configuration > Asynchronous invoication`
* `Timeout` and `Memory` limit may need to be increased if the file size is too big

#### Triggers
Add `S3` as a trigger with the following configuration:
* Specify the `bucket` you want to stream from
* Specify the event types you want to capture
* Optional prefix and suffix options are related to object paths within the specified bucket

**Notes**:
* This S3 Lambda function and S3 Bucket must be in the same availability zone
* You can specify a bucket only in one trigger and/or S3 Lambda function since a bucket accepts only one subscription but one S3 Lambda function can stream from multiple buckets at the same time

#### Permissions
For Execution role, assign a role that has the following policies:
* [`AmazonS3ReadOnlyAccess`](https://gist.github.com/bernadinm/6f68bfdd015b3f3e0a17b2f00c9ea3f8#file-all_aws_managed_policies-json-L4392-L4417)
* [`AWSLambdaBasicExecutionRole`](https://gist.github.com/bernadinm/6f68bfdd015b3f3e0a17b2f00c9ea3f8#file-all_aws_managed_policies-json-L1447-L1473)
![Policies](https://raw.githubusercontent.com/logdna/artwork/master/logdna-s3/permissions.png)
5. It is recommended to set the `Timeout` to 30 seconds if the function is going to be used to stream the logs from `gzipped` files. You may change `Memory (MB)` limit as well depending on how heavy those files are going to be

### Configure your AWS S3 Bucket
You have the option of connecting your Amazon S3 Bucket within the S3 Lambda function console or in your S3 console.

### In the S3 Lambda Function
1. Add S3 as a Trigger and click it to Configure:
![Configure](https://raw.githubusercontent.com/logdna/artwork/master/logdna-s3/designer.png)
2. Select the `Bucket` and `Event type` to stream the logs from to LogDNA
a. *Optional* You can also specify `Prefix` and `Suffix` for the files to capture
3. Make sure you check `Enable trigger` box:
![Trigger](https://raw.githubusercontent.com/logdna/artwork/master/logdna-s3/trigger.png)
4. Repeat steps 1-3 to add multiple buckets

### Optional Environment Variables
The following variables can be used to tune this S3 Lambda function for specific use cases.

* **LOGDNA_BATCH_INTERVAL**: How frequently (in `milliseconds`) to flush the batch of logs, *Optional*
* **Default**: 50
* **LOGDNA_BATCH_LIMIT**: The maximum number of logs in one batch, *Optional*
* **Default**: 25
* **LOGDNA_FREE_SOCKET_TIMEOUT**: How long (in `milliseconds`) to wait for inactivity before timing out on the free socket, *Optional*
* **Default**: 300000
* **Source**: [agentkeepalive#agentoptions](https://github.com/node-modules/agentkeepalive/blob/master/README.md#new-agentoptions)
* **LOGDNA_MAX_LINE_LENGTH**: The maximum character length for each line, *Optional*
* **Default**: 32000
* **LOGDNA_MAX_REQUEST_RETRIES**: The maximum number of retries for sending a line when there are network failures, *Optional*
* **Default**: 5
* **LOGDNA_MAX_REQUEST_TIMEOUT**: Time limit (in `seconds`) for requests made by this HTTP Client, *Optional*
* **Default**: 300
* **LOGDNA_REQUEST_RETRY_INTERVAL**: How frequently (in `milliseconds`) to retry for sending a line when there are network failures, *Optional*
* **Default**: 100

### Notes
* This S3 Lambda function and S3 Bucket should be in the same availability zone
* You can specify a bucket only in one trigger and/or S3 Lambda function since a bucket accepts only one subscription
* `Node.js.10.x` is the minimum runtime requirement for successfully running this S3 Lambda function

#### Environment Variables
Set `INGESTION_KEY` variable to your LogDNA ingestion key.

**Notes**:
* For more information about required and optional environment variables, please, check out [this](./doc/env.md)

#### Monitoring
Enabling monitoring means forwarding the metrics and logs about the execution of the S3 Lambda function to `CloudWatch`. You can also use [`logdna-cloudwatch`](github.com/logdna/logdna-cloudwatch) to monitor the performance of this S3 Lambda function.

### Test
You can test the configuration and code package using the following test input containing the bucket name and the path to the file you want to test the lambda on:
```json
{
"Records": [
{
"s3": {
"bucket": {
"name": "<bucket-name>"
},
"object": {
"key": "<object-path>"
}
}
}
]
}
```

## Migration Guide
If you have been using `v1` and want to switch to using the newer versions, check out [this guide](./doc/v2.md)

## License
Copyright © [LogDNA](https://logdna.com), released under an MIT license. See the [LICENSE](./LICENSE) file and https://opensource.org/licenses/MIT

## Contributing
Contributions are always welcome. See the [contributing guide](/CONTRIBUTING.md) to learn how you can help. Build instructions for the agent are also in the guide.
Contributions are always welcome. See the [contributing guide](/CONTRIBUTING.md) to learn how you can help.

*Happy Logging!*
22 changes: 22 additions & 0 deletions doc/v2.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
## Guide for Migration from v1 to v2

`v2` contains lots of refactoral changes to improve the performance and configuration by introducing [`@logdna/logger`](https://www.npmjs.com/package/@logdna/logger), [`@logdna/env-config`](https://www.npmjs.com/package/@logdna/env-config), and [`eslint-config-logdna`](https://www.npmjs.com/package/eslint-config-logdna). There are several changes in naming and handling environment variables as well. All environment variables are documented [here](./env.md)

### Renamed Variables
* `LOGDNA_KEY` has been replaced with [`INGESTION_KEY`](./env.md#ingestion_key)
* `LOGDNA_HOSTNAME` has been replaced with [`HOSTNAME`](./env.md#hostname)
* `LOGDNA_TAGS` has been replaced with [`TAGS`](./env.md#tags)
* `LOGDNA_URL` has been split into multiple new variables: [`INGESTION_ENDPOINT`](./env.md#ingestion_endpoint), [`INGESTION_HOST`](./env.md#ingestion_host), [`INGESTION_PORT`](./env.md#ingestion_port), and [`SSL`](./env.md#ssl).
* `LOGDNA_BATCH_INTERVAL` has been replaced with [`FLUSH_INTERVAL`](./env.md#flush_interval)
* `LOGDNA_BATCH_LIMIT` has been replaced with [`FLUSH_LIMIT`](./env.md#flush_limit)

### Removed Variables
* `LOGDNA_FREE_SOCKET_TIMEOUT` has been removed since it gets handled within the logger client
* `LOGDNA_MAX_LINE_LENGTH` has been removed since a line length gets handled in the server-side
* `LOGDNA_MAX_REQUEST_RETRIES` has been removed since it gets handled within the logger client
* `LOGDNA_MAX_REQUEST_TIMEOUT` has been removed since it gets handled within the logger client
* `LOGDNA_REQUEST_RETRY_INTERVAL` has been removed since it gets handled within the logger client

### New Variables
* [`PROXY`](./env.md#proxy), [`HTTPS_PROXY`](./env.md#https_proxy), and [`HTTP_PROXY`](./env.md#http_proxy) to provide proxy support
* [`USER_AGENT`](./env.md#user_agent) to support custom `user-agent` overrides
2 changes: 1 addition & 1 deletion index.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,5 +50,5 @@ async function handler(event, context) {

// Ensure logs have been flushed to LogDNA before finishing
await once(logger, 'cleared')
return
return 'Done!'
}

0 comments on commit e5ababb

Please sign in to comment.