Skip to content
This repository has been archived by the owner on Oct 17, 2023. It is now read-only.

Commit

Permalink
Prepare Release 1.0.0
Browse files Browse the repository at this point in the history
  • Loading branch information
anthonyalberto committed Nov 18, 2021
1 parent 1dbb7ad commit f1bfe79
Show file tree
Hide file tree
Showing 5 changed files with 39 additions and 20 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@
/pipeline.js
/transformer.js
.vscode
dist
22 changes: 18 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,25 @@
## v1.0.0 [TBD]

This release does not introduce any new functionalities, but catches up transporter with modern golang.
It also introduces a new standard way of running adaptor tests.
Only the mongodb and postgresql adaptors have been fully tested. We'll add more as we go.

- Bumped golang to 1.17
- Migrated to go mod
- Migrate from Travis CI to Github Actions
- Standardize how adaptor tests are run. Using docker-compose, we can run them the same way on CI and locally.
- Provide docker-compose and custom scripts to run mongodb and postgresql adaptor tests locally and on CI. See the respective adaptor README for more info.
- Fix for postgresql tailer when there are new lines in a record, via [#456](https://github.com/compose/transporter/pull/456), thanks @erft-engineering!
- Reduce mongo bulk operation size to 1000, via [#450](https://github.com/compose/transporter/pull/450), thanks @trinchan!

## v0.5.0 [2017-09-11]

### Features
- new `remap` function for changing the namespace with a native function versus needing a custom
JS function to do it, fixed via [#400](https://github.com/compose/transporter/pull/400)
- new `opfilter` function for whitelisting or blacklisting message operations versus needing a custom
- new `opfilter` function for whitelisting or blacklisting message operations versus needing a custom
JS function to do it, fixed via [#401](https://github.com/compose/transporter/pull/401)
- `elasticsearch` adaptor now supports parent/child relationships, see
- `elasticsearch` adaptor now supports parent/child relationships, see
fixed via [#406](https://github.com/compose/transporter/pull/406), thanks @johnjjung!

### Bugfixes
Expand All @@ -15,7 +29,7 @@ fixed via [#403](https://github.com/compose/transporter/pull/403)

### Changes
- the `-log_dir` flag is now `-xlog_dir` and the `log_dir` config property is now `xlog_dir` to match. This is to avoid confusion with the application logging's -log flags.
- `elasticsearch` adaptor now logs all errors returned from a bulk operation, [#413](https://github.com/compose/transporter/pull/413),
- `elasticsearch` adaptor now logs all errors returned from a bulk operation, [#413](https://github.com/compose/transporter/pull/413),
thanks @pmjhonwang!

## v0.4.0 [2017-08-15]
Expand All @@ -33,7 +47,7 @@ from others collections to be sent down the pipeline, fixed via [#386](https://g
## v0.3.1 [2017-03-24]

### Features
- added `js` alias for `goja` transform function [#335](https://github.com/compose/transporter/pull/335)
- added `js` alias for `goja` transform function [#335](https://github.com/compose/transporter/pull/335)
- `init` command will now prompt user when `pipeline.js` file already exists [#336](https://github.com/compose/transporter/pull/336)

### Bugfixes
Expand Down
7 changes: 3 additions & 4 deletions MAINTAINERS
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
Nick Stott <nick@compose.io> (@nstott)
JP Phillips <jp@compose.io> (@jipperinbham)
Kurt Mackey <kurt@compose.io> (@mrkurt)
Andrew Schwartz <andrew@compose.io> (@trinchan)
Anthony Alberto (@anthonyalberto)
Peter May (@pete-may)
Andrew Schwartz (@trinchan)
27 changes: 16 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
[![Build Status](https://travis-ci.org/compose/transporter.svg?branch=master)](https://travis-ci.org/compose/transporter) [![Build Status](https://semaphoreci.com/api/v1/compose/transporter/branches/master/shields_badge.svg)](https://semaphoreci.com/compose/transporter) [![Go Report Card](https://goreportcard.com/badge/github.com/compose/transporter)](https://goreportcard.com/report/github.com/compose/transporter) [![codecov](https://codecov.io/gh/compose/transporter/branch/master/graph/badge.svg)](https://codecov.io/gh/compose/transporter) [![Docker Repository on Quay](https://quay.io/repository/compose/transporter/status "Docker Repository on Quay")](https://quay.io/repository/compose/transporter) [![Gitter](https://img.shields.io/gitter/room/nwjs/nw.js.svg)](https://gitter.im/compose-transporter/Lobby)
[![Go Report Card](https://goreportcard.com/badge/github.com/compose/transporter)](https://goreportcard.com/report/github.com/compose/transporter) [![codecov](https://codecov.io/gh/compose/transporter/branch/master/graph/badge.svg)](https://codecov.io/gh/compose/transporter) [![Docker Repository on Quay](https://quay.io/repository/compose/transporter/status "Docker Repository on Quay")](https://quay.io/repository/compose/transporter) [![Gitter](https://img.shields.io/gitter/room/nwjs/nw.js.svg)](https://gitter.im/compose-transporter/Lobby)

Compose Transporter helps with database transformations from one store to another. It can also sync from one to another or several stores.

Transporter
===========

Warning About v1.0.0
------------

This version officially only supports the mongodb and postgresql adaptors. Support for other DBs will be added later on. Other adaptors may or may not work. You're encouraged to still use v0.5.2 for non mongo/postgres migrations.

About
-----

Expand All @@ -25,7 +30,7 @@ t.Config({"xlog_dir":"/data/transporter"})
.Save("sink", sink)
```

When using the above pipeline, all messages will be appended to a commit log and
When using the above pipeline, all messages will be appended to a commit log and
successful processing of a message is handled via consumer/sink offset tracking.

Below is a list of each adaptor and its support of the feature:
Expand All @@ -34,12 +39,12 @@ Below is a list of each adaptor and its support of the feature:
+---------------+-------------+----------------+
| adaptor | read resume | write tracking |
+---------------+-------------+----------------+
| elasticsearch | N/A | X |
| file | | X |
| mongodb | X | X |
| postgresql | | X |
| rabbitmq | X | |
| rethinkdb | | X |
| elasticsearch | N/A | X |
| file | | X |
| mongodb | X | X |
| postgresql | | X |
| rabbitmq | X | |
| rethinkdb | | X |
+---------------+-------------+----------------+
```

Expand Down Expand Up @@ -130,7 +135,7 @@ elasticsearch - an elasticsearch sink adaptor
file - an adaptor that reads / writes files
mongodb - a mongodb adaptor that functions as both a source and a sink
postgres - a postgres adaptor that functions as both a source and a sink
rabbitmq - an adaptor that handles publish/subscribe messaging with RabbitMQ
rabbitmq - an adaptor that handles publish/subscribe messaging with RabbitMQ
rethinkdb - a rethinkdb adaptor that functions as both a source and a sink
```

Expand Down Expand Up @@ -170,7 +175,7 @@ Prints out the state of connections at the end. Useful for debugging new configu
### xlog

The `xlog` command is useful for inspecting the current state of the commit log.
It contains 3 subcommands, `current`, `oldest`, and `offset`, as well as
It contains 3 subcommands, `current`, `oldest`, and `offset`, as well as
a required flag `-xlog_dir` which should be the path to where the commit log is stored.

***NOTE*** the command should only be run against the commit log when transporter
Expand Down Expand Up @@ -205,7 +210,7 @@ Prints out the entry stored at the provided offset.
### offset

The `offset` command provides access to current state of each consumer (i.e. sink)
offset. It contains 4 subcommands, `list`, `show`, `mark`, and `delete`, as well as
offset. It contains 4 subcommands, `list`, `show`, `mark`, and `delete`, as well as
a required flag `-xlog_dir` which should be the path to where the commit log is stored.

```
Expand Down
2 changes: 1 addition & 1 deletion release.fish
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ echo
set DISTDIR dist/v$VERSION
mkdir -p $DISTDIR

for pair in linux/amd64 darwin/amd64 dragonfly/amd64 freebsd/amd64 netbsd/amd64 openbsd/amd64 windows/amd64
for pair in darwin/arm64 linux/amd64 darwin/amd64 dragonfly/amd64 freebsd/amd64 netbsd/amd64 openbsd/amd64 windows/amd64
set GOOS (echo $pair | cut -d'/' -f1)
set GOARCH (echo $pair | cut -d'/' -f2)
set BIN $DISTDIR/transporter-$VERSION-$GOOS-$GOARCH
Expand Down

0 comments on commit f1bfe79

Please sign in to comment.