Skip to content

Commit

Permalink
Merge pull request #54 from ahaostudy/develop
Browse files Browse the repository at this point in the history
feat: Add method annotations to achieve Dubbo-Java type compatibility
  • Loading branch information
felix021 authored Oct 29, 2023
2 parents b120970 + 7a4b557 commit feac522
Show file tree
Hide file tree
Showing 39 changed files with 5,907 additions and 175 deletions.
68 changes: 45 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,39 +12,61 @@

基于已有的 **dubbo Interface API**[**类型映射**](#类型映射),编写 **api.thrift**。然后使用最新的 kitex 命令行工具和 thriftgo 生成 kitex 的脚手架代码(包括用于编解码的stub代码)。

除了默认的类型映射外,还可以在 **thrift** 中使用 [**方法注解**](#方法注解) 指定请求参数映射的 java 类型。

2. **dubbo -> kitex**

基于已有的 **api.thrift**[**类型映射**](#类型映射),编写 dubbo 客户端代码。

### 类型映射

| thrift 类型 | golang 类型 | hessian2 类型 | java 类型 |
|:------------------:|:----------------:|:-----------:|:----------------------:|
| bool | bool | boolean | java.lang.Boolean |
| byte | int8 | int | java.lang.Byte |
| i16 | int16 | int | java.lang.Short |
| i32 | int32 | int | java.lang.Integer |
| i64 | int64 | long | java.lang.Long |
| double | float64 | double | java.lang.Double |
| string | string | string | java.lang.String |
| binary | []byte | binary | byte[] |
| list\<bool> | []bool | list | List\<Boolean> |
| list\<i32> | []int32 | list | List\<Integer> |
| list\<i64> | []int64 | list | List\<Long> |
| list\<double> | []float64 | list | List\<Double> |
| list\<string> | []string | list | List\<String> |
| map\<bool, bool> | map[bool]bool | map | Map\<Boolean, Boolean> |
| map\<bool, i32> | map[bool]int32 | map | Map\<Boolean, Integer> |
| map\<bool, i64> | map[bool]int64 | map | Map\<Boolean, Long> |
| map\<bool, double> | map[bool]float64 | map | Map\<Boolean, Double> |
| map\<bool, string> | map[bool]string | map | Map\<Boolean, String> |
| thrift 类型 | golang 类型 | hessian2 类型 | 默认 java 类型 | 可拓展 java 类型 |
|:------------------:|:----------------:|:-----------:|:----------------------:|:-------------------------------:|
| bool | bool | boolean | java.lang.Boolean | boolean |
| byte | int8 | int | java.lang.Byte | byte |
| i16 | int16 | int | java.lang.Short | short |
| i32 | int32 | int | java.lang.Integer | int |
| i64 | int64 | long | java.lang.Long | long |
| double | float64 | double | java.lang.Double | double |
| string | string | string | java.lang.String | - |
| binary | []byte | binary | byte[] | - |
| list\<bool> | []bool | list | List\<Boolean> | boolean[] / ArrayList\<Boolean> |
| list\<i32> | []int32 | list | List\<Integer> | int[] / ArrayList\<Integer> |
| list\<i64> | []int64 | list | List\<Long> | long[] / ArrayList\<Long> |
| list\<double> | []float64 | list | List\<Double> | double[] / ArrayList\<Double> |
| list\<string> | []string | list | List\<String> | String[] / ArrayList\<String> |
| map\<bool, bool> | map[bool]bool | map | Map\<Boolean, Boolean> | HashMap\<Boolean, Boolean> |
| map\<bool, i32> | map[bool]int32 | map | Map\<Boolean, Integer> | HashMap\<Boolean, Integer> |
| map\<bool, i64> | map[bool]int64 | map | Map\<Boolean, Long> | HashMap\<Boolean, Long> |
| map\<bool, double> | map[bool]float64 | map | Map\<Boolean, Double> | HashMap\<Boolean, Double> |
| map\<bool, string> | map[bool]string | map | Map\<Boolean, String> | HashMap\<Boolean, String> |

**重要提示**
1. 映射表中的 map 类型并没有被完全列举,当前仅包含经过测试的用例。
请勿在map类型中使用包含 **i8****i16****binary** 的键值。
2. 当前仅支持表格中所记录的 thrift 类型和 java 类型映射。
更多映射关系(例如,**bool** <-> **boolean**)将在未来支持,请参考 [issue](https://github.com/kitex-contrib/codec-dubbo/issues/46)
3. 目前不支持float32,因为它在 thrift 中不是有效的类型。计划在后续迭代中支持该类型。
2. 目前不支持float32,因为它在 thrift 中不是有效的类型。计划在后续迭代中支持该类型。

### 方法注解

DubboCodec 支持在 **thrift** 中使用 **方法注解** 指定请求参数需要映射的 java 类型。

**方法注解格式:**
```thrift
(hessian.argsType="req1JavaType,req2JavaType,req3JavaType,...")
```
其中,每个 reqJavaType 可以使用 `-` 或不填写,表示该参数将使用默认的类型映射。

添加方法注解后,使用 kitex 命令行工具生成代码时添加选项 `-thrift with_reflection`,会在生成的脚手架代码中包含 thrift 的 **FileDescriptor**
在初始化 client 时使用 DubboCodec 提供的 `WithFileDescriptor` Option,传入生成的 **FileDescriptor**,即可指定 **kitex -> dubbo** 的类型映射。

**举例:**
```thrift
service EchoService {
EchoResponse Echo(1: i32 req1, 2: list<i32> req2, 3: map<i32, i32> req3) (hessian.argsType="int,int[],java.util.HashMap")
// 使用默认的类型映射
EchoDefaultTypeResponse EchoDefaultType(1: i32 req1, 2: i64 req2, 3: bool req3, 4: string req4) (hessian.argsType=",-,,-")
}
```

## 开始

Expand Down
69 changes: 45 additions & 24 deletions README_ENG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,40 +14,61 @@ English | [中文](README.md)
Write **api.thrift** based on existing **dubbo Interface API** and [**Type Mapping Table**](#type-mapping). Then use
the latest kitex command tool and thriftgo to generate Kitex's scaffold (including stub code).

In addition to the default type mapping, you can also specify the Java type for request parameter mapping in **thrift** using [**Method Annotation**](#method-annotation).

2. **dubbo -> kitex**

Write dubbo client code based on existing **api.thrift** and [**Type Mapping Table**](#type-mapping).

### Type Mapping

| thrift type | golang type | hessian2 type | java type |
|:------------------:|:----------------:|:-------------:|:----------------------:|
| bool | bool | boolean | java.lang.Boolean |
| byte | int8 | int | java.lang.Byte |
| i16 | int16 | int | java.lang.Short |
| i32 | int32 | int | java.lang.Integer |
| i64 | int64 | long | java.lang.Long |
| double | float64 | double | java.lang.Double |
| string | string | string | java.lang.String |
| binary | []byte | binary | byte[] |
| list\<bool> | []bool | list | List\<Boolean> |
| list\<i32> | []int32 | list | List\<Integer> |
| list\<i64> | []int64 | list | List\<Long> |
| list\<double> | []float64 | list | List\<Double> |
| list\<string> | []string | list | List\<String> |
| map\<bool, bool> | map[bool]bool | map | Map\<Boolean, Boolean> |
| map\<bool, i32> | map[bool]int32 | map | Map\<Boolean, Integer> |
| map\<bool, i64> | map[bool]int64 | map | Map\<Boolean, Long> |
| map\<bool, double> | map[bool]float64 | map | Map\<Boolean, Double> |
| map\<bool, string> | map[bool]string | map | Map\<Boolean, String> |
| thrift type | golang type | hessian2 type | default java type | extendable java type |
|:------------------:|:----------------:|:-------------:|:----------------------:|:-------------------------------:|
| bool | bool | boolean | java.lang.Boolean | boolean |
| byte | int8 | int | java.lang.Byte | byte |
| i16 | int16 | int | java.lang.Short | short |
| i32 | int32 | int | java.lang.Integer | int |
| i64 | int64 | long | java.lang.Long | long |
| double | float64 | double | java.lang.Double | double |
| string | string | string | java.lang.String | - |
| binary | []byte | binary | byte[] | - |
| list\<bool> | []bool | list | List\<Boolean> | boolean[] / ArrayList\<Boolean> |
| list\<i32> | []int32 | list | List\<Integer> | int[] / ArrayList\<Integer> |
| list\<i64> | []int64 | list | List\<Long> | long[] / ArrayList\<Long> |
| list\<double> | []float64 | list | List\<Double> | double[] / ArrayList\<Double> |
| list\<string> | []string | list | List\<String> | String[] / ArrayList\<String> |
| map\<bool, bool> | map[bool]bool | map | Map\<Boolean, Boolean> | HashMap\<Boolean, Boolean> |
| map\<bool, i32> | map[bool]int32 | map | Map\<Boolean, Integer> | HashMap\<Boolean, Integer> |
| map\<bool, i64> | map[bool]int64 | map | Map\<Boolean, Long> | HashMap\<Boolean, Long> |
| map\<bool, double> | map[bool]float64 | map | Map\<Boolean, Double> | HashMap\<Boolean, Double> |
| map\<bool, string> | map[bool]string | map | Map\<Boolean, String> | HashMap\<Boolean, String> |

**Important notes**:
1. The list of map types is not exhaustive and includes only tested cases.
Please do not use keys and values with **i8**, **i16** and **binary**.
2. Currently only the thrift type and java type mappings documented in the table are supported.
More mappings(e.g. **bool** <-> **boolean**) would be supported in the future.
Please see [issue](https://github.com/kitex-contrib/codec-dubbo/issues/46).
3. float32 is planned but currently not supported since it's not a valid type in thrift.
2. float32 is planned but currently not supported since it's not a valid type in thrift.

### Method Annotation

DubboCodec supports specifying the Java types needed for request parameter mapping in **thrift** using **method annotations**.

**Method Annotation Format:**
```thrift
(hessian.argsType="req1JavaType,req2JavaType,req3JavaType,...")
```
Here, each `reqJavaType` can either be left blank or use a `-`, indicating that the default type mapping will be used for that parameter.

After adding method annotations, use the kitex command line tool to generate code and add the option `-thrift with_reflection`. This will include the **FileDescriptor** of thrift in the generated scaffold code.
When initializing the client, use DubboCodec's `WithFileDescriptor` Option, and pass in the generated **FileDescriptor** to specify the **kitex -> dubbo** type mapping.

**Example:**
```thrift
service EchoService {
EchoResponse Echo(1: i32 req1, 2: list<i32> req2, 3: map<i32, i32> req3) (hessian.argsType="int,int[],java.util.HashMap")
// Use the default type mapping
EchoDefaultTypeResponse EchoDefaultType(1: i32 req1, 2: i64 req2, 3: bool req3, 4: string req4) (hessian.argsType=",-,,-")
}
```

## Getting Started

Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ require (
dubbo.apache.org/dubbo-go/v3 v3.1.0
github.com/apache/dubbo-go-hessian2 v1.12.2
github.com/cloudwego/kitex v0.6.2-0.20230815060351-88ea60530d40
github.com/cloudwego/thriftgo v0.3.0
github.com/dubbogo/tools v1.0.9
github.com/stretchr/testify v1.8.2
golang.org/x/net v0.17.0 // indirect
Expand Down
4 changes: 0 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -1465,7 +1465,6 @@ golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm
golang.org/x/crypto v0.0.0-20210711020723-a769d52b0f97/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20211108221036-ceb1ce70b4fa/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.7.0 h1:AvwMYaRytfdeVt3u6mLaxYtErKYjxA2OXjJ1HHq6t3A=
golang.org/x/crypto v0.7.0/go.mod h1:pYwdfH91IfpZVANVyUOhSIPZaFoJGxTFbZhFTx+dXZU=
golang.org/x/crypto v0.14.0 h1:wBqGXzWJW6m1XrIKlAH0Hs1JJ7+9KBwnIO8v66Q9cHc=
golang.org/x/crypto v0.14.0/go.mod h1:MVFd36DqK4CsrnJYDkBA3VC4m2GkXAM0PvzMCn4JQf4=
Expand Down Expand Up @@ -1599,7 +1598,6 @@ golang.org/x/net v0.0.0-20221014081412-f15817d10f9b/go.mod h1:YDH+HFinaLZZlnHAfS
golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
golang.org/x/net v0.4.0/go.mod h1:MBQ8lrhLObU/6UmLb4fmbmk5OcyYmqtbGd/9yIeKjEE=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.8.0 h1:Zrh2ngAOFYneWTAIAPethzeaQLuHwhuBkuV6ZiRnUaQ=
golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
Expand Down Expand Up @@ -1764,7 +1762,6 @@ golang.org/x/sys v0.0.0-20220908164124-27713097b956/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0 h1:MVltZSvRTcU2ljQOhs94SXPftV6DCNnZViHeQps87pQ=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.13.0 h1:Af8nKPmuFypiUBjVoU9V20FiaFXOcuZI21p0ycVYYGE=
Expand Down Expand Up @@ -1793,7 +1790,6 @@ golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.5.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.6.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.8.0 h1:57P1ETyNKtuIjB4SRd15iJxuhj8Gc416Y78H3qgMh68=
golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k=
Expand Down
18 changes: 16 additions & 2 deletions pkg/codec.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@ var _ remote.Codec = (*DubboCodec)(nil)

// DubboCodec NewDubboCodec creates the dubbo codec.
type DubboCodec struct {
opt *Options
opt *Options
methodCache hessian2.MethodCache
}

// NewDubboCodec creates a new codec instance.
Expand Down Expand Up @@ -227,7 +228,9 @@ func (m *DubboCodec) messageData(message remote.Message, e iface.Encoder) error
if !ok {
return fmt.Errorf("invalid data: not hessian2.MessageWriter")
}
types, err := hessian2.GetTypes(data)

typeAnno := m.getTypeAnnotation(message)
types, err := m.methodCache.GetTypes(data, typeAnno)
if err != nil {
return err
}
Expand Down Expand Up @@ -264,6 +267,17 @@ func (m *DubboCodec) messageAttachment(ctx context.Context, service *dubbo_spec.
return e.Encode(attachment)
}

func (m *DubboCodec) getTypeAnnotation(message remote.Message) *hessian2.TypeAnnotation {
var typeAnno *hessian2.TypeAnnotation
methodKey := message.ServiceInfo().ServiceName + "." + message.RPCInfo().To().Method()
if m.opt.TypeAnnotations != nil {
if t, ok := m.opt.TypeAnnotations[methodKey]; ok {
typeAnno = t
}
}
return typeAnno
}

// Unmarshal decode method
func (m *DubboCodec) Decode(ctx context.Context, message remote.Message, in remote.ByteBuffer) error {
// parse header part
Expand Down
43 changes: 43 additions & 0 deletions pkg/hessian2/annotation.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
/*
* Copyright 2023 CloudWeGo Authors
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package hessian2

import "strings"

// TypeAnnotation is used to store and parse a type annotation.
type TypeAnnotation struct {
anno string
fieldTypes []string
}

// NewTypeAnnotation is used to create a type annotation object.
func NewTypeAnnotation(anno string) *TypeAnnotation {
ta := &TypeAnnotation{anno: anno}
ta.fieldTypes = strings.Split(ta.anno, ",")
return ta
}

// GetFieldType retrieves the type annotation for a field by its index.
func (ta *TypeAnnotation) GetFieldType(i int) string {
if ta != nil && len(ta.fieldTypes) > i {
return ta.fieldTypes[i]
}
return ""
}
2 changes: 2 additions & 0 deletions pkg/hessian2/const.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,6 @@ import hessian "github.com/apache/dubbo-go-hessian2"

const (
NULL = hessian.BC_NULL

HESSIAN_ARGS_TYPE_TAG = "hessian.argsType"
)
Loading

0 comments on commit feac522

Please sign in to comment.