Skip to content

Commit

Permalink
Update version and improve ReadMe documentation
Browse files Browse the repository at this point in the history 
Updated the version in `Directory.Build.props` from `2.1.0` to `2.3.0`.

Revised `ReadMe.md` for better clarity and usability:
- Enhanced ActorSrcGen description.
- Added a "Getting Started" section with step-by-step instructions.
- Simplified instructions for pipeline setup and usage.
- Provided a generated code example.
- Consolidated benefits of ActorSrcGen and TPL Dataflow.
- Removed redundant explanations about TPL Dataflow and the Actor Model.
- Streamlined acknowledgements section.
  • Loading branch information
@aabs
aabs committed Aug 28, 2024
1 parent 6f19db8 commit 5f56c12
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 145 deletions.
2 changes: 1 addition & 1 deletion Directory.Build.props
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<Project>
<PropertyGroup>
<Version>2.1.0</Version>
<Version>2.3.0</Version>
<PackageReleaseNotes>
</PackageReleaseNotes>
</PropertyGroup>
Expand Down
165 changes: 21 additions & 144 deletions ReadMe.md
View file
Original file line number Diff line number Diff line change
@@ -1,36 +1,17 @@
# Welcome To ActorSrcGen

ActorSrcGen is a C# Source Generator allowing the conversion of simple C#
classes into Dataflow compatible pipelines.
ActorSrcGen is a C# Source Generator that converts simple C# classes into TPL Dataflow-compatible pipelines. It simplifies working with TPL Dataflow by generating boilerplate code to handle errors without interrupting the pipeline, ideal for long-lived processes with ingesters that continually pump messages into the pipeline.

ActorSrcGen simplifies the process of working with TPL Dataflow by generating
the boilerplate needed to safely trap and handle errors without interrupting
the operation of the pipeline. It's normally based on the assumption that
the pipeline will be a long lived process with '*ingesters*' that continually
pump incoming messages into the pipeline.

If you encounter any issues or have any questions, please don't hesitate to
submit an issue report. This helps me understand any problems or limitations of
the project and allows me to address them promptly.
## Getting Started

If you have an idea for a new feature or enhancement, I encourage you to submit
a feature request. Your input will shape the future direction of ActorSrcGen
and help make it even better.

If you have any code changes or improvements you'd like to contribute, I welcome
pull requests (PRs). I will review your changes and
provide feedback, helping you ensure a smooth integration process.


## How Do You Use It?

1. Get the latest version of the package into your project:
1. **Install the package**:

```shell
dotnet add package ActorSrcGen
```

1. From there, development follows a simple process. First declare the pipeline class.
2. **Declare the pipeline class**:

```csharp
[Actor]
Expand All @@ -39,17 +20,14 @@ provide feedback, helping you ensure a smooth integration process.
}
```

The class must be `partial`, since the boilerplate code is added to another part
of the class by the ActorSrcGen Source Generator.
The class must be `partial` to allow the source generator to add boilerplate code.

If you are using Visual Studio, you can see the generated part of the code under the
ActorSrcGen analyzer:

![File1](doc/file1.png)

1. Next, you create some '*ingester*' functions. Ingesters are functions that are able
to receive incoming work from somewhere. This could be requests coming in on a queue or
other async source, or be generated in situ.
3. **Create ingester functions**:

```csharp
[Ingest(1)]
Expand All @@ -60,24 +38,9 @@ provide feedback, helping you ensure a smooth integration process.
}
```

Each ingester defines a `Priority`, and the ingesters are visited in priority order.
The ingestion message pump will preferentially consume from the highest priority ingester
until it no longer yields any messages, at which point it will fall through to the next
highest priority ingester. If nothing comes from any of the ingesters then it will sleep
for a second and them repeat the cycle.

You can define as many ingesters as you like, all feeding into the pipeline, but
remember that the lowest priority ones only get a chance to run if there was nothing
available through any other channel. If you need to implement a more sophisticated load
balancing scheme to pull incoming work from multiple sources, you can do it from outside
of the pipeline instead.

1. The next step is to implement the pipeline functions themselves. These are the steps
in the pipeline that get the TPL Dataflow wrapper generated to link them together and
buffer all their incoming and outgoing data.
Ingesters define a `Priority` and are visited in priority order. If no messages are available, the pipeline sleeps for a second before retrying.

The first pipeline step to implement has the `[FirstStep]` attribute adornment. The
description is not used at present, but will be used in future for logging purposes.
4. **Implement pipeline steps**:

```csharp
[FirstStep("decode incoming poll request")]
Expand All @@ -90,13 +53,9 @@ provide feedback, helping you ensure a smooth integration process.
}
```

The first step is used to control how the interface to the pipeline looks from the
outside world. The pipeline can implement interfaces like `IDataflow<TIn, TOut>` depending
the parameter and return types of the first and last steps. This makes it easy to
treat your pipeline class as just another TPL Dataflow block to be inserted into other
pipelines, as needed.
The first step controls the pipeline's interface. Implement additional steps as needed, ensuring input and output types match.

1. Now implement whatever other steps are needed in the pipeline. The outputs and input types
1. Now **implement other steps** are needed in the pipeline. The outputs and input types
of successive steps need to match.

```csharp
Expand All @@ -110,10 +69,7 @@ provide feedback, helping you ensure a smooth integration process.
}
```

Again, you can have as many of these as you need, with branching done using multiple
`[NextStep]` attributes.

1. Finally, you define a last step, using the `[LastStep]` attribute:
5. **Define the last step**:

```csharp
[LastStep]
Expand All @@ -123,14 +79,8 @@ provide feedback, helping you ensure a smooth integration process.
}
```

As mentioned in the first step method, the return type of this function is used
to influence the interface types. It also helps in creating an *accepter* function that
can be used to get results out of the pipeline.

1. These functions are enough information for ActorSrcGen to be able to generate the
boilerplate around the pipeline connecting the steps using TPL Dataflow.

Here's what will be generated from the above
6. **Generated code example**:

```csharp
using System.Threading.Tasks.Dataflow;
Expand Down Expand Up @@ -252,14 +202,14 @@ provide feedback, helping you ensure a smooth integration process.
```


1. To use the pipeline, you can insert messages directly, using the `Call` or `Cast` methods,
or you can invoke the receiver message pump:
7. **Using the pipeline**:

```csharp
var actor = new MyActor();
var actor = new MyActor(); // this is your pipeline
try
{
// call into the pipeline synchronously
if (actor.Call("""
{ "something": "here" }
"""))
Expand Down Expand Up @@ -288,86 +238,13 @@ provide feedback, helping you ensure a smooth integration process.
```


## What It Does
Its purpose is to simplify the
usage of TPL Dataflow, a library that helps with writing robust and performant
asynchronous and concurrent code in .NET. In this case, the source
generator takes a regular C# class and extends it by generating the necessary
boilerplate code to use TPL Dataflow. The generated code creates a pipeline of
dataflow components that support the actor model. The code that you need to write is
simpler, and therefore much easier to test, since they are generally just pure
functions taking a value and returning a response object.
The generated code includes the necessary wiring to connect the methods of
your class together using the TPL Dataflow. This allows the
methods to be executed in a coordinated and concurrent manner.
Overall, the source generator simplifies the process of using TPL Dataflow by
automatically generating the code that would otherwise need to be written
manually. It saves developers from writing a lot of boilerplate code and allows
them to focus on the core logic of their application.
## Why Bother?
You might be wondering what the architectural benefits of using a model like
this might be.
Writing robust and performant asynchronous and concurrent code in .NET is a
laborious process. TPL Dataflow makes it easier - it "*provides dataflow
components to help increase the robustness of concurrency-enabled applications.
This dataflow model promotes actor-based programming by providing in-process
message passing for coarse-grained dataflow and pipelining tasks*" (see
[docs](https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/dataflow-task-parallel-library)).
ActorSrcGen allows you to take advantage of that model without needing to write
a lot of the necessary boilerplate code.
### The Actor Model
The Actor Model is a programming paradigm that is based on the concept of
actors, which are autonomous units of computation. It has several benefits in
programming:
1. **Concurrency**: Actors can be executed concurrently, allowing for efficient
use of multiple CPU cores. This can lead to significant performance
improvements in systems that require concurrent execution.
1. **Fault tolerance**: Actors can be designed to be fault-tolerant, meaning
that if an actor fails or crashes, it can be restarted without affecting the
rest of the system. This can improve the reliability and availability of the
system.
1. **Encapsulation**: Actors encapsulate their state and behavior, making it
easier to reason about and test the code. This can lead to better code
quality and maintainability.
### TPL Dataflow
The Task Parallel Library (TPL) Dataflow in .NET provides a powerful framework
for building high-throughput systems. Here are some benefits of using TPL
Dataflow for high-throughput systems:
1. **Efficiency**: TPL Dataflow is designed to optimize the execution of tasks
and dataflows. It automatically manages the execution of tasks based on
available resources, reducing unnecessary overhead and maximizing throughput.
1. **Scalability**: TPL Dataflow allows you to easily scale your system by
adding or removing processing blocks. You can dynamically adjust the number
of processing blocks based on the workload, ensuring that your system can
handle varying levels of throughput.
1. **Flexibility**: TPL Dataflow provides a variety of processing blocks, such
as buffers, transform blocks, and action blocks, which can be combined and
customized to fit your specific requirements. This flexibility allows you to
build complex dataflows that can handle different types of data and
processing logic.
## Benefits
- Simplifies TPL Dataflow usage: Automatically generates boilerplate code.
- Concurrency: Efficient use of multiple CPU cores.
- Fault tolerance: Errors in pipeline steps are trapped and handled.
- Encapsulation: Easier to reason about and test code.


## Acknowledgements

The generated source builds atop
[DataflowEx](https://github.com/gridsum/DataflowEx) for a clean stateful
object-oriented wrapper around your pipeline.
With thanks to:
- Gridsum [DataflowEx](https://github.com/gridsum/DataflowEx)
- [Bnaya.SourceGenerator.Template](https://github.com/bnayae/Bnaya.SourceGenerator.Template) (see [article](https://blog.stackademic.com/source-code-generators-diy-f04229c59e1a))
Built on [DataflowEx](https://github.com/gridsum/DataflowEx) and [Bnaya.SourceGenerator.Template](https://github.com/bnayae/Bnaya.SourceGenerator.Template).

0 comments on commit 5f56c12

Please sign in to comment.