System.Threading.Tasks.Dataflow 9.0.0

About

Provides dataflow components that are collectively referred to as the TPL Dataflow Library. This dataflow model promotes actor-based programming by providing in-process message passing for coarse-grained dataflow and pipelining tasks.

Key Features

  • Foundation for message passing and parallelizing CPU-intensive and I/O-intensive applications that have high throughput and low latency.
  • Provides multiple block types for various dataflow operations (e.g., BufferBlock, ActionBlock, TransformBlock).
  • Dataflow blocks support linking to form networks, allowing you to create complex processing topologies.

How to Use

This sample demonstrates a dataflow pipeline that downloads the book "The Iliad of Homer" from a website and searches the text to match individual words with words that reverse the first word's characters.

using System.Net;
using System.Text.RegularExpressions;
using System.Threading.Tasks.Dataflow;

var nonLetterRegex = new Regex(@"\P{L}", RegexOptions.Compiled);
var client = new HttpClient(new HttpClientHandler { AutomaticDecompression = DecompressionMethods.GZip });

// Setup blocks

// Downloads the requested resource as a string.
TransformBlock<string, string> downloadString = new TransformBlock<string, string>(async uri =>
{
    Console.WriteLine("Downloading '{0}'...", uri);

    return await client.GetStringAsync(uri);
});

// Separates the specified text into an array of words.
TransformBlock<string, string[]> createWordList = new TransformBlock<string, string[]>(text =>
{
    Console.WriteLine("Creating word list...");

    // Remove common punctuation by replacing all non-letter characters with a space character.
    text = nonLetterRegex.Replace(text, " ");

    // Separate the text into an array of words.
    return text.Split(new[] { ' ' }, StringSplitOptions.RemoveEmptyEntries);
});

// Removes short words.
TransformBlock<string[], string[]> filterWordList = new TransformBlock<string[], string[]>(words =>
{
    Console.WriteLine("Filtering word list...");

    return words
       .Where(word => word.Length > 3)
       .ToArray();
});

// Finds all words in the specified collection whose reverse also exists in the collection.
TransformManyBlock<string[], string> findReversedWords = new TransformManyBlock<string[], string>(words =>
{
    Console.WriteLine("Finding reversed words...");

    var wordsSet = new HashSet<string>(words);

    return from word in wordsSet
           let reverse = string.Concat(word.Reverse())
           where word != reverse && wordsSet.Contains(reverse)
           select word;
});

// Prints the provided reversed words to the console.
ActionBlock<string> printReversedWords = new ActionBlock<string>(reversedWord =>
{
    Console.WriteLine("Found reversed words {0}/{1}", reversedWord, string.Concat(reversedWord.Reverse()));
});


// Connect the dataflow blocks to form a pipeline.
var linkOptions = new DataflowLinkOptions { PropagateCompletion = true };

downloadString.LinkTo(createWordList, linkOptions);
createWordList.LinkTo(filterWordList, linkOptions);
filterWordList.LinkTo(findReversedWords, linkOptions);
findReversedWords.LinkTo(printReversedWords, linkOptions);

// Post data to the pipeline, "The Iliad of Homer" by Homer.
downloadString.Post("http://www.gutenberg.org/cache/epub/16452/pg16452.txt");

// Mark the head of the pipeline as complete.
downloadString.Complete();

// Wait for the last block in the pipeline to process all messages.
printReversedWords.Completion.Wait();

// Output:
// Downloading 'http://www.gutenberg.org/cache/epub/16452/pg16452.txt'...
// Creating word list...
// Filtering word list...
// Finding reversed words...
// Found reversed words parts/strap
// Found reversed words deer/reed
// Found reversed words deem/meed
// Found reversed words flow/wolf
// ...

More details can be found on Dataflow (Task Parallel Library) and Walkthrough: Creating a Dataflow Pipeline pages.

Main Types

The main types provided by this library are:

  • System.Threading.Tasks.Dataflow.ISourceBlock<TOutput>
  • System.Threading.Tasks.Dataflow.ITargetBlock<TInput>
  • System.Threading.Tasks.Dataflow.IPropagatorBlock<TInput,TOutput>
  • System.Threading.Tasks.Dataflow.ActionBlock<TInput>
  • System.Threading.Tasks.Dataflow.BatchBlock<T>
  • System.Threading.Tasks.Dataflow.BatchedJoinBlock<T1, T2>
  • System.Threading.Tasks.Dataflow.BroadcastBlock<T>
  • System.Threading.Tasks.Dataflow.BufferBlock<T>
  • System.Threading.Tasks.Dataflow.JoinBlock<T1, T2>
  • System.Threading.Tasks.Dataflow.TransformBlock<TInput, TOutput>
  • System.Threading.Tasks.Dataflow.TransformManyBlock<TInput, TOutput>
  • System.Threading.Tasks.Dataflow.WriteOnceBlock<T>

Additional Documentation

Feedback & Contributing

System.Threading.Tasks.Dataflow is released as open source under the MIT license. Bug reports and contributions are welcome at the GitHub repository.

Showing the top 20 packages that depend on System.Threading.Tasks.Dataflow.

Packages Downloads
Microsoft.AspNetCore.NodeServices
Invoke Node.js modules at runtime in ASP.NET Core applications.
37
Microsoft.AspNetCore.NodeServices
Invoke Node.js modules at runtime in ASP.NET Core applications.
39
Microsoft.AspNetCore.NodeServices
Invoke Node.js modules at runtime in ASP.NET Core applications.
42
Microsoft.Build
This package contains the Microsoft.Build assembly which is used to create, edit, and evaluate MSBuild projects.
38
Microsoft.Build
This package contains the Microsoft.Build assembly which is used to create, edit, and evaluate MSBuild projects.
39
Microsoft.Build
This package contains the Microsoft.Build assembly which is used to create, edit, and evaluate MSBuild projects.
40
Microsoft.Build
This package contains the Microsoft.Build assembly which is used to create, edit, and evaluate MSBuild projects.
41
Microsoft.Build
This package contains the Microsoft.Build assembly which is used to create, edit, and evaluate MSBuild projects.
42
Microsoft.Build
This package contains the Microsoft.Build assembly which is used to create, edit, and evaluate MSBuild projects.
52
Microsoft.Build.Tasks.Core
This package contains the Microsoft.Build.Tasks assembly which implements the commonly used tasks of MSBuild.
40
Microsoft.CodeAnalysis.Workspaces.MSBuild
.NET Compiler Platform ("Roslyn") support for analyzing MSBuild projects and solutions. This should be used with at least one of the following packages to add the appropriate language support: - Microsoft.CodeAnalysis.CSharp.Workspaces - Microsoft.CodeAnalysis.VisualBasic.Workspaces More details at https://aka.ms/roslyn-packages This package was built from the source at https://github.com/dotnet/roslyn/commit/e68227ea677b76a3c603bd616f03ea6d952b2458.
39
Microsoft.NETCore
Provides a set of packages that can be used when building portable libraries on .NETCore based platforms. \r\n TFS ID: 1599443, GitHub SHA: https://github.com/dotnet/corefx/tree/eede273a4dfabcea608621f5e1bbf8ad00584cfb
38
Microsoft.NETCore.App
A set of .NET API's that are included in the default .NET Core application model. 612cb8b183c52dff0a7f2e57cbfb58e2f18938cc When using NuGet 3.x this package requires at least version 3.4.
37
Microsoft.NETCore.App
A set of .NET API's that are included in the default .NET Core application model. bb155063c5d3787c70d8f4de4088c7f0f0ef719c When using NuGet 3.x this package requires at least version 3.4.
40
Microsoft.NETCore.App
A set of .NET API's that are included in the default .NET Core application model. cca5d72d48c1dd5633e92a8f2f8f3a4b9f9ff774 When using NuGet 3.x this package requires at least version 3.4.
39
Microsoft.NETCore.App
A set of .NET API's that are included in the default .NET Core application model. When using NuGet 3.x this package requires at least version 3.4.
37

https://go.microsoft.com/fwlink/?LinkID=799421

.NET Framework 4.6.2

  • No dependencies.

.NET 8.0

  • No dependencies.

.NET 9.0

  • No dependencies.

.NET Standard 2.0

  • No dependencies.

.NET Standard 2.1

  • No dependencies.

Version Downloads Last updated
9.0.0 9 11/14/2024
9.0.0-rc.2.24473.5 12 10/18/2024
9.0.0-rc.1.24431.7 7 10/17/2024
9.0.0-preview.7.24405.7 11 08/20/2024
9.0.0-preview.6.24327.7 30 07/19/2024
9.0.0-preview.5.24306.7 26 07/19/2024
9.0.0-preview.4.24266.19 26 07/19/2024
9.0.0-preview.3.24172.9 32 07/19/2024
9.0.0-preview.2.24128.5 32 07/19/2024
9.0.0-preview.1.24080.9 24 07/19/2024
8.0.1 32 07/19/2024
8.0.0 26 07/19/2024
8.0.0-rc.2.23479.6 29 07/19/2024
8.0.0-rc.1.23419.4 36 07/19/2024
8.0.0-preview.7.23375.6 29 07/19/2024
8.0.0-preview.6.23329.7 36 07/19/2024
8.0.0-preview.5.23280.8 31 07/19/2024
8.0.0-preview.4.23259.5 39 07/19/2024
8.0.0-preview.3.23174.8 26 07/19/2024
8.0.0-preview.2.23128.3 23 07/19/2024
8.0.0-preview.1.23110.8 28 07/19/2024
7.0.0 56 02/09/2024
7.0.0-rc.2.22472.3 29 07/19/2024
7.0.0-rc.1.22426.10 27 07/19/2024
7.0.0-preview.7.22375.6 31 07/19/2024
7.0.0-preview.6.22324.4 28 07/19/2024
7.0.0-preview.5.22301.12 35 07/19/2024
7.0.0-preview.4.22229.4 25 07/19/2024
7.0.0-preview.3.22175.4 22 07/19/2024
7.0.0-preview.2.22152.2 23 07/19/2024
7.0.0-preview.1.22076.8 34 07/19/2024
6.0.2-mauipre.1.22102.15 10 10/20/2024
6.0.0 49 02/04/2024
6.0.0-rc.2.21480.5 34 07/19/2024
6.0.0-rc.1.21451.13 32 07/19/2024
6.0.0-preview.7.21377.19 27 07/19/2024
6.0.0-preview.6.21352.12 31 07/19/2024
6.0.0-preview.5.21301.5 27 07/19/2024
6.0.0-preview.4.21253.7 23 07/19/2024
6.0.0-preview.3.21201.4 29 07/19/2024
6.0.0-preview.2.21154.6 27 07/19/2024
6.0.0-preview.1.21102.12 29 07/19/2024
5.0.0 33 07/19/2024
5.0.0-rc.2.20475.5 29 07/19/2024
5.0.0-rc.1.20451.14 30 07/19/2024
5.0.0-preview.8.20407.11 26 07/19/2024
5.0.0-preview.7.20364.11 29 07/19/2024
5.0.0-preview.6.20305.6 40 07/19/2024
5.0.0-preview.5.20278.1 29 07/19/2024
5.0.0-preview.4.20251.6 31 07/19/2024
5.0.0-preview.3.20214.6 33 07/19/2024
5.0.0-preview.2.20160.6 31 07/19/2024
5.0.0-preview.1.20120.5 26 07/19/2024
4.11.1 32 07/19/2024
4.11.0 29 07/19/2024
4.11.0-preview3.19551.4 26 07/19/2024
4.11.0-preview2.19523.17 30 07/19/2024
4.11.0-preview1.19504.10 24 07/19/2024
4.10.0 33 07/19/2024
4.10.0-rc1.19456.4 32 07/19/2024
4.10.0-preview9.19421.4 36 07/19/2024
4.10.0-preview9.19416.11 22 07/19/2024
4.10.0-preview8.19405.3 30 07/19/2024
4.10.0-preview7.19362.9 35 07/19/2024
4.10.0-preview6.19303.8 25 07/19/2024
4.10.0-preview6.19264.9 28 07/19/2024
4.10.0-preview5.19224.8 33 07/19/2024
4.10.0-preview4.19212.13 28 07/19/2024
4.10.0-preview3.19128.7 36 07/19/2024
4.10.0-preview.19073.11 37 07/19/2024
4.10.0-preview.18571.3 28 07/19/2024
4.9.0 43 02/04/2024
4.9.0-rc1 33 07/19/2024
4.9.0-preview2-26406-04 28 07/19/2024
4.9.0-preview1-26216-02 25 07/19/2024
4.8.0 32 07/19/2024
4.8.0-preview2-25405-01 38 07/19/2024
4.8.0-preview1-25305-02 28 07/19/2024
4.7.0 33 07/19/2024
4.7.0-preview1-24530-04 30 07/19/2024
4.6.0 30 07/19/2024
4.6.0-rc2-24027 26 07/19/2024
4.5.26-beta-23516 34 07/19/2024
4.5.26-beta-23409 34 07/19/2024
4.5.26-beta-23225 29 07/19/2024
4.5.25 29 07/19/2024
4.5.25-beta-23109 38 07/19/2024
4.5.25-beta-23019 28 07/19/2024
4.5.24 30 07/19/2024