Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Directly include parallel math #5

Merged
merged 51 commits into from
Jun 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
2e36908
Init actor implementation crypto3-math
Zerg1996 May 31, 2022
cbebe36
Implement actor for polynomial
Zerg1996 Jun 2, 2022
6ce803b
Addede polynomial dfs actor tests
Zerg1996 Jun 4, 2022
941356c
Remove simple actor operations code duplications
Zerg1996 Jun 4, 2022
f2952a1
Implement dfs async polynom
Zerg1996 Jun 5, 2022
b4bf316
Code style fixes
Zerg1996 Jun 5, 2022
5c91223
Return check arguments in release build
Zerg1996 Jun 5, 2022
20a4ffe
Merge bug fixes
Zerg1996 Jul 4, 2022
88690f9
Fast division with parallels cases
Zerg1996 Jul 7, 2022
4b93ef4
Merge PR #17 from crypto3-math
Zerg1996 Sep 1, 2022
8374c3a
Merge #19 #20 PR from crypto3-math
Zerg1996 Sep 1, 2022
bd6777a
Initial Commit, moving the changes from crypto3 math library up to PR…
martun Mar 17, 2023
4f2300b
Few fixes
martun Mar 20, 2023
1b037c7
Fixing a few tests.
martun Mar 21, 2023
010aa44
Fixing a test
martun Mar 21, 2023
69f0855
Changes related to review comments.
martun Mar 23, 2023
c10b3a3
Merge pull request #1 from NilFoundation/crypto3_math_changes_up_to_P…
martun Mar 23, 2023
f60b14c
Some minor changes to the actor math library.
martun Mar 28, 2023
35408f4
Merge pull request #2 from NilFoundation/crypto3_math_changes_up_to_P…
Zerg1996 Mar 28, 2023
78fe0e5
Move the pow function from crypto3-math. (#4)
martun Jun 7, 2023
affb1b5
5 move changes from crypto3 math up to latest master commit dd7fb188.…
martun Sep 29, 2023
5e416ec
Function basic_radix2_fft now looks similar to the crypto3 code. (#10)
martun Oct 12, 2023
b3d5e94
Inside block_execution helper function, keep core #0 idle if at least…
martun Oct 12, 2023
177988b
Small fix in in-place division operator operator of polynomial. (#12)
martun Oct 16, 2023
17f3c44
Improve parallelization of fft. (#14)
martun Oct 18, 2023
ced7497
Fix build
x-mass Nov 22, 2023
f9d38c0
Fix build
x-mass Nov 22, 2023
96d2575
16 move latest changes from crypto3 math up to commit b6e1a8ee9a70882…
martun Dec 19, 2023
50b59a0
Added function block_execution_vector with desired effect #18
vo-nil Jan 26, 2024
c4f6690
Merge pull request #20 from NilFoundation/18-block-execution-returns-…
vo-nil Jan 26, 2024
00af08a
22 move changes from crypto3 math up to commit 2c3d8bb40f595fc7b87c8d…
martun Feb 5, 2024
782e688
Resetting actor-math to crypto3-math
martun Feb 14, 2024
e0e9b15
New actor math. (#24)
martun Feb 18, 2024
ddcf93b
Initial implementation #67
nemothenoone Jan 4, 2024
3d18e20
Minor changes #67
nemothenoone Feb 24, 2024
def0ae1
Add evaluate domain tests for goldilocks64 field
Feb 27, 2024
8ed5e51
Add inline for bitreverse for multi-module usage
makxenov Mar 15, 2024
f0d38b2
Minor changes #67
nemothenoone Apr 20, 2024
ec25bbe
Unnecessary modifiers removed #67
nemothenoone Apr 28, 2024
b273f08
Unnecessary uniqueness (disabling a copy constructor) changed to be a
nemothenoone Apr 28, 2024
5584818
Unused modifiers removed #67
nemothenoone May 5, 2024
25291b9
Remove expressions
x-mass May 2, 2024
43e913d
Use per-element pow in polynomial_dfs #40
x-mass May 13, 2024
19b3b36
Some changes related to recent changes in multiprecision. (#84)
martun May 15, 2024
e17db48
operator *= now works on scalars
vo-nil May 21, 2024
37d880d
Improve polynomial sum [SyncWith: crypto3-math#87] (#87)
x-mass May 22, 2024
9592363
Fix tests
x-mass Jun 5, 2024
0a2d34a
Move all files into libs/parallel-math directory
AndreyMlashkin Jun 10, 2024
9c8ae36
Remove parallel-math submodule
AndreyMlashkin Jun 10, 2024
1ad44ac
Merge remote-tracking branch 'parallel-math/migration' into directly-…
AndreyMlashkin Jun 10, 2024
c01f26d
cleanup .git* files and submodules in ex-parallel-math submodule
AndreyMlashkin Jun 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
[submodule "libs/threaded-zk"]
path = libs/parallel-zk
url = git@github.com:NilFoundation/actor-zk
[submodule "libs/threaded-math"]
path = libs/parallel-math
url = git@github.com:NilFoundation/actor-math
[submodule "cmake/modules"]
path = cmake/modules
url = git@github.com:BoostCMake/cmake_modules.git
1 change: 0 additions & 1 deletion libs/parallel-math
Submodule parallel-math deleted from 959236
85 changes: 85 additions & 0 deletions libs/parallel-math/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
#---------------------------------------------------------------------------#
# Copyright (c) 2018-2020 Mikhail Komarov <nemo@nil.foundation>
#
# Distributed under the Boost Software License, Version 1.0
# See accompanying file LICENSE_1_0.txt or copy at
# http://www.boost.org/LICENSE_1_0.txt
#---------------------------------------------------------------------------#

cmake_minimum_required(VERSION 2.8.12)

cmake_policy(SET CMP0028 NEW)
cmake_policy(SET CMP0042 NEW)
cmake_policy(SET CMP0048 NEW)
cmake_policy(SET CMP0057 NEW)
cmake_policy(SET CMP0076 NEW)

list(APPEND CMAKE_MODULE_PATH
"${CMAKE_CURRENT_LIST_DIR}/cmake"
"${CMAKE_CURRENT_LIST_DIR}/cmake/packages"
"${CMAKE_CURRENT_LIST_DIR}/cmake/modules/share/modules/cmake")

include(CMConfig)
include(CMSetupVersion)

if(NOT CMAKE_WORKSPACE_NAME OR NOT ("${CMAKE_WORKSPACE_NAME}" STREQUAL "actor"))
cm_workspace(actor)
endif()

macro(cm_find_package NAME)
foreach(ITERATOR ${CMAKE_WORKSPACE_LIST})
if(NOT "${NAME}" MATCHES "^${ITERATOR}_.*$" AND NOT "${NAME}" STREQUAL CM)
find_package(${ARGV})
else()
set(${ARGV0}_FOUND ON CACHE BOOL "")
endif()
endforeach()
endmacro()

cm_project(math WORKSPACE_NAME ${CMAKE_WORKSPACE_NAME} LANGUAGES ASM C CXX)

cm_find_package(CM)
include(CMDeploy)

option(BUILD_TESTS "Build unit tests" FALSE)

list(APPEND ${CURRENT_PROJECT_NAME}_PUBLIC_HEADERS)

list(APPEND ${CURRENT_PROJECT_NAME}_UNGROUPED_SOURCES)

list(APPEND ${CURRENT_PROJECT_NAME}_HEADERS ${${CURRENT_PROJECT_NAME}_PUBLIC_HEADERS})

list(APPEND ${CURRENT_PROJECT_NAME}_SOURCES ${${CURRENT_PROJECT_NAME}_UNGROUPED_SOURCES})

cm_setup_version(VERSION 0.1.0 PREFIX ${CMAKE_WORKSPACE_NAME}_${CURRENT_PROJECT_NAME})

add_library(${CMAKE_WORKSPACE_NAME}_${CURRENT_PROJECT_NAME} INTERFACE)

set_target_properties(${CMAKE_WORKSPACE_NAME}_${CURRENT_PROJECT_NAME} PROPERTIES
EXPORT_NAME ${CURRENT_PROJECT_NAME})


target_include_directories(${CMAKE_WORKSPACE_NAME}_${CURRENT_PROJECT_NAME} INTERFACE
"$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>"
"$<BUILD_INTERFACE:${CMAKE_BINARY_DIR}/include>"

${Boost_INCLUDE_DIRS})

target_link_libraries(${CMAKE_WORKSPACE_NAME}_${CURRENT_PROJECT_NAME} INTERFACE

${CMAKE_WORKSPACE_NAME}::core

crypto3::algebra
crypto3::multiprecision

${Boost_LIBRARIES})

find_package(Boost COMPONENTS random)

cm_deploy(TARGETS ${CMAKE_WORKSPACE_NAME}_${CURRENT_PROJECT_NAME}
INCLUDE include
NAMESPACE ${CMAKE_WORKSPACE_NAME}::)

if(BUILD_TESTS)
add_subdirectory(test)
endif()
25 changes: 25 additions & 0 deletions libs/parallel-math/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Polynomial Arithmetics, Fast Fourier Transforms for =nil; Crypto3 C++ Cryptography Suite

Crypto3.Math extends the =nil; Foundation's Crypto3.Algebra and provides a set of Fast Fourier Transforms evaluation algorithms and Polynomial Arithmetics implemented in way C++ standard library implies: concepts, algorithms, predictable behavior, latest standard features support and clean architecture without compromising security and performance.

## Building

This library uses Boost CMake build modules (https://github.com/BoostCMake/cmake_modules.git). To actually include this
library in a project it is required to:

1. Add [CMake Modules](https://github.com/BoostCMake/cmake_modules.git) as submodule to target project repository.
2. Add all the internal dependencies using [CMake Modules](https://github.com/BoostCMake/cmake_modules.git) as
submodules to target project repository.
3. Initialize parent project with [CMake Modules](https://github.com/BoostCMake/cmake_modules.git) (Look
at [crypto3](https://github.com/nilfoundation/crypto3.git) for the example)

## Dependencies

### Internal

* [Algebra](https://github.com/NilFoundation/crypto3-algebra)
* [Multiprecision](https://github.com/NilFoundation/crypto3-multiprecision)

### External

* [Boost](https://boost.org) (>= 1.58)
87 changes: 87 additions & 0 deletions libs/parallel-math/crypto3.math.podspec.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
{
"name": "cas.fft",
"version": "0.1.0",
"summary": "=nil; Foundation C++ Computer Algebra System Fast Fourier Transforms",
"description": "CAS.FFT library extends the =nil; Foundation's computer algebra system and provides a set of Fast Fourier Transforms evaluation algorithms implemented in way C++ standard library implies: concepts, algorithms, predictable behavior, latest standard features support and clean architecture without compromising security and performance.",
"homepage": "http://crypto3.nil.foundation/projects/crypto3",
"license": "Boost Software License",
"authors": {
"Mikhail Komarov": "nemo@nil.foundation"
},
"platforms": {
"ios": "5.0",
"osx": "10.7"
},
"source": {
"git": "https://github.com/nilfoundation/fft.git",
"branch": "master"
},
"xcconfig": {
"CLANG_CXX_LANGUAGE_STANDARD": "c++14",
"CLANG_CXX_LIBRARY": "libc++",
"HEADER_SEARCH_PATHS": "\"${PODS_ROOT}/include/\""
},
"subspecs": [
{
"name": "include",
"subspecs": [
{
"name": "nil",
"subspecs": [
{
"name": "crypto3",
"subspecs": [
{
"name": "detail",
"source_files": [
"include/nil/crypto3/detail/*.{hpp}"
]
},
{
"name": "fft",
"source_files": [
"include/nil/crypto3/fft/*.{hpp}"
],
"subspecs": [
{
"name": "algorithm",
"source_files": [
"include/nil/crypto3/vdf/algorithm/*.{hpp}"
]
},
{
"name": "adaptor",
"source_files": [
"include/nil/crypto3/vdf/adaptor/*.{hpp}"
]
},
{
"name": "accumulators",
"source_files": [
"include/nil/crypto3/vdf/accumulators/*.{hpp}"
],
"subspecs": [
{
"name": "parameters",
"source_files": [
"include/nil/crypto3/vdf/accumulators/parameters/*.{hpp}"
]
}
]
},
{
"name": "detail",
"source_files": [
"include/nil/crypto3/vdf/detail/*.{hpp}"
]
}
]
}
]
}
]
}
]
}
]
}
3 changes: 3 additions & 0 deletions libs/parallel-math/docs/concepts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Concepts # {#fft_concepts}

@tableofcontents
3 changes: 3 additions & 0 deletions libs/parallel-math/docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Fourier Fast Transform {#fft_index}

@subpage fft_introduction @subpage fft_manual @subpage fft_concepts
96 changes: 96 additions & 0 deletions libs/parallel-math/docs/introduction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# Introduction # {#fft_introduction}

@tableofcontents

Crypto3.FFT library extends the =nil; Foundation's computer algebra system and provides a set of Fast Fourier Transforms
evaluation algorithms implemented in way C++ standard library implies: concepts, algorithms, predictable behavior,
latest standard features support and clean architecture without compromising security and performance.

Crypto3.FFT consists of several parts to review:

* [Manual](@ref fft_manual).
* [Implementation](@ref fft_impl).
* [Concepts](@ref fft_concepts).

## Background

There is currently a variety of algorithms for computing the Fast Fourier Transform (FFT) over the field of complex
numbers. For this situation, there exists many libraries, such as [FFTW](http://www.fftw.org/), that have been
rigorously developed, tested, and optimized. Our goal is to use these existing techniques and develop novel
implementations to address the more interesting case of FFT in finite fields. We will see that in many instances, these
algorithms can be used for the case of finite fields, but the construction of FFT in finite fields remains, in practice,
challenging.

Consider a finite field _F_ with _2^m_ elements. We can define a discrete Fourier transform by choosing a _2^m − 1_ root
of unity _ω ∈ F_. Operating over the complex numbers, there exists a variety of FFT algorithms, such as
the [Cooley-Tukey algorithm](http://en.wikipedia.org/wiki/Cooley%E2%80%93Tukey_FFT_algorithm) along with its variants,
to choose from. And in the case that _2^m - 1_ is prime - consider the Mersenne primes as an example - we can turn to
other algorithms, such as [Rader's algorithm](http://en.wikipedia.org/wiki/Rader%27s_FFT_algorithm)
and [Bluestein's algorithm](http://en.wikipedia.org/wiki/Bluestein%27s_FFT_algorithm). In addition, if the domain size
is an extended power of two or the sum of powers of two, variants of the radix-2 FFT algorithms can be employed to
perform the computation.

However, in a finite field, there may not always be a root of unity. If the domain size is not as mentioned, then one
can consider adjoining roots to the field. Although, there is no guarantee that adjoining such a root to the field can
render the same performance benefits, as it would produce a significantly larger structure that could cancel out
benefits afforded by the FFT itself. Therefore, one should consider other algorithms which continue to perform better
than the naïve evaluation.

## Domains

Given a domain size, the library will determine and perform computations over the best-fitted domain. Ideally, it is
desired to perform evaluation and interpolation over a radix-2 FFT domain, however, this may not always be possible.
Thus, the library provides the arithmetic and geometric sequence domains as fallback options, which we show to perform
better than naïve evaluation.

| | Basic Radix-2 | Extended Radix-2 | Step Radix-2 | Arithmetic Sequence | Geometric Sequence |
|---------------|:-------------:|:----------------:|:------------:|:--------------------:|:------------------:|
| Evaluation | O(n log n) | O(n log n) | O(n log n) | M(n) log(n) + O(M(n)) | 2M(n) + O(n) |
| Interpolation | O(n log n) | O(n log n) | O(n log n) | M(n) log(n) + O(M(n)) | 2M(n) + O(n) |

### Radix-2 FFTs

The radix-2 FFTs are comprised of three domains: basic, extended, and step radix-2. The radix-2 domain implementations
make use of pseudocode from [CLRS 2n Ed, pp. 864].

#### Basic radix-2 FFT

The basic radix-2 FFT domain has size _m = 2^k_ and consists of the _m_-th roots of unity. The domain uses the standard
FFT algorithm and inverse FFT algorithm to perform evaluation and interpolation. Multi-core support includes
parallelizing butterfly operations in the FFT operation.

#### Extended radix-2 FFT

The extended radix-2 FFT domain has size _m = 2^(k + 1)_ and consists of the _m_-th roots of unity, union a coset of
these roots. The domain performs two _basic\_radix2\_FFT_ operations for evaluation and interpolation in order to
account for the extended domain size.

#### Step radix-2 FFT

The step radix-2 FFT domain has size _m = 2^k + 2^r_ and consists of the _2^k_-th roots of unity, union a coset of _2^r_
-th roots of unity. The domain performs two _basic\_radix2\_FFT_ operations for evaluation and interpolation in order to
account for the extended domain size.

### Arithmetic sequence

The arithmetic sequence domain is of size _m_ and is applied for more general cases. The domain applies a basis
conversion algorithm between the monomial and the Newton bases. Choosing sample points that form an arithmetic
progression, _a\_i = a\_1 + (i - 1)*d_, allows for an optimization of computation over the monomial basis, by using the
special case of Newton evaluation and interpolation on an arithmetic progression, see \[BS05\].

### Geometric sequence

The geometric sequence domain is of size _m_ and is applied for more general cases. The domain applies a basis
conversion algorithm between the monomial and the Newton bases. The domain takes advantage of further simplications to
Newton evaluation and interpolation by choosing sample points that form a geometric progression, _a\_n = r^(n-1)_, see
\[BS05\].

## Dependencies ## {#fft_dependencies}

Internal dependencies:

1. [=nil; Algebra](https://github.com/nilfoundation/algebra.git)

Outer dependencies:

1. [Boost (optional)](https://boost.org) (>= 1.58)
3 changes: 3 additions & 0 deletions libs/parallel-math/docs/manual.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Manual # {#fft_manual}

@tableofcontents
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
//---------------------------------------------------------------------------//
// Copyright (c) 2020-2021 Mikhail Komarov <nemo@nil.foundation>
// Copyright (c) 2020-2021 Nikita Kaskov <nbering@nil.foundation>
//
// MIT License
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
//---------------------------------------------------------------------------//

#ifndef CRYPTO3_MATH_CALCULATE_DOMAIN_SET_HPP
#define CRYPTO3_MATH_CALCULATE_DOMAIN_SET_HPP

#include <nil/crypto3/math/domains/evaluation_domain.hpp>
#include <nil/crypto3/math/algorithms/make_evaluation_domain.hpp>

#include <nil/actor/core/thread_pool.hpp>
#include <nil/actor/core/parallelization_utils.hpp>

namespace nil {
namespace crypto3 {
namespace math {

template<typename FieldType>
std::vector<std::shared_ptr<evaluation_domain<FieldType>>>
calculate_domain_set(const std::size_t max_domain_degree, const std::size_t set_size) {

std::vector<std::shared_ptr<evaluation_domain<FieldType>>> domain_set(set_size);

// make_evaluation_domain uses LOW level thread pool, so this function needs to use
// ThreadPool::PoolLevel::HIGH.
parallel_for(0, set_size, [&domain_set, max_domain_degree](std::size_t i){
const std::size_t domain_size = std::pow(2, max_domain_degree - i);
std::shared_ptr<evaluation_domain<FieldType>> domain =
make_evaluation_domain<FieldType>(domain_size);
domain_set[i] = domain;
}, ThreadPool::PoolLevel::HIGH);

return domain_set;
}
} // namespace math
} // namespace crypto3
} // namespace nil

#endif // CRYPTO3_MATH_CALCULATE_DOMAIN_SET_HPP
Loading
Loading