Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Nile V3 specific fee support #518

Draft
wants to merge 3 commits into
base: add_linea
Choose a base branch
from
Draft

Conversation

Lesigh-3100
Copy link
Collaborator

Nile V3 is a Uni V3 fork that has an override for the fee actually charged by the pool, however, the tick spacing correlated to the original fee remains the same.

This means that the bot must:

  • Add handling to fetch the fee override & use it directly (divided by resolution 1e6) for the fee_float field in the TerraFormer & Exchange class.
  • Use the fee field for everything related to Uniswap V3 ticks.
  • Use the fee field for the customInt input to the Arb contract.
  • Use the fee_float field for exact calculations performed in RouteHandler.py.

@Lesigh-3100 Lesigh-3100 added the enhancement New feature or request label Apr 4, 2024
@Lesigh-3100 Lesigh-3100 self-assigned this Apr 4, 2024
@Lesigh-3100 Lesigh-3100 marked this pull request as draft April 4, 2024 10:24
@@ -231,6 +231,10 @@
{"inputs":[],"stateMutability":"nonpayable","type":"constructor"},{"anonymous":False,"inputs":[{"indexed":True,"internalType":"address","name":"oldImplementation","type":"address"},{"indexed":True,"internalType":"address","name":"newImplementation","type":"address"}],"name":"ImplementationChanged","type":"event"},{"anonymous":False,"inputs":[{"indexed":False,"internalType":"uint8","name":"version","type":"uint8"}],"name":"Initialized","type":"event"},{"anonymous":False,"inputs":[{"indexed":True,"internalType":"address","name":"oldOwner","type":"address"},{"indexed":True,"internalType":"address","name":"newOwner","type":"address"}],"name":"OwnerChanged","type":"event"},{"anonymous":False,"inputs":[{"indexed":True,"internalType":"address","name":"token0","type":"address"},{"indexed":True,"internalType":"address","name":"token1","type":"address"},{"indexed":False,"internalType":"bool","name":"stable","type":"bool"},{"indexed":False,"internalType":"address","name":"pair","type":"address"},{"indexed":False,"internalType":"uint256","name":"","type":"uint256"}],"name":"PairCreated","type":"event"},{"anonymous":False,"inputs":[{"indexed":False,"internalType":"uint8","name":"toFeesOld","type":"uint8"},{"indexed":False,"internalType":"uint8","name":"toTreasuryOld","type":"uint8"},{"indexed":False,"internalType":"uint8","name":"toFeesNew","type":"uint8"},{"indexed":False,"internalType":"uint8","name":"toTreasuryNew","type":"uint8"}],"name":"SetFeeSplit","type":"event"},{"anonymous":False,"inputs":[{"indexed":False,"internalType":"address","name":"pool","type":"address"},{"indexed":False,"internalType":"uint8","name":"toFeesOld","type":"uint8"},{"indexed":False,"internalType":"uint8","name":"toTreasuryOld","type":"uint8"},{"indexed":False,"internalType":"uint8","name":"toFeesNew","type":"uint8"},{"indexed":False,"internalType":"uint8","name":"toTreasuryNew","type":"uint8"}],"name":"SetPoolFeeSplit","type":"event"},{"inputs":[],"name":"MAX_FEE","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"acceptPauser","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"uint256","name":"","type":"uint256"}],"name":"allPairs","outputs":[{"internalType":"address","name":"","type":"address"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"allPairsLength","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"address","name":"tokenA","type":"address"},{"internalType":"address","name":"tokenB","type":"address"},{"internalType":"bool","name":"stable","type":"bool"}],"name":"createPair","outputs":[{"internalType":"address","name":"pair","type":"address"}],"stateMutability":"nonpayable","type":"function"},{"inputs":[],"name":"feeManager","outputs":[{"internalType":"address","name":"","type":"address"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"feeSplit","outputs":[{"internalType":"uint8","name":"","type":"uint8"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"bool","name":"_stable","type":"bool"}],"name":"getFee","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"address","name":"","type":"address"},{"internalType":"address","name":"","type":"address"},{"internalType":"bool","name":"","type":"bool"}],"name":"getPair","outputs":[{"internalType":"address","name":"","type":"address"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"address","name":"_pool","type":"address"}],"name":"getPoolFeeSplit","outputs":[{"internalType":"uint8","name":"_poolFeeSplit","type":"uint8"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"implementation","outputs":[{"internalType":"address","name":"","type":"address"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"address","name":"_voter","type":"address"},{"internalType":"address","name":"msig","type":"address"},{"internalType":"address","name":"_owner","type":"address"},{"internalType":"address","name":"_implementation","type":"address"},{"internalType":"address","name":"_feeManager","type":"address"}],"name":"initialize","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"address","name":"","type":"address"}],"name":"isPair","outputs":[{"internalType":"bool","name":"","type":"bool"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"isPaused","outputs":[{"internalType":"bool","name":"","type":"bool"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"owner","outputs":[{"internalType":"address","name":"","type":"address"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"pairCodeHash","outputs":[{"internalType":"bytes32","name":"","type":"bytes32"}],"stateMutability":"pure","type":"function"},{"inputs":[{"internalType":"address","name":"_pool","type":"address"}],"name":"pairFee","outputs":[{"internalType":"uint256","name":"fee","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"pauser","outputs":[{"internalType":"address","name":"","type":"address"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"pendingPauser","outputs":[{"internalType":"address","name":"","type":"address"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"bool","name":"_stable","type":"bool"},{"internalType":"uint256","name":"_fee","type":"uint256"}],"name":"setFee","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"address","name":"_feeManager","type":"address"}],"name":"setFeeManager","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"uint8","name":"_toFees","type":"uint8"},{"internalType":"uint8","name":"_toTreasury","type":"uint8"}],"name":"setFeeSplit","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"address","name":"_implementation","type":"address"}],"name":"setImplementation","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"address","name":"_owner","type":"address"}],"name":"setOwner","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"address","name":"_pair","type":"address"},{"internalType":"uint256","name":"_fee","type":"uint256"}],"name":"setPairFee","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"bool","name":"_state","type":"bool"}],"name":"setPause","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"address","name":"_pauser","type":"address"}],"name":"setPauser","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"address","name":"_pool","type":"address"},{"internalType":"uint8","name":"_toFees","type":"uint8"},{"internalType":"uint8","name":"_toTreasury","type":"uint8"}],"name":"setPoolFeeSplit","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"address","name":"_treasury","type":"address"}],"name":"setTreasury","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[],"name":"stableFee","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"treasury","outputs":[{"internalType":"address","name":"","type":"address"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"volatileFee","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"voter","outputs":[{"internalType":"address","name":"","type":"address"}],"stateMutability":"view","type":"function"}
]

NILE_V3_POOL_ABI = [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The required part of this ABI seems identical to UNISWAP_V3_POOL_ABI, so you can technically get rid of it (as well as any associated area in the code where you redirect to this ABI instead of the other one).

I'm writing this under the assumption that we need this pool's fee function, and not currentFee function, which is currently used in the code.

If the latter is indeed the one needed, then you should keep this ABI, but please:

  1. Move it to the section where all V3 Pool ABIs are located, for ease of comparison in the future
  2. Minimize it to include only the part necessary for the bot to interact with the contract:
NILE_V3_POOL_ABI = [
    {
        "type": "event",
        "name": "Swap",
        "anonymous": False,
        "inputs": [{"indexed": True, "internalType": "address", "name": "sender", "type": "address"}, {"indexed": True, "internalType": "address", "name": "recipient", "type": "address"}, {"indexed": False, "internalType": "int256", "name": "amount0", "type": "int256"}, {"indexed": False, "internalType": "int256", "name": "amount1", "type": "int256"}, {"indexed": False, "internalType": "uint160", "name": "sqrtPriceX96", "type": "uint160"}, {"indexed": False, "internalType": "uint128", "name": "liquidity", "type": "uint128"}, {"indexed": False, "internalType": "int24", "name": "tick", "type": "int24"}]
    },
    {
        "type": "function",
        "name": "currentFee",
        "stateMutability": "view",
        "inputs": [],
        "outputs": [{"internalType": "uint24", "name": "", "type": "uint24"}]
    },
    {
        "type": "function",
        "name": "liquidity",
        "stateMutability": "view",
        "inputs": [],
        "outputs": [{"internalType": "uint128", "name": "", "type": "uint128"}]
    },
    {
        "type": "function",
        "name": "slot0",
        "stateMutability": "view",
        "inputs": [],
        "outputs": [{"internalType": "uint160", "name": "sqrtPriceX96", "type": "uint160"}, {"internalType": "int24", "name": "tick", "type": "int24"}, {"internalType": "uint16", "name": "observationIndex", "type": "uint16"}, {"internalType": "uint16", "name": "observationCardinality", "type": "uint16"}, {"internalType": "uint16", "name": "observationCardinalityNext", "type": "uint16"}, {"internalType": "uint8", "name": "feeProtocol", "type": "uint8"}, {"internalType": "bool", "name": "unlocked", "type": "bool"}]
    },
    {
        "type": "function",
        "name": "tickSpacing",
        "stateMutability": "view",
        "inputs": [],
        "outputs": [{"internalType": "int24", "name": "", "type": "int24"}]
    },
    {
        "type": "function",
        "name": "token0",
        "stateMutability": "view",
        "inputs": [],
        "outputs": [{"internalType": "address", "name": "", "type": "address"}]
    },
    {
        "type": "function",
        "name": "token1",
        "stateMutability": "view",
        "inputs": [],
        "outputs": [{"internalType": "address", "name": "", "type": "address"}]
    }
]

fee_float = float(fee) / 1000000
if exchange == NILE_V3_NAME:
pool = web3.eth.contract(address=web3.to_checksum_address(pool_address), abi=NILE_V3_POOL_ABI)
fee_float = pool.caller.currentFee() / 1000000
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why function currentFee and not function fee?
For UniV3 pools, we are using function fee.
Might be worth verifying exactly which function is required here.
You can do that by looking into the source code of this pool contract, checking which fee is used in the swap function, and then following it to see which of these two functions returns that specific fee.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And why you using 1000000 in some cases and 1e6 in other cases?

Copy link
Collaborator

@barakman barakman Apr 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, creating a web3.eth.contract instance clearly doesn't belong in here (as evident by how it is done elsewhere in the code).
You can't just "throw stuff in" without taking into consideration the broader SW design and structure.

Copy link
Collaborator

@barakman barakman Apr 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, it doesn't make sense that we need to make an RPC at this point, as the fee has already been obtained previously.
So during that previous point - either it was already obtained correctly using this contract function, or it there was an attempt to use the other contract function, which would have reverted for this contract.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. The 1000000 vs 1e6 is just poor consistency - I will update this to use 1e6.
  2. Nile V3 uses a fee override function - currentFee. This results in an awkward situation in the Terraformer, as in all other cases the bot obtains the fee from the pool creation event. Due to the override, the best way to obtain the pool's current fee is by directly calling the contract.
  3. Regarding the overall SW design, do you have a suggestion for a more appropriate place for this, given that it's only relevant for Nile V3? I'm happy to move it if it makes more sense elsewhere.

Copy link
Collaborator

@barakman barakman Apr 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see the PR spawned off of this one.
In order to handle this in the uniswap_v3 module, for example, I used a paradigm similar to the one implemented in the solidly_v2 module.
That might give you an idea on how to deal with it here.

The part:

pool = web3.eth.contract(address=web3.to_checksum_address(pool_address), abi=NILE_V3_POOL_ABI)
fee_float = pool.caller.currentFee() / 1000000

Doesn't look like it belongs here.
Even without being acquainted with this module, a simple observation reveals that no other DEX requires an RPC for fetching the fee at this point, so why should the Nile V3 DEX require one?
Subsequently, placing it here breaks the structure of the code and the overall design of this module.
Wherever the fee for all the other DEXs is fetched, that's where the fee for Nile V3 should be fetched.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct that no other Uni V3 fork requires an additional call to fetch the fee, however, Nile V3 does.

The fee for other Uni V3 forks is obtained from the pool creation event. However, this is not sufficient for Nile V3 due to implementing a fee override.

Where else do you suggest placing this call given that no other Uni V3 forks require it?

@@ -206,7 +206,7 @@ def get_custom_int(self) -> int:
pool = self.pool
custom_int = 0
if self.exchange_name in self.ConfigObj.UNI_V3_FORKS:
custom_int = int(Decimal(pool.fee_float) * Decimal("1000000"))
custom_int = int(pool.fee)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this change related to this PR?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nile V3 includes a fee override.

This PR stores the fee override for the pool in the fee_float field, and the original fee (which is necessary for Uniswap V3 details such as tick-spacing etc) in the fee field.

The Fast Lane arb contract must know the fee of the Uni V3 pool being traded on, so using the fee_float to calculate the fee is no longer valid in instances of override.

Consequently the fee field is now used as it does not change.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not understand this description.
But Nile V3 should not be handled differently than any other DEX, other than in the dedicated sections (for example, in the uniswap_v3 module, where all of these DEXs are handled).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and to handle it the same, we must now use the fee field - which remains constant, instead of the fee_float which has an override for Nile V3.

@@ -520,10 +520,10 @@ def _univ3_to_cpc(self) -> List[Any]:
"tick": self.tick,
"liquidity": self.liquidity,
}
feeconst = self.FEE_LOOKUP.get(float(self.fee_float))
feeconst = self.FEE_LOOKUP.get(int(self.fee)/1e6)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this change related to this PR?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I maintained the original format of this by switching fee_float -> fee due to fee_float no longer being reliable. See above.

if feeconst is None:
raise ValueError(
f"Illegal fee for Uniswap v3 pool: {self.fee_float} [{self.FEE_LOOKUP}]]"
f"Illegal fee for Uniswap v3 pool: {int(self.fee)} [{self.FEE_LOOKUP}]]"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this change related to this PR?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above regarding fee_float & fee.

@@ -42,6 +47,9 @@ def get_events(self, contract: Contract) -> List[Type[Contract]]:
async def get_fee(self, address: str, contract: Contract) -> Tuple[str, float]:
fee = await contract.functions.fee().call()
fee_float = float(fee) / 1e6
if self.exchange_name == NILE_V3_NAME:
fee_float = await contract.functions.currentFee().call()
Copy link
Collaborator

@barakman barakman Apr 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is something really wrong here, as in one case you are dividing by the resolution, and in one case you don't.
So fee_float is sometimes returned after being divided by 16e, and sometimes being returned as read from the contract.
How could that possibly work correctly further down the road, when the caller of this function receives this returned value and needs to processes it?
Also note that you seem to have made a similar change in file run_blockchain_terraformer.py, where you actually do divide by 1000000 in both cases.
So even without taking the above into consideration, it is pretty clear that at least one of these two changes is incorrect.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition to that, it makes much more sense to implement a separate file called nile_v3.py.
This type of logic becomes increasingly confusing:

if self.exchange_name in [PANCAKESWAP_V3_NAME, AGNI_V3_NAME, FUSIONX_V3_NAME, ECHODEX_V3_NAME, SECTA_V3_NAME]:
    return PANCAKESWAP_V3_POOL_ABI
elif self.exchange_name == NILE_V3_NAME:
    return NILE_V3_POOL_ABI
return UNISWAP_V3_POOL_ABI

The guiding thumb-rule should be that once a simple branching (if/else) becomes too obfuscated, it is time to consider an alternative approach.
Note that an additional logic was also applied in function get_fee further down in this file, and on top of making it more cumbersome, this one actually introduces a bug:

fee_float = float(fee) / 1e6
if self.exchange_name == NILE_V3_NAME:
    fee_float = await contract.functions.currentFee().call()

As the value of fee_float is scaled down by 1e6 in some scenarios, and not scaled down at all in other scenarios.
So unless you apply the same logic (if exchange_name == NILE_V3_NAME) in every function which calls this function, an incorrect calculation most likely takes place.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After a short research, the best approach here would be a paradigm similar to the one in solidly_v2.py.

@@ -42,6 +47,9 @@ def get_events(self, contract: Contract) -> List[Type[Contract]]:
async def get_fee(self, address: str, contract: Contract) -> Tuple[str, float]:
fee = await contract.functions.fee().call()
fee_float = float(fee) / 1e6
if self.exchange_name == NILE_V3_NAME:
fee_float = await contract.functions.currentFee().call()
Copy link
Collaborator

@barakman barakman Apr 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is something really wrong here, as in one case you are dividing by the resolution, and in one case you don't.
So fee_float is sometimes returned after being divided by 16e, and sometimes being returned as read from the contract.
How could that possibly work correctly further down the road, when the caller of this function receives this returned value and needs to processes it?
Also note that you seem to have made a similar change in file run_blockchain_terraformer.py, where you actually do divide by 1000000 in both cases.
So even without taking the above into consideration, it is pretty clear that at least one of these two changes is incorrect.

@@ -42,6 +47,9 @@ def get_events(self, contract: Contract) -> List[Type[Contract]]:
async def get_fee(self, address: str, contract: Contract) -> Tuple[str, float]:
fee = await contract.functions.fee().call()
fee_float = float(fee) / 1e6
if self.exchange_name == NILE_V3_NAME:
fee_float = await contract.functions.currentFee().call() / 1e6
Copy link
Collaborator

@barakman barakman Apr 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential risk of a runtime problem related to this change is described in issue #314.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants