Merge branch '41-add-support-for-new-carbon-contract-custom-fees-per-…

…strategy' of https://github.com/bancorprotocol/fastlane-bot into 41-add-support-for-new-carbon-contract-custom-fees-per-strategy
bancorprotocol · Aug 15, 2023 · c630bfa · c630bfa
2 parents bca8da0 + 5af3136
commit c630bfa
Show file tree

Hide file tree

Showing 12 changed files with 839 additions and 2,031 deletions.
diff --git a/.github/workflows/release-and-pypi-publish.yml b/.github/workflows/release-and-pypi-publish.yml
@@ -171,6 +171,7 @@ jobs:
         run: |
           pip install wheel
           python setup.py bdist_wheel
+          find dist -type f ! -name "*.whl" -exec rm -f {} +
 
       - name: Publish package to PyPI
         uses: pypa/gh-action-pypi-publish@release/v1

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,14 +2,57 @@
 
 ## [Unreleased](https://github.com/bancorprotocol/fastlane-bot/tree/HEAD)
 
-[Full Changelog](https://github.com/bancorprotocol/fastlane-bot/compare/v2.7.3...HEAD)
+[Full Changelog](https://github.com/bancorprotocol/fastlane-bot/compare/v2.7.8...HEAD)
 
-- Cleanup of Unused Top-Level Files [\#47](https://github.com/bancorprotocol/fastlane-bot/issues/47)
+- Bugfix: Remove DAI from `flashloan_tokens` [\#78](https://github.com/bancorprotocol/fastlane-bot/issues/78)
+- Remove DAI from default `flashloan_tokens` to fix logging error message. [\#79](https://github.com/bancorprotocol/fastlane-bot/pull/79) ([mikewcasale](https://github.com/mikewcasale))
+
+Closed issues
+
+- Broken Link To Github [\#64](https://github.com/bancorprotocol/fastlane-bot/issues/64)
+
+## [v2.7.8](https://github.com/bancorprotocol/fastlane-bot/tree/v2.7.8) (2023-08-10)
+
+[Full Changelog](https://github.com/bancorprotocol/fastlane-bot/compare/v2.7.7...v2.7.8)
+
+- Fix broken pypi automation [\#66](https://github.com/bancorprotocol/fastlane-bot/issues/66)
+
+Merged pull requests
+
+- Fix Github Link [\#65](https://github.com/bancorprotocol/fastlane-bot/pull/65) ([shmuel44](https://github.com/shmuel44))
+
+## [v2.7.7](https://github.com/bancorprotocol/fastlane-bot/tree/v2.7.7) (2023-08-10)
+
+[Full Changelog](https://github.com/bancorprotocol/fastlane-bot/compare/v2.7.6...v2.7.7)
+
+- Update release-and-pypi-publish.yml [\#67](https://github.com/bancorprotocol/fastlane-bot/pull/67) ([mikewcasale](https://github.com/mikewcasale))
+
+- Feature Request: Add `target_tokens` Setting to Narrow Search Space [\#62](https://github.com/bancorprotocol/fastlane-bot/issues/62)
+
+## [v2.7.6](https://github.com/bancorprotocol/fastlane-bot/tree/v2.7.6) (2023-08-09)
+
+[Full Changelog](https://github.com/bancorprotocol/fastlane-bot/compare/v2.7.5...v2.7.6)
+
+- Change the `--flashloan_tokens` flag type to list [\#57](https://github.com/bancorprotocol/fastlane-bot/issues/57)
+
+- 62 add target tokens setting to narrow search space [\#63](https://github.com/bancorprotocol/fastlane-bot/pull/63) ([mikewcasale](https://github.com/mikewcasale))
+
+## [v2.7.5](https://github.com/bancorprotocol/fastlane-bot/tree/v2.7.5) (2023-08-09)
+
+[Full Changelog](https://github.com/bancorprotocol/fastlane-bot/compare/v2.7.4...v2.7.5)
+
+- Fix to ensure the @click flag for flashloan\_tokens is respected [\#58](https://github.com/bancorprotocol/fastlane-bot/pull/58) ([mikewcasale](https://github.com/mikewcasale))
 
 Closed issues
 
 - Installation Issue with pyyaml==5.4.1 and Brownie [\#30](https://github.com/bancorprotocol/fastlane-bot/issues/30)
 
+## [v2.7.4](https://github.com/bancorprotocol/fastlane-bot/tree/v2.7.4) (2023-08-07)
+
+[Full Changelog](https://github.com/bancorprotocol/fastlane-bot/compare/v2.7.3...v2.7.4)
+
+- Cleanup of Unused Top-Level Files [\#47](https://github.com/bancorprotocol/fastlane-bot/issues/47)
+
 Merged pull requests
 
 - Creates a bash install script to handle the conda vs pip installation… [\#51](https://github.com/bancorprotocol/fastlane-bot/pull/51) ([mikewcasale](https://github.com/mikewcasale))

diff --git a/fastlane_bot/__init__.py b/fastlane_bot/__init__.py
@@ -1,7 +1,7 @@
 from .bot import CarbonBot as Bot, __VERSION__, __DATE__
 from .config import Config, ConfigNetwork, ConfigDB, ConfigLogger, ConfigProvider
 
-__version__ = '2.7.4'
+__version__ = '2.7.9'
 
 
 
diff --git a/fastlane_bot/events/interface.py b/fastlane_bot/events/interface.py
@@ -60,6 +60,26 @@ class QueryInterface:
     def cfg(self) -> Config:
         return self.ConfigObj
 
+    def filter_target_tokens(self, target_tokens: List[str]):
+        """
+        Filter the pools to only include pools that are in the target pools list
+
+        Parameters
+        ----------
+        target_tokens: List[str]
+            The list of tokens to filter pools by. Pools must contain both tokens in the list to be included.
+        """
+        initial_state = self.state.copy()
+        self.state = [
+            pool
+            for pool in self.state
+            if pool["tkn0_key"] in target_tokens and pool["tkn1_key"] in target_tokens
+        ]
+
+        self.cfg.logger.info(
+            f"Limiting pools by target_tokens. Removed {len(initial_state) - len(self.state)} non target-pools. {len(self.state)} pools remaining"
+        )
+
     def remove_unsupported_exchanges(self) -> None:
         initial_state = self.state.copy()
         self.state = [
@@ -479,8 +499,8 @@ def get_pool(self, **kwargs) -> Optional[PoolAndTokens]:
             )
             pool.exchange_name
         except AttributeError:
-            if 'cid' in kwargs:
-                kwargs['cid'] = int(kwargs['cid'])
+            if "cid" in kwargs:
+                kwargs["cid"] = int(kwargs["cid"])
                 pool = next(
                     (
                         pool

diff --git a/main.py b/main.py
@@ -20,7 +20,11 @@
 from fastlane_bot.config import Config
 from fastlane_bot.events.interface import QueryInterface
 from fastlane_bot.events.managers.manager import Manager
-from fastlane_bot.events.utils import complex_handler, filter_latest_events
+from fastlane_bot.events.utils import (
+    complex_handler,
+    filter_latest_events,
+)
+from fastlane_bot.tools.cpc import T
 
 load_dotenv()
 
@@ -52,7 +56,10 @@
     ),
 )
 @click.option(
-    "--flashloan_tokens", default=None, type=str, help="See flashloan_tokens in bot.py"
+    "--flashloan_tokens",
+    default=f"{T.WETH},{T.USDC},{T.USDT},{T.WBTC},{T.BNT},{T.NATIVE_ETH}",
+    type=str,
+    help="The --flashloan_tokens flag refers to those token denominations which the bot can take a flash loan in. By default, these are [WETH, DAI, USDC, USDT, WBTC, BNT, NATIVE_ETH]. If you override the default to TKN, the search space is decreased for all modes, including the b3_two_hop mode (assuming that --limit_bancor3_flashloan_tokens=True).",
 )
 @click.option("--config", default=None, type=str, help="See config in config/*")
 @click.option("--n_jobs", default=-1, help="Number of parallel jobs to run")
@@ -129,6 +136,12 @@
     type=int,
     help="Set to the timeout in seconds. Set to None for no timeout.",
 )
+@click.option(
+    "--target_tokens",
+    default=None,
+    type=str,
+    help="A comma-separated string of tokens to target. Use None to target all tokens. Use `flashloan_tokens` to target only the flashloan tokens.",
+)
 def main(
     cache_latest_only: bool,
     backdate_pools: bool,
@@ -150,6 +163,7 @@ def main(
     limit_bancor3_flashloan_tokens: bool,
     default_min_profit_bnt: int,
     timeout: int,
+    target_tokens: str,
 ):
     """
     The main entry point of the program. It sets up the configuration, initializes the web3 and Base objects,
@@ -159,7 +173,7 @@ def main(
         cache_latest_only (bool): Whether to cache only the latest events or not.
         backdate_pools (bool): Whether to backdate pools or not. Set to False for quicker testing runs.
         arb_mode (str): The arbitrage mode to use.
-        flashloan_tokens (str): Tokens that the bot can use for flash loans.
+        flashloan_tokens (str): Comma seperated list of tokens that the bot can use for flash loans.
         config (str): The name of the configuration to use.
         n_jobs (int): The number of jobs to run in parallel.
         exchanges (str): A comma-separated string of exchanges to include.
@@ -175,8 +189,11 @@ def main(
         limit_bancor3_flashloan_tokens (bool): Whether to limit the flashloan tokens to the ones supported by Bancor v3 or not.
         default_min_profit_bnt (int): The default minimum profit in BNT.
         timeout (int): The timeout in seconds.
+        target_tokens (str): A comma-separated string of tokens to target. Use None to target all tokens. Use `flashloan_tokens` to target only the flashloan tokens.
 
     """
+    start_time = time.time()
+
     # Set config
     loglevel = (
         Config.LOGLEVEL_DEBUG
@@ -203,6 +220,36 @@ def main(
     cfg.DEFAULT_MIN_PROFIT_BNT = Decimal(str(default_min_profit_bnt))
     cfg.DEFAULT_MIN_PROFIT = Decimal(str(default_min_profit_bnt))
 
+    # Log the flashloan tokens
+    flashloan_tokens = flashloan_tokens.split(",")
+    flashloan_tokens = [
+        QueryInterface.cleanup_token_key(token) for token in flashloan_tokens
+    ]
+    cfg.logger.info(
+        f"Flashloan tokens are set as: {flashloan_tokens}, {type(flashloan_tokens)}"
+    )
+
+    if target_tokens:
+        if target_tokens == "flashloan_tokens":
+            target_tokens = flashloan_tokens
+        else:
+            target_tokens = target_tokens.split(",")
+            target_tokens = [
+                QueryInterface.cleanup_token_key(token) for token in target_tokens
+            ]
+
+            # Ensure that the target tokens are a subset of the flashloan tokens
+            for token in flashloan_tokens:
+                if token not in target_tokens:
+                    cfg.logger.warning(
+                        f"Falshloan token {token} not in target tokens. Adding it to target tokens."
+                    )
+                    target_tokens.append(token)
+
+        cfg.logger.info(
+            f"Target tokens are set as: {target_tokens}, {type(target_tokens)}"
+        )
+
     # Set external exchanges
     exchanges = exchanges.split(",")
     cfg.logger.info(f"Running data fetching for exchanges: {exchanges}")
@@ -242,6 +289,10 @@ def main(
         cfg.logger.error("Error parsing the CSV file")
         raise
 
+    if timeout == 1:
+        cfg.logger.info("Timeout to test the bot flags")
+        return
+
     # Initialize web3
     static_pool_data["cid"] = [
         cfg.w3.keccak(text=f"{row['descr']}").hex()
@@ -288,6 +339,7 @@ def main(
         run_data_validator,
         randomizer,
         timeout,
+        target_tokens,
     )
 
 
@@ -306,6 +358,8 @@ def run(
     run_data_validator: bool,
     randomizer: int,
     timeout: int,
+    target_tokens: List[str] or None,
+
 ) -> None:
     """
     The main function that drives the logic of the program. It uses helper functions to handle specific tasks.
@@ -325,6 +379,7 @@ def run(
         run_data_validator (bool): Whether to run the data validator or not.
         randomizer (bool): Whether to randomize the polling interval or not.
         timeout (int): The timeout for the polling interval.
+        target_tokens (List[str]): List of tokens that the bot will target for arbitrage.
     """
 
     def get_event_filters(
@@ -686,6 +741,10 @@ def update_pools_from_contracts(
                 bot.db.remove_zero_liquidity_pools()
                 bot.db.remove_unsupported_exchanges()
 
+                # Filter the target tokens
+                if target_tokens:
+                    bot.db.filter_target_tokens(target_tokens)
+
                 # Run the bot
                 bot.run(
                     polling_interval=polling_interval,