Cant pass synch state 2/7 and auto shuts down. #73
Comments
|
Windows 10 . same situation ! |
|
I got same issue when syncing from scratch. The first stuck at 82% during PIBD, and the second one at 100%. |
|
Thanks everyone for testing. Could everyone having sync issues try to sync with the standalone command-line node and see if the same thing occurs? |
|
I had the same issue. I synced with grin.exe then copied chain_data into the grin_node folder and the node now runs. |
I'm not clear what standalone command-line? You mean original grin node command line? I tried grin-gui.exe --help but not found argument to sync the chain |
|
I use to have problems syncing on the window full node, I think changing it to archive mode helped |
|
@yeastplume which specific versions of the wallet and node should we use try to see if the same issue occurs? |
The node is from grin master branch, the wallet from the contracts branch.
I mean the build from the core node project itself, not the gui: https://github.com/mimblewimble/grin I've been trying to reproduce this myself by syncing from scratch, but I haven't had a situation where it stops or shuts down. I'll keep trying. |
|
I will run multiple tests and each time update this comment:
|
|
Yep, okay I'm seeing this as well, but only on the released CI built version. When I run a local build, it's fine and moves to stage 3. in |
|
I will try running my own compiled version in a bit.
|
|
Precompiled version 5.2.0-beta3 shows these errors before crashing in the log in the UI |
Oh, thanks very much for this, it gives me a potential better place to look for the issue |
|
It looks a bit like it rewinds block by block beyond the PIBD horizon and then totally craps out. More detailed log from the log file:
|
|
Okay, this is just about the worst kind of bug there is:
So my strategy was to try and have CI produce a debug build with symbols so I could get some information, but of course that version doesn't crash. I thought it may have something to do with the bin being built with the windows subsystem (that avoids spawning the console window), so I tried a version that spawned a console window, built on CI on my own fork, and that crashes as well. This feels like some arcane native lib thing (or possibly llvm?) but have no idea how to go about finding the culprit now given that a debug build built on the same CI system doesn't exhibit the crash. Fact that the debug build doesn't crash and the release build does definitely points to some kind of mismanaged memory underneath rust (either in an FFI layer or C lib), so it could be just about anywhere. A few things would help if someone could try:
Not sure how to proceed myself at the moment, need to think. |
|
That's an illegal instruction.. and the crash occurs on validation. Perhaps the release build on CI is compiling for some arcane instruction set? |
|
Ok, I can verify that the local build of at least the main node works fine. One thing that came to mind is that perhaps the build is for x86 on Windows, In which case we should try putting the release build in C:\Program Files (x86) and run from there, some different .dll settings are applied in that case. Running from another directory automatically assume its 64 bit. I have no time to test it today, perhaps tomorrow, perhaps you have time @yeastplume to test this wild guess of mine. I will try to make time tomorrow to test the GUI with local build, and will try running debug .exe built (in alpha.4) |
|
it's an x86_64 build. I now think the release build is optimizing something on the CI server for the particular architecture of whatever CPU it's running and compiling in some instruction set not found in most consumer CPUs. I'll try and track down what CI is optimized for by default and compare to a local build. |
|
Tested running from Programs(x86), did not work, after checking the executable by opening it in notepad I see it contains "PE d†" at the beginning of the file, which is a market that it is indeed a 64 bit executable. So indeed it must be some optimizations for the CPU or the virtual system you compiled it in. |
|
Perhaps useful thread:
Might be or not be useful. Our case is a bit different since the programs runs, but contains a weird bug. |
|
I have run the release debug version (alpha3), and that one DOES WORK. Advice from a friend in another crypto project, compile on an older machine to avoid optimizations for newer processors like AVIX2 AVIX5, AVIX5.12. |
|
-System: Windows 10 / Grin version 0.1.0-alpha.3 Still cant pass 2/7 synch and it shuts down auto. LOGS 20240119 13:41:47.350 INFO grin_gui_core::node - This is Grin version 0.1.0-alpha.3 (git v0.1.0-alpha.3), built for x86_64-pc-windows-msvc by rustc 1.75.0 (82e1608df 2023-12-21). WALLET LOG 20240111 14:37:21.079 WARN grin_wallet_libwallet::api_impl::owner_updater - Updater Thread unable to contact node |
|
Ok, now things get interesting. It does work for me and @yeastplume, meaning it was compiled for some optimizations that both our CPU's support, but not your CPU. @cekickafa @yeastplume Can you tell me what CPU you use (Windows about, CPU information shown there)? MY PC runs a Ryzen 7 2700X, which has AVX2 standard support but not AVX-512. So probably @yeastplume and I have CPU's of a generation that supports AVX2. The trick is probably that the release should be compiled for an older CPU type that does not use AVX, AVX2 or AVX512. Slight hit in performance, but better compatibility. Those who wants maximum performance can compile themselves. |
|
My CPU is AMD Ryzen 7 1700 Eight-Core Processor 3.00 GHz. |
|
I checked, both your and my CPU have the exact same instruction set (both Ryzen 7 family, same cores and cash, only difference is clock speed), both support AVX, AVX2 but not AVX512. @cekickafa Can you try again, but first run
|
|
@yeastplume Which cross platform compatibility flags do you use when compiling?: |
|
0.1.0-alpha.3 still can't pass 2/7 and it shuts down auto
|
|
Windows 7 / laptop x230i CAN PASS 2/7
|
|
@z3j3 @cekickafa Can you perhaps check if you have Hyper-V disabled or enabled? I suspect this might have influence, when its enabled, virtualization will not work which is perhaps needed for Windows subsystems. In case you had it enabled @cekickafa , try disabling, restart PC, then try again running 0.1.0-alpha.3. |
|
i checked it is OFF. I am not using virtual Machine. I deleted and reinstalled 0.1.0-alpha.3. Waiting for synch now. Almost synched to 2/7 waiting still. |
20240120 09:59:37.300 INFO grin_gui_core::node - This is Grin version 0.1.0-alpha.3 (git v0.1.0-alpha.3), built for x86_64-pc-windows-msvc by rustc 1.75.0 (82e1608df 2023-12-21).
|
|
Ok, and if you shutdown and restart grin-gui? I think the being stuck bug is different from not being able to get past 100% at sync step 2. |
|
it is off
|
|
I've checked differences in target architecture between local and the CI build, LLVM versions, still haven't found any fix. Because I can't actually get any useful debug info (since this issue doesn't occur on a debug build,) I'm going to have to start spraying debug statements all over the place to try and find out exactly where this crash is occurring. Will do that on my own fork |
|
Alright, managed to create a build with enough debug info that still exhibits the crash. Very much looking like this is some optimization flag within our old friend croaring bitmaps:
|
|
More fun here because it looks like the instruction is AVX2 which most processors should support now. Tried changing some croaring build flags.. no change. |
|
Something message ?!
|
|
@z3j3 Interesting observation, can you check in the node tab, just to be sure the syncing has finished also on the Desktop with the AMD CPU? |
the node show 'running' , |
|
@z3j3 In that case I think you should file it as a new bug/issue, this makes it easier for @yeastplume to keep track of it. |
Okay, that was confusing me because within a function called |
|
Still tearing hair out over this, but this indeed looks like an issue in CRoaringRS or windows CI: Edit: No, looks like a VC compiler bug |
|
Could everyone having this issue now try using the latest alpha, 0.1.0-alpha.4! |
|
Successfully running, performed a transaction! Thank you @yeastplume. |
|
Fast sync to end and 'Running'. |
|
I confirmed the sync went through in version alpha 4 |
|
Looks like this issue is resolved in alpha 4. The node is fully synced and running. |
|
thanks all for testing! Closing |
20240112 14:21:36.088 INFO grin_gui - Grin GUI 0.1.0-alpha.2 has started.
20240112 14:21:36.090 DEBUG grin_gui::gui - config loaded:
Config {
wallets: [
Wallet {
tld: Some(
"C:\Users\*****\.grin\main\grin_wallet\default",
),
display_name: "Default",
use_embedded_node: true,
chain_type: Mainnet,
},
],
current_wallet_index: Some(
0,
),
mixnet_keys: None,
theme: Some(
"Ayu",
),
currency: GRIN,
tx_column_config: V1 {
local_version_width: 150,
remote_version_width: 150,
status_width: 85,
},
window_size: Some(
(
1067,
656,
),
),
scale: None,
backup_directory: None,
self_update_channel: Stable,
alternating_row_colors: false,
is_keybindings_enabled: false,
language: English,
auto_update: false,
compression_format: Zip,
zstd_compression_level: 0,
close_to_tray: false,
autostart: true,
start_closed_to_tray: false,
tx_method: Legacy,
}
20240112 14:21:36.090 DEBUG grin_gui::gui - antialiasing: true
20240112 14:21:36.576 DEBUG grin_gui::gui::update - Font Loaded: Ok(())
20240112 14:21:36.577 DEBUG grin_gui::gui::update - Font Loaded: Ok(())
20240112 14:21:36.591 INFO grin_gui_core::logger - log4rs configuration changed - gui-stdout - DEBUG, gui-file - DEBUG, node-stdout - WARN, node-file - INFO, min level: DEBUG
20240112 14:21:40.085 DEBUG grin_gui::gui::element::menu - Interaction::ModeSelectedSettings(Node)
20240112 14:23:10.182 DEBUG grin_gui::gui::element::menu - Interaction::ModeSelectedSettings(Wallet)
20240112 14:23:14.984 DEBUG grin_gui::gui::element::wallet::operation::open - setup::wallet::operation::open::OpenWallet
20240112 14:23:14.986 INFO grin_gui_core::logger - log4rs configuration changed - gui-stdout - DEBUG, gui-file - DEBUG, node-stdout - WARN, node-file - INFO, wallet-stdout - WARN, wallet-file - INFO, min level: DEBUG
20240112 14:23:15.372 DEBUG grin_gui::gui::element::wallet::operation::home - Update Wallet Info Summary: false, WalletInfo { last_confirmed_height: 0, minimum_confirmations: 2, total: 0, amount_awaiting_finalization: 0, amount_awaiting_confirmation: 0, amount_immature: 0, amount_currently_spendable: 0, amount_locked: 0, amount_reverted: 0 }
20240112 14:23:15.372 DEBUG grin_gui::gui::element::wallet::operation::home - Updating transactions based on new wallet info data: None (old) vs WalletInfo { last_confirmed_height: 0, minimum_confirmations: 2, total: 0, amount_awaiting_finalization: 0, amount_awaiting_confirmation: 0, amount_immature: 0, amount_currently_spendable: 0, amount_locked: 0, amount_reverted: 0 } (new)
20240112 14:23:15.412 DEBUG grin_gui::gui::element::wallet::operation::tx_list_display - Update Tx List Summary: false
20240112 14:23:15.412 DEBUG grin_gui::gui::element::wallet::operation::tx_list_display - Update Wallet Txs Summary: []
20240112 14:23:25.453 DEBUG grin_gui::gui::element::wallet::operation::home - Update Wallet Info Summary: false, WalletInfo { last_confirmed_height: 0, minimum_confirmations: 2, total: 0, amount_awaiting_finalization: 0, amount_awaiting_confirmation: 0, amount_immature: 0, amount_currently_spendable: 0, amount_locked: 0, amount_reverted: 0 }
20240112 14:21:36.591 INFO grin_gui_core::node - This is Grin version 0.1.0-alpha.2 (git v0.1.0-alpha.2), built for x86_64-pc-windows-msvc by rustc 1.75.0 (82e1608df 2023-12-21).
20240112 14:21:36.591 INFO grin_gui_core::node - Chain: Mainnet
20240112 14:21:36.591 INFO grin_gui_core::node - Accept Fee Base: 500000
20240112 14:21:36.591 INFO grin_gui_core::node - Future Time Limit: 300
20240112 14:21:36.591 INFO grin_gui_core::node - Feature: NRD kernel enabled: false
20240112 14:21:36.591 INFO grin_servers::grin::server - Starting server, genesis block: 40adad0aec27
20240112 14:21:40.139 INFO grin_servers::grin::server - Starting rest apis at: 127.0.0.1:3413
20240112 14:21:40.139 WARN grin_api::handlers - Starting HTTP Node APIs server at 127.0.0.1:3413.
20240112 14:21:40.139 WARN grin_api::handlers - HTTP Node listener started.
20240112 14:21:40.139 INFO grin_servers::grin::server - Starting dandelion monitor: 127.0.0.1:3413
20240112 14:21:40.139 WARN grin_servers::grin::server - Grin server started.
20240112 14:21:40.139 WARN grin_gui_core::node - Running Mainnet
20240112 14:21:40.140 INFO grin_servers::common::types - DandelionEpoch: next_epoch: is_stem: true (90%), relay: None
20240112 14:21:41.294 INFO grin_servers::common::adapters - Received 2 block headers from 54.39.49.39:3414
The text was updated successfully, but these errors were encountered: