Bringing back features from HFS2

[DRSDavidSoft]

This is a continuation of: rejetto/hfs2#43

notifications

it's interesting, but it's quite big in size and i'm not sure what kind of notifications hfs is supposed to show when the web is closed. Also, i'm trying to avoid native packages at the moment, as they are complicating my building process.

You're quite right, I was actually thinking more of releasing a plugin package that would make use of that. In fact, all features like this need to be a plugin since HFS 3 supports this so well.

Regarding the use case, it is used to deliver important notifications to the user, and also it can be used to notify of updates, etc.

Additionally, I'll be very interested in a plugin that would display the progress of file download in the tray bar, like how HFS 2 (R.I.P) used to do.

icon

i'm not sure if/how i can customize the exe icon. I'm generting binaries using "pkg", that doesn't offer such option and has been abandoned recently. I expect within a year to have to change tool, but i'm not on this now.

I see, I can understand why pkg is abandoned by @vercel.

I did some research and while pkg doesn't provide such feature, it is indeed possible to change the icon using methods to directly modify the resources that is included in the .exe file. I'm interested in doing this and submitting a PR, you'll need to add some development packages such as resedit. This tool is also written entirely in js so it's a) cross platform and b) doesn't execute any binaries so it can be used for cross building of HFS.

This can also be used to attach version information and company info to the resulting .exe file and can also be used with whatever build process that you may use later. I also implore you to setup GitHub Actions so that HFS can be entirely built on the cloud using CI, instead of building it on your own machine and uploading it to the releases page.

Are you fine with me submitting PRs that would address the icon issue? I would be very happy to contribute to other areas as well. I'm a bit busy now but I would love to code for the admin page as well, I believe it can improve in many areas. What stack are you using for JS/CSS/components? It isn't vanilla JS, right?

Please commit a .ico file that can be used for this (if you haven't had already). I can also make use of utilities that would generate this on the fly from the .svg/.png files included, it takes ~500KB of the development packages to do so.

Lastly please also consider a plugin or something that would utilize electron. All modern apps use something like this and one important use case for it in my opinion is the drag & drop functionality that it provides similar to HFS 2.

I also believe the icon/tray/... other areas can also be addressed with this too, however I can predict how users will react when seeing its ~90mb size 😆

In any case, the era of 2-5MB exe files is over in my opinion. We must all embrace the new era of cool web-based apps. I use things like Balena Etcher, VS Code, Discord, etc. Instead of Win32 Disk Imager, Notepad++, TeamSpeak now.

I also use Postman, Insomnia, and other apps that use Web technologies. I welcome and embrace HFS' transition from a Delphi app to a full-fledged and well-made app that would be used to serve HTTP files 👍🏻 Thanks for the development!

Originally posted by @DRSDavidSoft in rejetto/hfs2#43 (comment)

[jbrown123]

Yes, please please please restore the tons of missing functionality from v2. Alternatively, please update v2 to fix the security issue. V2 was TONS better than v3 for my use case. I'm considering risking the security threat rather than use v3, it's that much worse.

Icons for added items (directories, files, etc.)
Having different VFS files for different sites
SIMPLE management interface, like a normal app
SMALL size (2M vs 43M!!)
No way to save / display descriptions of virtual or real folders & files

No import / upgrade of existing VFS files from v2 -> v3 (huge management pain!!)

[webd90kb]

If you don't need VFS, you can try my 90kB one #679
Also filesystem's symbolic links can achieve HFS's VFS functionality. NTFS and most Linux file systems support directory symbolic links.

[rejetto]

you understand that the problem in v2 allows strangers to execute any command on your computer?
anyone is welcome to continue working on v2. I really hope someone does it.

Icons for added items (directories, files, etc.)

it seems fair

Having different VFS files for different sites

the vfs is in config, and you can have different configs. You can specify the file when starting, as a parameter at command line.

SIMPLE management interface, like a normal app

what you mean by simple?

SMALL size (2M vs 43M!!)

hfs3 is actually ~3MB. The rest is nodejs, that is bundled in the exe for convenience.
You can opt to install nodejs separately.
I can't do much about it (well, except working for 3 years just to let people save 40MB 😄)

No way to save / display descriptions of virtual or real folders & files

i guess this is the issue you recently opened

No import / upgrade of existing VFS files from v2 -> v3 (huge management pain!!)

that's not planned, not by me at least.

I will reply to the first post asap.

[DRSDavidSoft]

I will reply to the first post asap.

In the meantime, I'd like to address the huge (~40mb) node.js dependency.

Sure, it's a great thing that the dependency is included with HFS 3, but it appears people are taking it the wrong way. 😄

For reference, HFS 2 also required dependencies, those are native .DLL files that are built in with Windows. If @rejetto developed HFS 2 with C++ instead of Delphi, then it would require an additional msvcr.dll or something like that, but VC runtime is usually installed on the machine so HFS 2 wouldn't need to include any runtime DLLs really. Furthermore, the HFS 2 could be packed with UPX which would further save up to ~40% by compressing resources and executable data in the .exe file.

Now, HFS 3 is using a more modern dependency (Node.js), which is a great development platform/runtime. Even though it very well can be installed on a user's machine (and IMO it should have been), most developers simply choose to bundle it with their software. A closely related dependency is Electron.js that is used by apps like Discord, Skype, VS Code, and many other things. Do they engineer the architecture of software delivery in a way that they would all utilize a shared installation of Electron (or at least the assets)? No, they do not! The same issue is in Node.js apps as well, at least on Windows.

Now, it's super useful for HFS 3 to be built in a way that already includes the required dependency, since all you need to do is simply download the zip file and run the exe file.

However, another approach might be what .NET core apps do: they don't really include the .NET runtime. Instead, when opening an exe file, it will ask the user to install the runtime if it doesn't exist.

I believe HFS 3 can do the same thing. Keep a version of it with the bundled Node.js, and add a new version without the Node.js overhead. Then, when it's opened, it will check if a global Node.js with the required version is installed and on the PATH. If it is, HFS 3 can use that to run itself. If not, then it would prompt the user to either install the appropriate and required Node.js version, or use the one that has Node.js already built into itself. I can submit a PR for a feature like this if you agree, @rejetto.

I did something similar for a PHP app I was developing back in 2015. I didn't want to bundle a whole PHP package with my app, even though it was only ~20mb in size. So I wrote a "PHP Installer" that would exactly do this.

Additionally it might be worth it to look into better JavaScript runtimes too, such as bun which is a fine replacement to both Node.js and even Deno, another Node.js alternative.

[jbrown123]

Icons for added items (directories, files, etc.)

it seems fair

Specifically including icons for items in virtual directories like home. I have several folders that are the contents of websites I manage. It's nice to be able to pull them up locally via HFS to preview them before I upload to the production servers. I used to have them shown with a "world" icon in v2. Now they are just a generic folder icon in v3.

SIMPLE management interface, like a normal app

what you mean by simple?

As in right click on the tray icon and pick items from the menu. I find the new web interface much less intuitive (plus I miss my cute little HFS tray icon). Things are spread around in seemingly random places. For example, creating an HTTPS cert is on the internet menu, but the HTTPS config is on the options menu (plus it doesn't let you use the let's encrypt functionality from the options tab).

Additionally, there's no tray icon so you have to launch a browser, go to localhost, pick options, then pick admin panel. Lots more work.

Plus, I have this very annoying, and utterly useless, node window open the entire time the server is running.

No way to save / display descriptions of virtual or real folders & files

i guess this is the issue you recently opened

Yes, the issue with descript.ion vs. storing a comment / description in the VFS itself (please enable the latter option, I'd much prefer it in the config.yaml rather than in yet another file).

No import / upgrade of existing VFS files from v2 -> v3 (huge management pain!!)

that's not planned, not by me at least.

If you have a significant VFS it's quite a pain to recreate it in the clunky v3 interface. It took me over an hour to move things over. Is there documentation somewhere of the v2 VFS format? I might take a crack at creating an "export" tool to create the appropriate yaml file, at least for the VFS components themselves.

[rejetto]

You're quite right, I was actually thinking more of releasing a plugin package that would make use of that. In fact, all features like this need to be a plugin since HFS 3 supports this so well.

you can also consider calling a powershell script, if the module proves to be not a nice option.
i just did this kind of solution with file-icons plugin.

I'm interested in doing this and submitting a PR, you'll need to add some development packages such as resedit. This tool is also written entirely in js so it's a) cross platform and b) doesn't execute any binaries so it can be used for cross building of HFS.

sounds promising, we can explore that

Please commit a .ico file that can be used for this (if you haven't had already). I can also make use of utilities that would generate this on the fly from the .svg/.png files included, it takes ~500KB of the development packages to do so.

well, i guess it's best if we manually convert hfs-logo-icon.svg in admin/public instead.
that's supposed to change rarely.
do you want me to do that?

Lastly please also consider a plugin or something that would utilize electron.

but would the server part stay separated from the gui?
this is not hfs2, this is designed to run in the background.
also, i expect this task to take many hours.

it in my opinion is the drag & drop functionality that it provides similar to HFS 2.

are you sure electron allows that? that's not so obvious

however I can predict how users will react when seeing its ~90mb size 😆

true

In any case, the era of 2-5MB exe files is over in my opinion. We must all embrace the new era of cool web-based apps. I use things like Balena Etcher, VS Code, Discord, etc. Instead of Win32 Disk Imager, Notepad++, TeamSpeak now.

you must also consider "competition". File servers and Web servers tend to be smaller.

[webd90kb]

you must also consider "competition". File servers and Web servers tend to be smaller.

Yes, the smaller the better. :-)
How about move some of the GUI to a webui.

[rejetto]

i'm not sure i get it, the gui is already a webui.
if you mean to just split it, the gui is not the bloating part, node is.

[webd90kb]

Sorry, my bad, I havn't try HFS 3 yet, only win7 available for me now.

And this lib maybe help you to make a simple native gui https://github.com/raysan5/raygui , it's quite small too.

It's written in C, but the API and examples are simple.

[rejetto]

I believe HFS 3 can do the same thing. Keep a version of it with the bundled Node.js, and add a new version without the Node.js overhead.

ok, but you cannot make this "version" with nodejs, of course :)
i already considered having a small exe that offers tray icon, start/stop the server, and yes, possibly installs nodejs.
that could be done with delphi or others.
i was starting experimenting with it (on delphi) more than a year ago, but i suspended after having achieved very little.

i considered bun long ago when it was still too soon for some missing features, especially on Windows.
it's worth reconsidering, soon.

[rejetto]

As in right click on the tray icon and pick items from the menu.

at the moment tray icon is not a possibility, i miss the necessary commands.

Things are spread around in seemingly random places. For example, creating an HTTPS cert is on the internet menu, but the HTTPS config is on the options menu (plus it doesn't let you use the let's encrypt functionality from the options tab).

not random, there's a reason to this: both https protocol and certificate are not limited to the internet, while letsencrypt is.

Additionally, there's no tray icon so you have to launch a browser, go to localhost, pick options, then pick admin panel. Lots more work.

you can go directly to the admin-panel address, no need to pass by the options. You should consider adding it to the favorite.
Otherwise you can also enter the show-admin command inside hfs console.

Plus, I have this very annoying, and utterly useless, node window open the entire time the server is running.

there are tools that let you hide to tray apps.
if you have hfs always running and don't want to see the console, you should consider setting it up as a service.

Is there documentation somewhere of the v2 VFS format?

i don't think there's any. The only source of information is the source code, sorry.

[jbrown123]

Is there documentation somewhere of the v2 VFS format?

i don't think there's any. The only source of information is the source code, sorry.

Is the source for V2 available anywhere? If not, can you export the portion that deals with the vfs file so I can build a conversion app?

[rejetto]

the function you are searching for is at
https://github.com/rejetto/hfs2/blob/140528bac87e4ae37d444b218b5ef6ff793edc80/main.pas#L9549

[rejetto]

@jbrown123

Specifically including icons for items in virtual directories like home.

this is ready to land with 0.54

I used to have them shown with a "world" icon in v2. Now they are just a generic folder icon in v3.

i wasn't sure about any choice with the web icon, so i didn't pick any

[rejetto]

Specifically including icons for items in virtual directories like home.

version 0.54.0-alpha1 is out and includes this

[jbrown123]

Just tried 0.54.0-alpha1 and the icons work, but there's a caching issue.

Steps to reproduce:

Set the icon to a file and save that change.
Load the website in your browser. You see the image you selected as the icon.
Now change the icon to a different file.
Reload the website in the browser.
The icon is the old / original image. No amount of reloading, forced reloading, etc. will get it to change to the new image.

Now change the icon to default or to an embedded icon.
Reload the website in the browser. The selected icon is displayed.

Now change the icon to a file again (different from the file used the first time).
Reload the website in the browser. You get the original image you set for the file.
Right click the image and "inspect" to look at the icon entry.
Click the URL CSS attribute to open the image.
You see the original image you selected, not the currently selected image.
Now reload THIS view in the browser (the view of the CSS URL attribute)
You now see the new / currently selected image.
Now return to the normal website view and refresh / reload the page.
Viola! You now have the new image displayed as the icon.

BTW, completely exiting the HFS server and restarting it also does not fix this caching issue. I therefore suspect it is actually the browser caching this image and the server is simply not setting the correct expiry headers on the image to force it to be reloaded.

I'm doing this on Win 10, 64 bit using Chrome Version 126.0.6478.128 (Official Build) (64-bit)

[rejetto]

i had lost track of this report!
i finally studied it and fixed it in 0.54.0-alpha3, just released

[jbrown123]

I'm interested in what the fix was. Was it just content expiry or something more?

[rejetto]

yes, i asked the browser to check again if the file was changed, after a few seconds.
If the file isn't changed, server says "use cache"

[jbrown123]

I'm still seeing the same behavior. I changed the file associated with a directory and reloading the client doesn't change the icon. This was for an existing directory (from before I loaded alpha3).

I then tried the same sequence as in the note above using a newly added directory and I still see exactly the same behavior. It retains the original icon after changing files.

I confirmed that I'm running alpha3 (says so in the top left of the console page).

I'm switching between these two icon files:

[jbrown123]

I'm also seeing other odd behavior with alpha3. Deleting a directory from the files list in the console has no effect. It's still there. It puts up the confirmation window and behaves as though everything is fine, but it doesn't actually delete the entry. Examining the yaml file confirms it's still there. Moving a directory to another place in the hierarchy also doesn't work.

I also can't quit HFS via the console interface on the login tab. It presents the confirmation, but it never exits and I don't get the "goodbye" popup.

I have hfs1.exe (alpha1) and hfs3.exe (alpha3) in the same directory. Running hfs1 from the same dir with the same yaml works fine on all the above (moving and deleting entries, quiting). Launching hfs3 in the same dir with the same yaml does not.

[rejetto]

the fix is not retroactive, meaning that if your browser has still an old response in the cache, you will still see the problem.
Ensure you make your test after a hard reload

[jbrown123]

I did. That's what I figured would be the case and it's the reason that I created a new entry in HFS and tested with that. I'm still seeing the same behavior.

[rejetto]

ok thanks, i will test it again.
in the meantime i retired alpha3 because it has an ugly (but harmless) error message on login.
will publish alpha4 asap

[jbrown123]

I'm seeing other odd behaviors with alpha3. Moving or deleting an entry in the console doesn't work. And exiting with "quit" from the logout screen also doesn't work.

See details above at #657 (reply in thread)

[rejetto]

The problems that you are referring to are caused by the same bug for which i decided to retire alpha3: it is affecting not just login as i initially thought, but most dialogs.
I'm sorry for the inconvenience. I have automated tests for the server itself, but not for the web interface.
I already fixed it, but now I have to double check before publishing again.

[rejetto]

@jbrown123 thanks to your verifications, I've found out that the caching information wasn't enough, and it would reuse the previous file in case it has a newer timestamp on file system.
I missed this because I probably made just 1 or 2 tests, with timestamps only increasing by chance.
Now it will invalidate every time it changes path (disk) or timestamp.
alpha4 published

[jbrown123]

Still no joy on the alpha3.1 you sent me.

Switching from trs-80-icon.jpg to trs-80-icon2.jpg worked. Switching back did not.

Here's the file data from the filesystem (windows 10):
07/21/2024 03:32 PM 3,392 trs-80-icon.jpg
09/28/2024 12:05 PM 9,070 trs-80-icon2.jpg

The server is still returning a "not modified" (304) header when requesting the icon.
http://10.0.1.131/icon%20test%203.1/?get=icon -> 304

Definitely a file date/time problem because touching the 'new' file so it is later than the other causes it to update.

[jbrown123]

Testing with alpha4 does work correctly!

[rejetto]

nice