Why does index.yaml exist?

[leetNightshade]

If the comment is being stored with the page, why is it also being saved in a single index file? This seems like a waste of space and unneeded processing time.

[codeshell]

Ref: getgrav#65 (comment)

That one is for administrative and backup reasons. No comment is ever deleted from the index file. Just marked as deleted (soft delete). In case you delete pages and lose comments you didn't mean to delete. There are other (better) ways like git sync or backup cron jobs.
For development it's helpful, for production it might/could/should be disabled.

Most importantly, to use the admin panel to manage all comments (e.g. batch delete them) there is probably no way around an index file of some kind (compare it to how TNTsearch works).