Add grails.mongodb.buildIndexes and buildIndexesAsync for MongoDB index creation on startup - #16208
Open
codeconsole wants to merge 8 commits into
Open
Add grails.mongodb.buildIndexes and buildIndexesAsync for MongoDB index creation on startup#16208codeconsole wants to merge 8 commits into
codeconsole wants to merge 8 commits into
Conversation
…rtup GORM created and reconciled every index declared in a domain class mapping block each time the datastore started, with no way to switch it off. That is unwanted when deploying against live data whose indexes are managed separately: the deployment builds indexes against production collections and reconciles the index set the running application depends on. Setting grails.mongodb.buildIndexes = false leaves the server's indexes exactly as they are - no createIndex or collMod command is issued for any domain class, including for domain classes registered after startup. Persistence and querying are unaffected and continue to use whichever indexes already exist. The setting is resolved per connection like every other connection setting, so index building can be switched off globally and left on for an individual connection.
The constructors that take a MongoClient (or a MongoClientSettings.Builder) built the default connection source from a bare MongoConnectionSourceSettings, so every setting under grails.mongodb that describes how the datastore behaves - stateless, transactional, engine, flush mode, decimalType, buildIndexes - was silently left at its default. Only databaseName survived, because it is set explicitly from the mapping context. That path is not obscure: Spring Boot's MongoDB auto-configuration contributes a MongoClient bean, and MongoDbGormAutoConfiguration hands that client to GORM whenever it is present, so a Spring Boot application configuring GORM through grails.mongodb was being ignored. The settings are now bound from the configuration on this path too. The connection details in them go unused, as the client is supplied already connected.
… thread MongoDB answers a createIndex command only once the index has been built, and the datastore builds every declared index from its constructor, so an application waits at startup for all of them in turn. On an empty collection that is instant; on a large existing collection it is minutes of a deployment spent waiting. Setting grails.mongodb.buildIndexesAsync = true hands the startup build to a single daemon thread per connection, named gorm-mongo-index-build-<connection>, and returns immediately. Indexes are still built one at a time rather than all at once against the server. Two consequences are documented rather than hidden: a query issued before its index exists is served without it (a unique index likewise constrains nothing until the build completes), and a build failure can no longer fail startup, so it is logged at error level instead. The setting covers the startup build only - a domain class registered later is still indexed on the registering thread, which keeps the tenant context that path can depend on.
A successful index build said nothing at all: only conflicts, TTL updates and
failures were logged, so there was no way to tell how much of startup went on
building indexes, and with buildIndexesAsync no signal that the background
build had finished.
The build now logs one summary when it completes:
Applied 7 index declaration(s) from 3 domain class(es) to database [myDb] in 412ms
The count is declarations applied rather than indexes built, because
createIndex is idempotent and the server accepts one that already exists
without doing any work - claiming otherwise would be wrong on every restart
after the first. A build with failures is logged at warn and reports how many.
Each index also logs its own elapsed time at debug, which is how to find the
one slow index behind a slow summary.
The tests now run against logback rather than the no-op SLF4J binding, as the
sibling data modules do, so that a test can assert on what was logged; the
root logger is quietened to WARN so this does not add noise to the build.
The previous summary counted declarations applied and said, in as many words,
that it could not tell whether any of them did work. That was wrong about the
server: createIndexes answers with numIndexesBefore, numIndexesAfter and a
"note: all indexes already exist", and it is the driver's createIndex helper
that discards the response and returns only the index name.
Rather than hand-roll the command to read that response - the mapping from
IndexOptions to an index specification is not worth reimplementing - the build
lists the indexes a collection already has, once per collection, and reports:
Index build for database [myDb] finished in 412ms: 2 created, 5 already present, from 3 domain class(es)
That split is what makes the elapsed time mean something. Measured against
MongoDB 7.0 with 200k documents: creating an index took 172ms on the calling
thread, re-issuing the same declaration 1.2ms. A restart that changed no
mappings reports everything as already present, so a summary reporting created
indexes is the one that accounts for a slow start.
The listing is lazy, so an entity declaring no indexes costs no round trip, and
it replaces the listing the conflict path used to make for itself. If it cannot
be read the build still runs and the summary falls back to reporting how many
declarations were applied.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## 8.0.x #16208 +/- ##
==================================================
+ Coverage 54.1238% 54.7703% +0.6465%
- Complexity 20307 20456 +149
==================================================
Files 2107 2101 -6
Lines 101144 101062 -82
Branches 17921 17921
==================================================
+ Hits 54743 55352 +609
+ Misses 38595 37845 -750
- Partials 7806 7865 +59
🚀 New features to boost your workflow:
|
…n.groovy The configuration examples were written only in the application.groovy DSL, which reads as though the Groovy form were the only one; application.yml is what a generated Grails application actually ships with. Both forms are now shown, and the settings are noted as being read by name: they are not relaxed-bound the way Spring Boot's own properties are, so a kebab-case spelling is ignored and silently leaves the default in place.
Codecov put the patch at 75%, and what it was missing was every path where something goes wrong - which is where behaviour that has been claimed in the documentation most needs pinning. Four of them are now covered: - a declaration the server refuses, and one whose conflict cannot be reconciled, are counted and reported: the build carries on, the summary is raised to WARN and says how many failed - a background build that fails logs the failure at error level, since nothing waits on that thread to notice it - a background build interrupted by shutdown is logged as the shutdown it is, not as a failure - a role that may create an index but not list the existing ones still gets its indexes, with the summary falling back to how many declarations it applied The last two need conditions a server will not produce on demand, so the driver is wrapped in a proxy that makes one operation fail or block. Only that one method is intercepted; everything else is the real driver talking to the real server, and the datastore under test is the real one built through its public constructor. Patch coverage is now 97.8%. What remains is two error branches in conflict reconciliation that predate this change and are not reachable without a server that contradicts itself.
✅ All tests passed ✅Test SummaryCI / Functional Tests (Java 21, indy=true) > :grails-test-examples-scaffolding:integrationTest
🏷️ Commit: b873801 Test FailuresUserControllerSpec > User list (:grails-test-examples-scaffolding:integrationTest in CI / Functional Tests (Java 21, indy=true) | Attempt 1/2)Learn more about TestLens at testlens.app/docs. |
codeconsole
assigned
borinquenkid,
sbglasius,
jdaugherty and
jamesfredley and unassigned
borinquenkid,
sbglasius,
jdaugherty and
jamesfredley
codeconsole
requested review from
borinquenkid,
jamesfredley,
jdaugherty and
sbglasius
September 2, 2026 03:39
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two settings for the MongoDB index build that runs when the datastore starts, plus the reporting to make either decision an informed one.
All of these are runtime configuration, set in
grails-app/conf/application.yml(orapplication.groovy) — nothing inbuild.gradle.grails.mongodb.buildIndexesGORM creates and reconciles every index declared in a mapping block each time the datastore starts, with no way to switch it off. That is unwanted when deploying against live data whose indexes are managed separately, by a DBA or a migration step: the deployment builds indexes against production collections and reconciles the index set the running application depends on.
With the setting off, no
createIndexorcollModis issued for any domain class and the indexes on the server are left exactly as they are. Persistence and querying are unaffected and keep using whichever indexes exist. It also covers domain classes registered after startup, and resolves per connection like every other connection setting, so index building can be switched off globally and left on for one connection:A collection that has never started up with index building enabled will have no declared indexes at all, so this is for environments where the indexes are already in place or are applied by other means. It governs only what GORM derives from mapping blocks; an explicit
createIndexin application code is unaffected.grails.mongodb.buildIndexesAsyncMongoDB answers a
createIndexonly once the index has been built, so by default startup waits for every declared index in turn. Measured against MongoDB 7.0 with 200k documents: 172ms to create one index on the calling thread, 1.2ms to re-issue the same declaration.background: truehas been ignored since MongoDB 4.2 and costs the same.With this enabled the startup build runs on one daemon thread per connection, named
gorm-mongo-index-build-<connection>, and startup continues without waiting. Indexes are still built one at a time rather than all at once against the server.Worth planning for:
uniqueindex likewise constrains nothing until the build finishes.Index build reporting
A successful build previously logged nothing at all — only conflicts, TTL updates and failures — so there was no way to tell how much of startup went on indexes, and with the async build no signal that it had finished. It now logs one summary:
The created/already-present split is what makes the elapsed time interpretable: a restart that changed no mappings reports everything as already present and costs milliseconds, so a summary reporting indexes created is the one that accounts for a slow start. A build with failures is logged at
WARNand reports how many. Each index also logs its own elapsed time atDEBUGunderorg.grails.datastore.mapping.core, which is how to find the single slow index behind a slow summary.createIndexesreportsnumIndexesBefore/numIndexesAfter, but the driver'screateIndexhelper discards the response and returns only the index name. Rather than hand-roll the command — theIndexOptionsto index-specification mapping is not worth reimplementing — the build lists the indexes a collection already has. That listing is lazy, so an entity declaring no indexes costs no round trip, and it replaces the listing the conflict path used to make for itself. If it cannot be read, the build still runs and the summary falls back to reporting how many declarations were applied.Settings were ignored when the application supplies its own
MongoClientFound while testing the above. The constructors taking a
MongoClient(or aMongoClientSettings.Builder) built the default connection source from a bareMongoConnectionSourceSettings, so everygrails.mongodbsetting describing how the datastore behaves —stateless,transactional,engine, flush mode,decimalType, and nowbuildIndexes— was silently left at its default. OnlydatabaseNamesurvived, because it is set explicitly from the mapping context.That path is not obscure: Spring Boot's MongoDB auto-configuration contributes a
MongoClientbean andMongoDbGormAutoConfigurationhands that client to GORM whenever one is present, so a Spring Boot application configuring GORM throughgrails.mongodbwas being ignored. The settings are now bound from the configuration on that path too; the connection details in them go unused, as the client arrives already connected.This is a behaviour change for anyone who had been running on defaults without knowing it — a
stateless = trueortransactional = truethat was previously discarded now takes effect.Notes
grails-data-mongodb/docsunder Querying Indexing and Advanced Configuration, with the 8.0 release notes updated. Those pages showed configuration only in theapplication.groovyDSL, so they now show theapplication.ymlform as well, and note that these settings are read by name — they are not relaxed-bound like Spring Boot's own properties, so a kebab-case spelling such asbuild-indexesis ignored and leaves the default in place.grails-data-neo4j/corealready does, so a test can assert on what was logged. The root logger is pinned toWARNby alogback-test.xml, and the build output is unchanged.